A big data intelligent classification method and system

Abstract

The present invention relates to the technical field of data classification, and specifically to a big data intelligent classification method and system. The method comprises: obtaining an access correlation coefficient of any two modules based on the number of visits and access time interval of each user to any two modules in software, combined with the software registration duration of each user and the number of software visits and access duration; obtaining the data content correlation of any two modules based on the data change relationship between any two modules; fusing the access correlation coefficient and the data content correlation to obtain the comprehensive correlation of any two modules; and classifying the data of each module in the software based on the comprehensive correlation of all two modules in the software, which can improve the accuracy and reliability of data classification, can accurately analyze the correlation between the various modules of the software, and is beneficial for software designers to analyze the collaborative performance between the various modules, thereby timely optimizing the functions of each module of the software and improving the user experience of the software.

Description

Technical Field

[0001] The present invention relates to the technical field of data classification, and in particular to a method and system for intelligent classification of big data. Background Art

[0002] With advancements in internet technology, AI (artificial intelligence) computing power continues to rise. However, this is accompanied by an explosive growth in the speed and volume of data generated. For example, IoT devices, social media, and surveillance systems continuously generate massive amounts of data. Traditional manual classification or simple rule-based engines are no longer able to handle petabytes of data, requiring automated, intelligent classification technologies for efficient processing. Therefore, intelligent classification of big data is a key trend in future development.

[0003] Software usually includes multiple modules, such as user information module, software log module, software function module, etc. The division of functional modules here is mainly based on the different functions of the software. For example, a shopping software includes order module, shopping cart module, product browsing module, etc. Each module in the software usually involves various types of data information, and general data classification is mainly based on different modules, dividing the data of the same module into one category, and dividing the data of different modules into different categories. This classification method is relatively rough and simple, with poor classification accuracy and reliability. It cannot analyze the correlation between the various modules of the software, which is not conducive to software designers analyzing the collaborative performance between the various modules. As a result, it is impossible to optimize the functions of the various modules of the software in a timely manner, affecting the user's experience of the software. Summary of the Invention

[0004] In order to solve the technical problem of poor reliability of existing software data classification methods, the purpose of the present invention is to provide a big data intelligent classification method and system. The technical solutions adopted are as follows:

[0005] In a first aspect of the present invention, a method for intelligent classification of big data is provided, comprising:

[0006] The access correlation coefficient of any two modules in the software is obtained by combining the number of visits and access time interval of each user to the two modules in the software with the software registration duration, the number of visits and the access time of each user;

[0007] Based on the data change relationship between any two modules, obtaining the data content correlation between any two modules;

[0008] The access correlation coefficient and the data content correlation are integrated to obtain the comprehensive correlation between any two modules;

[0009] According to the comprehensive correlation between any two modules in the software, the data of each module in the software is classified.

[0010] In an exemplary embodiment, the process of obtaining the access correlation coefficient includes:

[0011] Determine the correlation performance of the two modules for each user based on the number of visits to the two modules and the correlation therebetween, and in combination with the time interval between visits to the two modules;

[0012] Based on each user's software registration time, as well as the number of times and duration of each user's software access, the user's access influence weight on the software is obtained;

[0013] Based on the access influence weight of each user, a weighted sum is performed on the association performance of any two modules of each user to obtain an access association coefficient of the any two modules.

[0014] In an exemplary embodiment, the process of obtaining the associated representation includes:

[0015] Obtaining a proportion of consecutive visits to any two modules among the number of visits to any two modules, where the consecutive visits refer to a time interval between visits to any two modules being less than a preset time interval threshold;

[0016] The association performance of any two modules is obtained based on the proportion of consecutive visits, the difference in visits and the average time interval between visits; the difference in visits is the difference in the number of visits of the user to the any two modules, and the average time interval between visits is the average time interval between the user's visits to the any two modules; the association performance is directly proportional to the proportion of consecutive visits, and inversely proportional to the difference in visits and the average time interval between visits.

[0017] In an exemplary embodiment, the process of obtaining the access influence weight includes:

[0018] The user's access frequency is obtained based on the number of times the user accesses the software and the duration of the accesses, as well as the total number of days in which all the accesses occurred. The access frequency is proportional to the number of times the user accesses the software and the duration of the accesses, and inversely proportional to the total number of days.

[0019] Obtaining a user's access reference value based on the access frequency and software registration duration;

[0020] The access reference values ​​of all users are weighted and normalized to obtain the access influence weight of each user on the software.

[0021] In an exemplary embodiment, the process of obtaining data content relevance includes:

[0022] determining whether new data is generated in a second data type in a second module within a preset time period after a change in first data in a first data type in a first module of a user; the first module and the second module constitute any two modules, the first data type is any data type in the first module, the first data is any piece of data in the first data type, and the second data type is any data type in the second module;

[0023] If new data is generated, the data content correlation between the user's first data type in the first module and the second data type in the second module is obtained according to the judgment result and the time interval for generating the new data.

[0024] In an exemplary embodiment, obtaining, based on the judgment result and the time interval for generating new data, the data content relevance of the user for the first data type in the first module and the second data type in the second module includes:

[0025] Determine a correlation feature between first data of a first data type in the first module and a second data type in the second module; wherein if new data is generated, the correlation feature is inversely proportional to a time interval between the generation of the new data; and if no new data is generated, the correlation feature is 0;

[0026] Calculate the average value of the relevant features corresponding to all changed data of the first data type in the first module by the user, and obtain the data content correlation between the first data type in the first module and the second data type in the second module by the user.

[0027] In an exemplary embodiment, the process of obtaining the comprehensive correlation includes:

[0028] According to preset weights, the access association coefficients of the first module and the second module and the data content correlations of the first data type in the first module and the second data type in the second module are weighted and summed to obtain the comprehensive correlation between the first data type in the first module and the second data type in the second module.

[0029] In an exemplary embodiment, classifying the data of each module in the software according to the comprehensive correlation between any two modules in the software includes:

[0030] Each comprehensive correlation is compared with a preset comprehensive correlation threshold, and corresponding data types of modules corresponding to comprehensive correlations greater than the preset comprehensive correlation threshold are classified into one category.

[0031] In an exemplary embodiment, the big data intelligent classification method further includes:

[0032] The data of each module in the classified software is stored in different storage areas in the storage device.

[0033] In a second aspect of the present invention, a big data intelligent classification system is provided, comprising: a memory and a processor; the memory is connected to the processor; the memory is used to store program instructions; and the processor is used to implement the above-mentioned big data intelligent classification method when the program instructions are executed.

[0034] The present invention has the following beneficial effects: the present invention first determines the access correlation coefficient of any two modules in terms of user access based on the user's access to any two modules in the software, and then obtains the data content correlation of any two modules from the aspect of the correlation between the data content changes based on the data change relationship in any two modules, thereby combining the access correlation coefficient and the data content correlation to obtain the comprehensive correlation of any two modules, thereby classifying the data of each module in the software based on the comprehensive correlation of all any two modules in the software. The present invention simultaneously considers the user's access to the software and the data content correlation of the software itself, realizes the classification of the data of each module in the software, can improve the accuracy and reliability of data classification, can accurately analyze the correlation between the modules of the software, is beneficial for software designers to analyze the collaborative performance between the modules, thereby timely optimizing the functions of the modules of the software and improving the user's experience of the software. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of a big data intelligent classification method provided by one embodiment of the present invention;

[0036] Figure 2 is a flow chart of obtaining an access correlation coefficient provided by one embodiment of the present invention;

[0037] Figure 3 is a flowchart of obtaining associated representations provided by one embodiment of the present invention;

[0038] Figure 4 This is a flow chart for obtaining access influence weights provided by one embodiment of the present invention;

[0039] Figure 5 This is a flow chart for obtaining data content relevance provided by one embodiment of the present invention;

[0040] Figure 6 It is a flowchart of a specific implementation process of step S2-2 provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] To further illustrate the technical means and effects employed by the present invention to achieve its intended objectives, the following detailed description of the specific embodiments, structures, features, and effects of the present invention is provided in conjunction with the accompanying drawings and preferred embodiments. In the following description, references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this invention pertains. The data and information collected in this application were obtained with full consent and authorization.

[0043] This invention primarily uses intelligent data classification based on the vast amount of data collected. The data primarily originates from various software modules. Software modules typically involve various types of data, and while data classification is typically based on modules, the various types of data generated by user usage within the software often exhibit certain correlations, enabling user behavior decoding and software module optimization.

[0044] In the process of classifying the data corresponding to each software module, different data types may have different correlation performances, which generally show that: the correlation performance between data types corresponding to modules with higher correlation is better; the correlation performance between data types corresponding to modules with lower correlation is poor, which helps to analyze the correlation performance between data. Therefore, based on the data corresponding to each software module, a specific analysis is conducted. First, the access correlation coefficient between modules is determined, and then the specific information of each type of data corresponding to the module is combined for analysis to obtain the correlation performance between each type of data, thereby completing the intelligent classification of data in each software module. Based on this, if Figure 1 As shown, this embodiment provides a big data intelligent classification method, including:

[0045] Step S1: Based on the number of visits and access time intervals of each user to any two modules in the software, combined with the software registration duration, number of visits and access time of each user, obtain the access correlation coefficient of any two modules;

[0046] Step S2: Based on the data change relationship between any two modules, obtain the data content correlation between any two modules;

[0047] Step S3: Fusing the access correlation coefficient and the data content correlation to obtain the comprehensive correlation between any two modules;

[0048] Step S4: Classify the data of each module in the software according to the comprehensive correlation between any two modules in the software.

[0049] The specific implementation process of each step is described below in conjunction with the accompanying drawings.

[0050] Step S1: Based on the number of visits and access time interval of each user to any two modules in the software, combined with the software registration duration of each user and the number of software visits and access time, the access correlation coefficient of any two modules is obtained.

[0051] Different software modules have different functions and provide different user experiences, resulting in varying user access patterns. User access patterns can, to a certain extent, reveal the connections between these modules. When the number of visits to two modules and the time interval between visits are relatively close, this indicates a close relationship between the two modules. Therefore, the access correlation coefficient between any two modules is calculated by combining the number of visits and time interval between visits by each user, along with the length of time each user has registered for the software and the number and duration of their visits.

[0052] In an exemplary embodiment, Figure 2 As shown in the figure, a specific process of obtaining the access correlation coefficient is given:

[0053] Step S1-1: Determine the correlation performance of any two modules of each user based on the number of visits to any two modules of each user and the correlation therebetween, and in combination with the visit time interval between any two modules.

[0054] This embodiment predefines a monitoring period, the length of which is set based on actual needs, such as one month. By analyzing the data from each software module within this monitoring period, data classification processing is achieved for each module. Furthermore, the software is typically accessed multiple times by multiple users during this monitoring period. First, for each user (e.g., user ID), access data for the software is obtained. Specifically, data on access to the software as a whole and access data for each module within the software is obtained. Two modules are selected from the various modules within the software for analysis.

[0055] For any user, each time the user accesses a module in the software, corresponding access information will be generated, including the time of each access to the module, thereby recording the number of accesses within the monitoring period. Based on the number of accesses to any two modules of the user and the correlation between them, and combined with the access time interval of any two modules, the correlation performance of any two modules of the user is determined. The correlation performance represents the degree of correlation between the user's access to any two modules. The higher the degree of correlation, the more similar the user's access to the two modules is. In an exemplary embodiment, if Figure 3 As shown, a specific process of obtaining the associated representation is given below:

[0056] Step S1-1-1: Obtain the proportion of consecutive access times of any two modules in the number of access times of any two modules.

[0057] For ease of description, it is assumed that the first module and the second module constitute any two modules. The number of times a user accesses the first module and the number of times a user accesses the second module within a monitoring time period are obtained.

[0058] A time interval threshold is preset to determine whether the time interval between accessing the first and second modules is short. If the time interval between accessing the first and second modules is less than the preset time interval threshold, the access is determined to be continuous, and the number of consecutive accesses by the user to the first and second modules during the monitoring period is counted as the number of continuous accesses. It should be understood that the specific value of the preset time interval threshold can be flexibly set by the implementer, such as 1 minute, provided that the above determination requirements are met. The time interval between accessing the first and second modules is the time interval between the end of accessing one module and the start of accessing the other.

[0059] Based on the number of times the user accesses the first module, the number of times the user accesses the second module, and the number of consecutive visits during the monitoring period, a percentage of consecutive visits of the user to the two modules during the monitoring period is obtained. In an exemplary embodiment, the sum of the number of times the user accesses the first module and the number of times the user accesses the second module during the monitoring period is calculated, and the ratio of the number of consecutive visits to the sum is calculated as the percentage of consecutive visits of the user to the two modules during the monitoring period.

[0060] The greater the proportion of consecutive visits, the more often users immediately visit another module after visiting one during the monitoring period. The more frequent these visits, the stronger the correlation between the two modules. Therefore, the correlation is directly proportional to the proportion of consecutive visits.

[0061] Step S1-1-2: Obtain the correlation performance of any two modules based on the proportion of consecutive access times, the difference in access times, and the average access time interval.

[0062] The difference between the number of visits the user made to the first module and the number of visits the user made to the second module during the monitoring period is obtained, specifically the absolute value of the difference, as the difference in the number of visits the user made to the first module and the second module during the monitoring period. The smaller the difference in the number of visits, the closer the number of visits the user made to the first module and the second module during the monitoring period, and thus the stronger the association between the two modules. Therefore, the association is inversely proportional to the difference in the number of visits.

[0063] Obtain the time intervals between the user's visits to the first and second modules during the monitoring period, thereby obtaining multiple visit time intervals. Calculate the average of these multiple visit time intervals during the monitoring period as the average visit time interval for the user's visits to the two modules. The smaller the average visit time interval, the shorter the time interval between the user's visits to the two modules during the monitoring period, and thus, the stronger the association between the two modules. Therefore, the association is inversely proportional to the average visit time interval.

[0064] During the monitoring period, for the first module and the second module, the user's correlation performance for the two modules is obtained based on the proportion of consecutive visits, the difference in visits, and the average time interval between visits. In an exemplary embodiment, a specific quantitative method for correlation performance is given as follows:

[0065] ;

[0066] in, Indicates the User for the modules and The correlation performance of the modules, modules and A module represents any two modules; Indicates the User for the modules and The proportion of consecutive visits to each module; Indicates the User for the modules and The difference in the number of visits to each module; Indicates the User for the modules and The average time interval between visits to each module; Represents an exponential function with the natural constant e as its base.

[0067] Represents the normalization function. The normalization method here can be: Get all users for the modules and modules The maximum and minimum values ​​of the first users Perform normalization.

[0068] Step S1-2: Obtain the access influence weight of each user on the software based on the software registration time of each user, the number of times each user accesses the software, and the access duration.

[0069] Since different users have different overall access conditions to the software, the access influence weight of each user on the software is obtained based on the software registration time of each user, as well as the number of times and duration of each user's software access. The access influence weight of each user is used as the contribution of each user to the correlation performance of any two modules. Therefore, the contribution of different users to the correlation performance is also different.

[0070] In an exemplary embodiment, Figure 4 As shown, a specific process of obtaining the access influence weight is given as follows:

[0071] Step S1-2-1: Obtain the user's access frequency based on the number of times the user accesses the software, the duration of the access, and the total number of days for all accesses.

[0072] Obtain the number of software accesses by a user during the monitoring period. A software access refers to a user entering the software. For example, a user taps the app (application) icon on the phone screen and then enters the app's start interface. This counts as a software access, thus counting the number of software accesses during the monitoring period. The more software accesses a user has, the more frequently they access the software. Therefore, the higher the user's access frequency, the higher the access frequency, and the access frequency is proportional to the number of software accesses.

[0073] For any software access, the time interval from clicking the software app icon on the phone screen to exiting the app is counted as a visit duration. The total duration of each access is then calculated and summed to form the total duration of the user's software access during the monitoring period. The longer the user's total software access duration, the more frequent the software access. Therefore, the higher the user's access frequency, the higher the access frequency is, and the access frequency is directly proportional to the user's total software access duration.

[0074] Obtain the total number of days during which the user accessed the software during the monitoring period. This means obtaining the date of each user's software access during the monitoring period. Then, summing up all the different dates during the monitoring period, the total number of days is used as the total number of days. For example, suppose the user accessed the software 10 times during the monitoring period, with three accesses on the first day, four on the third day, two on the fourth day, and one on the sixth day. Therefore, the different dates during the monitoring period are: the first day, the third day, the fourth day, and the sixth day. The total number of days during which the user accessed the software is four. Therefore, the longer the total number of days during which the user accessed the software, the more dispersed and less concentrated the user's access to the software, indicating less frequent access to the software. Consequently, the lower the user's access frequency, and the access frequency is inversely proportional to the total number of days. It should be understood that the number of software accesses by the user is greater than or equal to the total number of days during which the user accessed the software. In an exemplary embodiment, a specific quantitative method for access frequency is given as follows: the product of the number of software accesses and the total access duration of the software is calculated, and the ratio of the product to the total number of days is calculated, and the result obtained is the access frequency.

[0075] Step S1-2-2: Obtain the user's access reference value based on the access frequency and software registration time.

[0076] Obtain the user's software registration duration, that is, the duration from successful software registration to the end of the monitoring period. The more frequent the user's access, the higher the user's access reference value to the software; the longer the software registration duration, the higher the user's access reference value to the software. Therefore, the access reference value is proportional to both the access frequency and the software registration duration. In an exemplary embodiment, a specific method for quantifying the access reference value is given as follows:

[0077] ;

[0078] in, Indicates the The reference value of a user's visit, Indicates the The number of software accesses by users who access the software, Indicates the The total access time of the software by each user, Indicates the The total number of days that all users accessed the software, Indicates the The software registration duration of each user.

[0079] Indicates the The frequency of access by a user.

[0080] Therefore, the longer the user has registered for the software, the more times he accesses the software, and the longer the access time, the more frequently the user uses the software and the more familiar he is with the various modules in the software. The data information generated by the user has a higher reference value, and the user's access reference value is higher.

[0081] Step S1-2-3: Normalize the weights of the access reference values ​​of all users to obtain the access influence weight of each user on the software.

[0082] According to step S1-2-2, the access reference value of each user can be obtained. The access reference values ​​of all users are weighted and normalized to obtain the access influence weight of each user on the software. The calculation formula is as follows:

[0083] ;

[0084] in, Indicates the The access influence weight of each user, I represents the number of users.

[0085] Step S1-3: Based on the access influence weight of each user, weighted sum is performed on the correlation performance of any two modules of each user to obtain the access correlation coefficient of any two modules.

[0086] According to steps S1-1 and S1-2, the correlation performance of any two modules for each user and the access influence weight of each user are obtained. Since a single user cannot accurately represent the correlation between two modules, when analyzing the correlation between modules by combining multiple users, the correlation coefficient between modules cannot be represented by a simple average due to the different access conditions of different users. Therefore, based on the access influence weight of each user, the correlation performance of any two modules for each user is weighted and summed to obtain the access correlation coefficient of any two modules. The calculation formula is as follows:

[0087] ;

[0088] in, Indicates the modules and The access correlation coefficient of each module.

[0089] At this point, the access correlation coefficient between any two modules is obtained.

[0090] Step S2: Based on the data change relationship between any two modules, the data content correlation between any two modules is obtained.

[0091] Step S1 obtains the access correlation coefficient between any two modules of the software based on the user's access to the software. A high access correlation coefficient between two modules indicates that the user is accustomed to continuously using certain functions of these two modules, meaning that the data of certain data types generated by the two modules are highly correlated. Data between modules with high access correlation coefficients is likely to have high correlation; conversely, data between modules with low access correlation coefficients is likely to have low correlation.

[0092] This step obtains the data content correlation of the two modules from the data change relationship between the two modules when the user accesses the software, and further analyzes the correlation between the two modules, thereby improving the accuracy of software data classification.

[0093] In an exemplary embodiment, Figure 5 As shown, a specific process of obtaining data content relevance is given below:

[0094] Step S2-1: Determine whether new data is generated in the second data type in the second module within a preset time period after the first data in the first data type in the first module of the user changes.

[0095] Typically, each module includes multiple data types, and each data type includes multiple pieces of data information. For ease of explanation, the first data type is set to any data type in the first module, and the first data is set to any piece of data in the first data type. At the same time, the second data type is set to any data type in the second module.

[0096] The data of each data type in the first module and the second module are monitored in real time. After the first data of the first data type in the first module of the user changes, it is determined whether new data is generated in the second data type in the second module within a preset time period. The first data change generally refers to a change in the data type of the first data, that is, a change from the first data type to another data type. The length of the preset time period is set according to actual needs. In this embodiment, the length of the preset time period is relatively short to reflect the close relationship between data changes and the generation of new data. In an exemplary embodiment, the preset time period is 1 second, that is, it is determined whether new data is generated in the second data type in the second module within 1 second.

[0097] If new data is generated, it means that the first data type in the user's first module and the second data type in the second module are associated with each other; if no new data is generated, it means that the first data type in the user's first module and the second data type in the second module are not associated with each other.

[0098] Step S2-2: If new data is generated, then the data content correlation between the user's first data type in the first module and the second data type in the second module is obtained based on the judgment result and the time interval for generating the new data.

[0099] If new data is generated, the time interval between the change in the first data type in the first module of the user and the generation of new data in the second data type in the second module is obtained. Then, if new data is generated, the data content correlation between the first data type in the first module and the second data type in the second module of the user is obtained based on the judgment result and the time interval between the generation of the new data. The shorter the time interval between the generation of new data, the stronger the data content correlation between the first data type in the first module and the second data type in the second module.

[0100] Data content relevance indicates whether the data type in another module changes when the corresponding data of a certain data type in one module changes, and the corresponding correlation of the changes.

[0101] In an exemplary embodiment, Figure 6 As shown, a specific implementation process of step S2-2 is given as follows:

[0102] Step S2-2-1: Determine the relevant features of the user's first data of the first data type in the first module and the second data type in the second module.

[0103] The relevant characteristics of each data of the first data type are determined based on whether new data is generated in the second data type in the second module within a preset time period after each data of the first data type in the first module changes. Taking the first data of the first data type as an example, if new data is generated in the second data type in the second module within a preset time period after the first data of the first data type in the first module changes, then the relevant characteristics of the first data are a positive value, and the relevant characteristics are inversely proportional to the time interval between the generation of new data, that is, the longer the time interval between the generation of new data, the smaller the value of the relevant characteristics; if no new data is generated in the second data type in the second module within a preset time period after the first data of the first data type in the first module changes, then the relevant characteristics of the first data are 0. Therefore, the relevant characteristics corresponding to the generation of new data are greater than the relevant characteristics corresponding to the absence of new data generation.

[0104] In one exemplary embodiment, the correlation feature is inversely proportional to the time interval between new data generation by performing negative correlation normalization on the time interval between new data generation, and the result is the correlation feature. For example, if the preset time period is 1 second, the time interval between new data generation must be greater than 0 and less than 1. Therefore, the method for performing negative correlation normalization on the time interval between new data generation can be to subtract the time interval between new data generation from 1, and the difference obtained is the correlation feature.

[0105] By adopting the above process, the user can obtain the relevant features of each changed data of the first data type in the first module and the second data type in the second module.

[0106] Step S2-2-2: Calculate the average value of the relevant features corresponding to all changed data in the first data type in the first module, and obtain the data content correlation between the first data type in the first module and the second data type in the second module.

[0107] The average value of the relevant features corresponding to all changes in the first data type in the first module is calculated. The calculated result is the data content correlation between the first data type in the first module and the second data type in the second module. It should be understood that when calculating the average value, relevant features with a value of 0 are also included in the average value calculation. The average value is calculated for all changes in the first data type in the first module.

[0108] The user characterizes the degree of correlation between the data contents of the first data type in the first module and the second data type in the second module based on the data content correlation between the first data type in the first module and the second data type in the second module. The shorter the time interval for generating new data, the closer the connection between the data contents of the first data type in the first module and the second data type in the second module, that is, the higher the data content correlation, the more relevant the data contents.

[0109] Using the above process, the data content correlation between each data type in the first module and each data type in the second module for each user is obtained. It should be understood that the above analysis is an analysis of the data content correlation between each data type in the first module and each data type in the second module for the same user.

[0110] Then, taking the first data type in the first module and the second data type in the second module as the analysis objects, obtain the data content correlation of each user for the first data type in the first module and the second data type in the second module, and then calculate the average value of the data content correlation of each user for the first data type in the first module and the second data type in the second module, and obtain the data content correlation of the first data type in the first module and the second data type in the second module for all users. The data content correlations of the first data type in the first module and the second data type in the second module mentioned below are all data content correlations of the first data type in the first module and the second data type in the second module for all users.

[0111] Step S3: Fusing the access correlation coefficient and the data content correlation to obtain the comprehensive correlation between any two modules.

[0112] The access correlation coefficient measures the correlation between the first and second modules based on user access to the modules. Data content correlation measures the correlation between the data content of the first data type in the first module and the second data type in the second module. By combining the access correlation coefficient and data content correlation, we can obtain the overall correlation between the two modules.

[0113] In an exemplary embodiment, according to preset weights, the access correlation coefficients of the first module and the second module and the data content correlations of the first data type in the first module and the second data type in the second module are weighted and summed to obtain the comprehensive correlation of the first data type in the first module and the second data type in the second module. Among them, the weight coefficients of the access correlation coefficient and the data content correlation are set according to actual needs. Since this embodiment is dominated by behavioral correlation (that is, the access correlation coefficient between modules is the main factor), supplementary information is provided in combination with data content (that is, the data content correlation between data types is a supplementary factor) to obtain the comprehensive correlation performance between data types. Then, the weight of the access correlation coefficient can be greater than the weight of the data content correlation, for example: the weight of the access correlation coefficient is 0.6, and the weight coefficient of the data content correlation is 0.4.

[0114] Using the above process, the comprehensive correlation between each data type in the first module and each data type in the second module is obtained. Furthermore, the comprehensive correlation between each data type in each module and each data type in other modules in the software is obtained, that is, the comprehensive correlation between any two data types in different modules.

[0115] Step S4: Classify the data of each module in the software according to the comprehensive correlation between any two modules in the software.

[0116] The greater the comprehensive correlation, the more closely related the two data types in the corresponding two modules are, and the more likely they are to be classified into the same category. In an exemplary embodiment, a comprehensive correlation threshold is preset to determine whether the comprehensive correlation between the two data types in the two modules is strong, that is, whether they can be classified into the same category. The specific value of the comprehensive correlation threshold is set by the implementer based on actual conditions. In this embodiment, it is set to 0.6.

[0117] The comprehensive correlation between each data type in each module and each data type in each other module is compared with a preset comprehensive correlation threshold. For example, the comprehensive correlation between the first data type in the first module and the second data type in the second module is used as an example. If the comprehensive correlation between the first data type in the first module and the second data type in the second module is greater than the preset comprehensive correlation threshold, it indicates that the first data type in the first module and the second data type in the second module have a strong correlation, and the first data type in the first module and the second data type in the second module are classified into the same category. Specifically, the first data type in the first module and the second data type in the second module are path-connected. Using this method, it is determined whether each data type in each module is classified into the same category with each data type in each other module. All data types are then path-linked, and all data types connected along the same path are classified into the same category. For example, if the first data type in the first module and the second data type in the second module are path-connected, and the first data type in the first module and the third data type in the third module are also path-connected, then the first data type in the first module, the second data type in the second module, and the third data type in the third module are considered to be of the same category, completing data classification between different modules.

[0118] The present invention can improve the accuracy and reliability of data classification, can accurately analyze the correlation between software modules, and based on the data types classified into the same category, it is helpful for software designers to analyze the collaborative performance between modules. Software designers can decode user behavior to discover the functions that users often use continuously in the software modules, help optimize the software modules, and enhance the user experience of the software.

[0119] In the follow-up, the big data intelligent classification method provided by this embodiment also includes: dividing a plurality of different storage areas in the storage device, and storing different types of data in different storage areas in the storage device, that is, storing software data of the same data type in the same storage area, and storing software data of different data types in different storage areas, to achieve targeted storage and facilitate later data calls.

[0120] This embodiment also provides a big data intelligent classification system, including: a memory and a processor; the memory is connected to the processor, and the memory is used to store program instructions; the processor is used to implement the steps in the above-mentioned big data intelligent classification method embodiment when the program instructions are executed.

[0121] In an exemplary embodiment, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the above-mentioned big data intelligent classification method embodiment.

[0122] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0123] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A big data intelligent classification method, characterized by: include: The access correlation coefficient of any two modules in the software is obtained by combining the number of visits and access time interval of each user to the two modules in the software with the software registration duration, the number of visits and the access time of each user; Based on the data change relationship between any two modules, obtaining the data content correlation between any two modules; The access correlation coefficient and the data content correlation are integrated to obtain the comprehensive correlation between any two modules; Classify the data of each module in the software according to the comprehensive correlation between any two modules in the software; The process of obtaining the access correlation coefficient includes: Determine the correlation performance of the two modules for each user based on the number of visits to the two modules and the correlation therebetween, and in combination with the time interval between visits to the two modules; Based on each user's software registration time, as well as the number of times and duration of each user's software access, the user's access influence weight on the software is obtained; Based on the access influence weight of each user, performing weighted summation on the correlation performance of any two modules of each user to obtain the access correlation coefficient of the any two modules; The process of obtaining the data content relevance includes: determining whether new data is generated in a second data type in a second module within a preset time period after a change in first data in a first data type in a first module of a user; the first module and the second module constitute any two modules, the first data type is any data type in the first module, the first data is any piece of data in the first data type, and the second data type is any data type in the second module; If new data is generated, the data content correlation between the user's first data type in the first module and the second data type in the second module is obtained according to the judgment result and the time interval for generating the new data.

2. A big data intelligent classification method as claimed in claim 1, characterized in that: The process of obtaining the associated representation includes: Obtaining a proportion of consecutive visits to any two modules among the number of visits to any two modules, where the consecutive visits refer to a time interval between visits to any two modules being less than a preset time interval threshold; The association performance of any two modules is obtained based on the proportion of consecutive visits, the difference in visits and the average time interval between visits; the difference in visits is the difference in the number of visits of the user to the any two modules, and the average time interval between visits is the average time interval between the user's visits to the any two modules; the association performance is directly proportional to the proportion of consecutive visits, and inversely proportional to the difference in visits and the average time interval between visits.

3. The method for intelligent classification of big data according to claim 1, wherein: The process of obtaining the access influence weight includes: The user's access frequency is obtained based on the number of times the user accesses the software and the duration of the accesses, as well as the total number of days in which all the accesses occurred. The access frequency is proportional to the number of times the user accesses the software and the duration of the accesses, and inversely proportional to the total number of days. Obtaining a user's access reference value based on the access frequency and software registration duration; The access reference values ​​of all users are weighted and normalized to obtain the access influence weight of each user on the software.

4. The method for intelligent classification of big data according to claim 1, wherein: Obtaining, based on the judgment result and the time interval for generating new data, a data content correlation between the first data type in the first module and the second data type in the second module by the user, includes: Determine a correlation feature between first data of a first data type in the first module and a second data type in the second module; wherein if new data is generated, the correlation feature is inversely proportional to a time interval between the generation of the new data; and if no new data is generated, the correlation feature is 0; Calculate the average value of the relevant features corresponding to all changed data of the first data type in the first module by the user, and obtain the data content correlation between the first data type in the first module and the second data type in the second module by the user.

5. The method for intelligent classification of big data according to claim 1, wherein: The process of obtaining the comprehensive correlation includes: According to preset weights, the access association coefficients of the first module and the second module and the data content correlations of the first data type in the first module and the second data type in the second module are weighted and summed to obtain the comprehensive correlation between the first data type in the first module and the second data type in the second module.

6. The method for intelligent classification of big data according to claim 1, wherein: The data of each module in the software is classified according to the comprehensive correlation between any two modules in the software, including: Each comprehensive correlation is compared with a preset comprehensive correlation threshold, and corresponding data types of modules corresponding to comprehensive correlations greater than the preset comprehensive correlation threshold are classified into one category.

7. The method for intelligent classification of big data according to claim 1, wherein: The big data intelligent classification method further includes: The data of each module in the classified software is stored in different storage areas in the storage device.

8. A big data intelligent classification system, characterized by comprising: Memory and processor; The memory is connected to the processor; The memory is used to store program instructions; The processor is used to implement the big data intelligent classification method according to any one of claims 1 to 7 when the program instructions are executed.

Images