Disease control big data analysis method and system

Abstract

The invention discloses a disease control big data analysis method and system, and the method comprises the following steps: obtaining a data set X comprising a plurality of class attributes, and carrying out the preprocessing; performing independent optimal sequence search on each category attribute in the data set to obtain an optimal sequence, determining a distance structure suitable for disease control big data analysis, calculating distance measurement between a sample and a cluster according to the obtained optimal sequence, and constructing a clustering model; and by minimizing a target function, updating a sample attribution matrix and sequence, and dynamically adjusting a clustering result. Compared with the prior art, the method has the advantages that the system can adaptively adjust the clustering result by dynamically learning the optimal value sequence of the data and the corresponding distance structure, so that the clustering accuracy is remarkably improved, and visual interpretation is provided for understanding of the data.

Description

Technical Field

[0001] The present invention relates to the field of data analysis and processing technology, and more specifically, to a disease control big data analysis method and system. Background Art

[0002] In the field of disease control, data analysis, especially cluster analysis of categorical data, plays a vital role. Disease control data usually comes from a variety of complex monitoring systems and contains a large amount of categorical data, such as disease type, symptoms, and patient health status. These categorical data often have fuzzy or unknown value relationships, and traditional clustering methods are often difficult to apply directly. Especially in scenarios such as disease diagnosis and vaccine effectiveness evaluation, how to accurately cluster data to discover potential disease patterns or group characteristics has become an important challenge.

[0003] In order to make categorical data applicable to clustering algorithms, the most common practice is to convert it into numerical form, which is usually achieved through data encoding techniques. The most basic encoding method is one-hot encoding, which converts each category value into a binary vector for easy application in distance calculation. However, this method ignores the different "distances" or similarities that may exist between categorical data. For example, in the analysis of disease symptoms, "high fever" and "low fever" may be more similar than "high fever" and "headache", but one-hot encoding cannot reflect this. Therefore, simple distance metrics, such as Hamming distance, often lead to loss of information. To make up for this deficiency, more and more studies have introduced distance metrics based on statistical information, which help better distinguish the similarities between data samples by considering the frequency distribution of attribute values ​​and the distribution characteristics of samples.

[0004] As research deepens, some new encoding methods begin to take into account the relationship between attributes. For example, some attributes (such as "gender" and "age") may have intrinsic correlations, and simply encoding them independently cannot fully reflect their interactions in cluster analysis. Therefore, encoding methods based on coupling relationships have emerged, which improve the clustering effect by taking the coupling relationship between attributes into consideration. This method can help the algorithm better identify the inherent structure of the data, especially in multi-dimensional data, and can effectively capture the potential correlation between different attributes.

[0005] In the clustering of categorical data, representation learning technology has also gradually gained attention. Representation learning can reveal deeper structures in the data by automatically learning the high-dimensional feature representation of the data. For example, some methods dynamically adjust the data representation in the clustering process by learning the "distance" between different attribute values, making the clustering effect more accurate. This method is different from traditional manual feature design. It can directly learn useful representations from the data, has strong adaptability, and can show good results on different data sets. However, representation learning often requires complex models and a large amount of training data, and the selection of hyperparameters of the model has a great influence on the clustering effect, and the workload of parameter adjustment cannot be ignored.

[0006] In addition, the order of values ​​in categorical data (i.e., the sequential relationship between attribute values) is also particularly important in some application scenarios. For example, some attributes (such as "temperature" or "severity of patient symptoms") have a clear sequential relationship, and this sequential information is significantly helpful for clustering analysis. Traditional categorical data clustering methods often ignore this point, while some advanced clustering algorithms improve the clustering effect by introducing the concept of value order. These methods can better handle categorical data with a natural order, such as the stage of the disease, the severity of symptoms, etc. However, in many practical applications, the order of values ​​is not predetermined. Especially when faced with complex data with fuzzy value relationships, how to dynamically learn the most appropriate order of values ​​has become a problem that needs to be solved.

[0007] The prior art discloses a medical image enhancement method based on a clustering algorithm, which relates to a medical image enhancement method based on a clustering algorithm, including: clustering the grayscale values ​​of each pixel in a lung CT image using different category values ​​to obtain the lung regions of each cluster segmentation image; calculating the foreground degree value of each category in the lung region of each cluster segmentation image, thereby obtaining the necessity of enhancement of each pixel; obtaining the pixel to be enhanced according to the necessity of enhancement of each pixel; calculating the grayscale increment of each pixel to be enhanced according to the maximum grayscale value of all pixels to be enhanced and the necessity of enhancement of each pixel to be enhanced, and each pixel to be enhanced and its grayscale increment value constitute a grayscale increment image; adding the grayscale increment image to the grayscale value of the corresponding pixel in the lung CT image to obtain the enhanced lung CT image. The defect of this scheme is that the value order of the category data is not taken into account in the clustering process, resulting in the inability to obtain the most appropriate value order when facing complex data with fuzzy value relationships.

[0008] To this end, in combination with the above requirements and the defects of the existing technology, this application proposes a disease control big data analysis method and system. Summary of the invention

[0009] The present invention provides a disease control big data analysis method and system, which improves the accuracy of clustering by simultaneously optimizing the tasks of data clustering and defining the relationship between data, and can effectively process large-scale data in disease monitoring and epidemic early warning.

[0010] The primary purpose of the present invention is to solve the above technical problems. The technical solutions of the present invention are as follows:

[0011] The first aspect of the present invention provides a disease control big data analysis method, the method comprising the following steps:

[0012] S1. Obtain a data set X including multiple category attributes and perform preprocessing.

[0013] S2. Perform an independent optimal order search for each category attribute in the data set to obtain the optimal order and determine the distance structure suitable for disease control big data analysis.

[0014] S3. According to the obtained optimal order, the distance metric between samples and clusters is calculated to build a clustering model.

[0015] S4. By minimizing the objective function, the sample attribution matrix and order are updated, and the clustering results are dynamically adjusted.

[0016] Furthermore, the data set X consists of n data samples, each sample x i Represented as a d-dimensional row vector in the form of x i =[x i,1 ,x i,2 ,…,x i,d ] · , where d is the number of attributes of the sample, and each attribute a r The value of comes from a finite set of possible values l r Represents attribute a r The number of possible values ​​of

[0017] Furthermore, in step S1, the preprocessing process includes: checking missing values ​​and taking filling or deletion operations, converting the values ​​of all category attributes into a unified format, wherein the category variables are encoded and converted, the category data are converted into numerical data using label encoding, and sequential data are constructed based on variables with sequential properties. After data reception and preprocessing are completed, data verification is performed, and integrity checks are performed through statistical descriptions.

[0018] Furthermore, in step S2, each category attribute a r Has an optimal order An independent sequential search is performed for each attribute, and the effect of different sequential combinations is evaluated using an objective function based on the clustering task. The objective function is specifically:

[0019]

[0020] Among them, Θ(x i ,C m ; O) represents sample x i With cluster C m The distance between them, Q is the sample belonging matrix, which is an n×k matrix, where n is the number of samples, k is the number of clusters, indicating which cluster the sample in the data set belongs to, q i,m Represents sample x i With cluster C m The belonging between i,m =1 means x i Belongs to cluster C m ,q i,m =0 means x i Does not belong to cluster C m .

[0021] Furthermore, step S2 adopts the strategy of performing optimal order search within each cluster, traverses the samples in each cluster, calculates the distances between the samples and each possible value, and evaluates the relative importance of the value based on these distances. By summarizing the performance of each attribute value in all clusters, a weighted average ranking of each attribute value is obtained, and the weighted average ranking of each attribute in a specific cluster C is defined. m The optimal order within is By minimizing the internal component L r,m To obtain the optimal order, the form is:

[0022]

[0023] Among them, θ(x i,r ,p m,r ) represents the distance between the value of the sample on attribute r and the probability distribution of the cluster center. By sorting the weighted average of each attribute value, the algorithm finally obtains the global optimal order. And sort the obtained rankings.

[0024] Furthermore, step S3 includes the following specific steps: first, define the distance θ(x i ,C m ; O), this distance reflects the sample x i With cluster C m The similarity and distance between them are calculated as follows:

[0025]

[0026] Where d is the number of attributes, θ(x i,r ,p m,r ) represents the sample x under attribute r i,r With cluster center p m,r The distance between them, θ(x i,r ,p m,r ) can be expressed as:

[0027]

[0028] where d i,r is a sample x i,r is a vector of distances from all possible values, and p m,r is the description cluster C m A vector of probability distributions of the possible values ​​in .

[0029] Furthermore, when calculating the specific distance between the sample and the cluster, we first need to construct the distance vector It is defined as the normalized ordinal distance between the value of a sample at attribute r and every possible value:

[0030]

[0031] Among them l r is the number of possible values ​​of attribute r, o(x i,r ) and o(v r,g ) represent the samples x i,r and possible values ​​v r,g After completing the distance definition, update the sample attribution matrix Q. The specific update rules are as follows:

[0032]

[0033] The update rule indicates that if the sample x i With cluster C y The distance between them is the smallest, then the sample belongs to cluster C y .

[0034] Furthermore, step S4 includes the following specific steps: optimizing the clustering result by minimizing the clustering objective function, wherein minimizing the clustering objective function includes two sub-steps of fixing sample attribution and updating order and fixing attribute order and updating sample attribution, and the two sub-steps are performed alternately until the clustering objective function completes convergence, and the clustering objective function is defined as:

[0035]

[0036] Where Q is the sample attribution matrix, O is the order of the attributes, Θ(x i ,C m; O) represents sample x i With cluster C m The distance between.

[0037] Furthermore, the fixed sample attribution and update order are specifically as follows: fix the current sample attribution matrix And by minimizing the objective function To optimize the attribute order O, the algorithm obtains a new attribute order O by selecting the order that minimizes L. new , and update the distance metric D; the fixed attribute order and updated sample attribution are specifically: fix the latest attribute order And by minimizing the objective function To update the sample belonging matrix Q, recalculate the distance between each sample and all clusters according to the updated distance metric, and decide which cluster each sample should belong to based on these distances.

[0038] The second aspect of the present invention provides a disease control big data analysis system, which is used for the disease control big data analysis method, and includes: a data preprocessing module, an optimal sequence search module, a clustering module and a clustering update module.

[0039] The data preprocessing module obtains a data set X including multiple category attributes and performs preprocessing. The optimal order search module receives the preprocessed data set, performs an independent optimal order search for each category attribute in the data set, obtains the optimal order, and determines a distance structure suitable for disease control big data analysis. The clustering module calculates the distance metric between samples and clusters based on the obtained optimal order and constructs a clustering model. The clustering update module updates the sample attribution matrix and order by minimizing the objective function, and dynamically adjusts the clustering results.

[0040] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0041] The present invention provides a disease control big data analysis method and system, which can significantly improve the accuracy of clustering results by performing independent optimal order search for each category attribute in a data set, especially when facing complex and multidimensional data, and can better identify and optimize the relationship between categories, avoiding errors caused by simple category labels; by defining highly adaptable sequential distances and joint optimization mechanisms, it can handle potential ambiguity and uncertainty in the data, and is particularly suitable for complex and diverse health data in the field of disease control, and can achieve better clustering effects in the absence of clear relationships. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 The present invention is a flow chart of a disease control big data analysis method.

[0043] Figure 2 This is a visualization diagram of the category data clustering effect of the CBDM method in an embodiment of the present invention.

[0044] Figure 3 This is a visualization diagram of the clustering effect of category data of the ADC method in one embodiment of the present invention.

[0045] Figure 4 This is a visualization diagram of the clustering effect of category data using the OCL method in one embodiment of the present invention.

[0046] Figure 5 This is a schematic diagram of a disease control big data analysis system of the present invention. DETAILED DESCRIPTION

[0047] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0049] Example 1

[0050] like Figure 1 As shown, the present invention provides a disease control big data analysis method, which includes the following steps:

[0051] S1. Obtain a data set X including multiple category attributes and perform preprocessing.

[0052] S2. Perform an independent optimal order search for each category attribute in the data set to obtain the optimal order and determine the distance structure suitable for disease control big data analysis.

[0053] S3. According to the obtained optimal order, the distance metric between samples and clusters is calculated to build a clustering model.

[0054] S4. By minimizing the objective function, the sample attribution matrix and order are updated, and the clustering results are dynamically adjusted.

[0055] In a specific embodiment, the multiple category attributes come from disease control related data sets from different data sources, including: disease incidence, mortality, time and geographical information of epidemic outbreaks, environmental factors related to specific diseases, demographic characteristics, geographic information, clinical diagnosis and treatment data, laboratory test results, and the implementation of policies and prevention and control measures.

[0056] According to the above technical features, the present invention aims at the clustering problem of complex data in the field of disease control and prevention. By simultaneously optimizing the tasks of data clustering and defining the relationship between data, the accuracy of clustering is improved, and large-scale data in disease monitoring and epidemic warning can be effectively processed.

[0057] In the step S1, the preprocessing process includes: checking missing values ​​and taking filling or deletion operations, converting the values ​​of all categorical attributes into a unified format, wherein the categorical variables are encoded and converted, the categorical data are converted into numerical data using label encoding, and sequential data are constructed based on variables with sequential properties. After data reception and preprocessing are completed, data verification is performed, and integrity checks are performed through statistical descriptions. It should be noted that the system in the present invention focuses on the processing of sequential data and categorical data, so the system will encode and convert categorical variables, and convert categorical data into numerical data using label encoding for further analysis. At the same time, the system will also construct sequential data based on variables with sequential properties to ensure that these sequential information can be effectively reflected in the clustering process. The variables include different stages of the disease, the order of implementation of policy measures, etc., and the statistical description includes the number of samples and the distribution of attribute values.

[0058] The data set X consists of n data samples, each sample xi is represented by a d-dimensional row vector in the form of x i =[x i,1 ,x i,2 ,…,x i,d ] · , where d is the number of attributes of the sample, and each attribute a r The value of comes from a finite set of possible values l r Represents attribute a r The number of possible values ​​of

[0059] In step S2, each category attribute a r Has an optimal order An independent sequential search is performed for each attribute, and the effect of different sequential combinations is evaluated using an objective function based on the clustering task. The objective function is specifically:

[0060]

[0061] Among them, Θ(x i ,C m ; O) represents sample x i With cluster C m The distance between them, Q is the sample belonging matrix, which is an n×k matrix, where n is the number of samples, k is the number of clusters, indicating which cluster the sample in the data set belongs to, qi,m Represents sample x i With cluster C m The belonging between i,m =1 means x i Belongs to cluster C m ,q i,m =0 means x i Does not belong to cluster C m .

[0062] The step S2 adopts the strategy of performing optimal sequential search within each cluster, traverses the samples in each cluster, calculates the distances between the samples and each possible value, and evaluates the relative importance of the value based on these distances. By summarizing the performance of each attribute value in all clusters, a weighted average ranking of each attribute value is obtained, and the weighted average ranking of each attribute value in a specific cluster C is defined. m The optimal order within is By minimizing the internal component L r,m To obtain the optimal order, the form is:

[0063]

[0064] Among them, θ(x i,r ,p m,r ) represents the distance between the value of the sample on attribute r and the probability distribution of the cluster center. By sorting the weighted average of each attribute value, the algorithm finally obtains the global optimal order. And sort the obtained rankings.

[0065] It should be noted that in this step, the main goal of the algorithm is to dynamically learn the optimal order of attributes for each category in order to determine a distance structure suitable for a specific clustering task. This optimal order not only reflects the semantic relationship between attribute values, but is also crucial to improving the accuracy and interpretability of clustering. In order to improve search efficiency, the algorithm adopts a strategy of searching for the optimal order within each cluster. During the calculation process, the algorithm traverses the samples in each cluster, calculates the distance between the sample and each possible value, and evaluates the relative importance of the value based on these distances. Then, by summarizing the performance of each attribute value in all clusters, the algorithm can obtain a weighted average ranking for each attribute value. This ranking process takes into account the number of samples in each cluster to ensure that attribute values ​​that perform better in larger clusters occupy a more important position in the final order. By weighted average sorting of each attribute value, the algorithm finally obtains the global optimal order. This order provides the basis for defining the distance between samples and clusters in subsequent steps. It is worth noting that the final optimal order is not necessarily an integer, and the rankings obtained need to be sorted to ensure that the generated order is valid. Through this dynamic learning mechanism, the algorithm not only improves the accuracy of clustering, but also enhances the interpretability of clustering results, making the final clustering results more targeted and valuable for application.

[0066] The step S3 includes the following specific steps: First, define the distance θ(x i ,C m ; O), this distance reflects the sample x i With cluster C m The similarity and distance between them are calculated as follows:

[0067]

[0068] Where d is the number of attributes, θ(x i,r ,p m,r ) represents the sample x under attribute r i,r With cluster center p m,r The distance between them, θ(x i,r ,p m,r ) can be expressed as:

[0069]

[0070] where d i,r is a sample x i,r is a vector of distances from all possible values, and p m,r is the description cluster C m A vector of probability distributions of the possible values ​​in .

[0071] When calculating the specific distance between a sample and a cluster, we first need to construct a distance vector It is defined as the normalized ordinal distance between the value of a sample at attribute r and every possible value:

[0072]

[0073] Among them l r is the number of possible values ​​of attribute r, o(x i,r ) and o(v r,g ) represent the samples x i,r and possible values ​​v r,g After completing the distance definition, update the sample attribution matrix Q. The specific update rules are as follows:

[0074]

[0075] The update rule indicates that if the sample xi With cluster C y The distance between them is the smallest, then the sample belongs to cluster C y .

[0076] It should be noted that the algorithm implements the clustering task by defining the distance between samples and clusters, and updates the sample's attribution relationship accordingly. This process is the core of cluster analysis and directly affects the accuracy and effectiveness of clustering results. Through these calculations, the algorithm can update the sample's attribution according to the latest distance metric in each iteration, thereby continuously optimizing the clustering results. Ultimately, this process of distance definition and sample attribution update lays the foundation for the subsequent minimization of the objective function and optimization of clustering results.

[0077] The step S4 includes the following specific steps: optimizing the clustering result by minimizing the clustering objective function, wherein minimizing the clustering objective function includes two sub-steps of fixing sample attribution and updating order and fixing attribute order and updating sample attribution, and the two sub-steps are performed alternately until the clustering objective function completes convergence, and the clustering objective function is defined as:

[0078]

[0079] Where Q is the sample attribution matrix, O is the order of the attributes, Θ(x i ,C m ; O) represents sample x i With cluster C m The distance between.

[0080] The fixed sample attribution and update order are specifically as follows: fix the current sample attribution matrix And by minimizing the objective function To optimize the attribute order O, the algorithm obtains a new attribute order O by selecting the order that minimizes L. new , and update the distance metric D; the fixed attribute order and updated sample attribution are specifically: fix the latest attribute order And by minimizing the objective function To update the sample belonging matrix Q, recalculate the distance between each sample and all clusters according to the updated distance metric, and decide which cluster each sample should belong to based on these distances.

[0081] It should be noted that in this step, the goal of the algorithm is to optimize the clustering results by minimizing the clustering objective function, thereby improving the classification accuracy of the samples. This process involves multiple iterative steps to continuously adjust the clustering results according to the current sample attribution and attribute order. Through this process, the algorithm can achieve effective clustering of samples and dynamically adjust the sample attribution and attribute order, making the clustering results more accurate and more interpretable. This optimization mechanism based on minimization of the objective function enables the algorithm to adapt to different types of data sets and can produce reliable clustering results in practical applications. Finally, after multiple iterative optimizations, the algorithm outputs the clustering results and their corresponding attribute order, which provides important support for subsequent data analysis and decision-making.

[0082] The present invention has the following advantages and effects compared with the prior art:

[0083] 1. Improve clustering accuracy: Traditional clustering methods in the prior art are usually optimized for numerical data, and lack effective processing of categorical data, especially sequential categorical data. The present invention introduces a sequence-based categorical data clustering method combined with a sequential learning mechanism to effectively solve the problem of low clustering accuracy of categorical data. By considering the sequential relationship of categorical data in the clustering process, the accuracy of the clustering results is significantly improved, especially when facing complex and multidimensional data, it can better identify and optimize the relationship between categories, avoiding errors caused by simple category labels.

[0084] 2. Enhanced data adaptability: Clustering methods in the prior art often assume that there is a clear numerical relationship or distance definition between data, and cannot effectively handle fuzzy or uncertain data relationships. The system of the present invention can handle potential fuzziness and uncertainty in data by defining highly adaptable sequential distances and joint optimization mechanisms. It is particularly suitable for complex and diverse health data in the field of disease control, and can achieve better clustering effects in the absence of clear relationships.

[0085] 3. Strong interpretability: Unlike existing deep learning or black box models, the joint optimization strategy based on sequential learning and category data in the present invention has good interpretability. By analyzing the order of clustering results and the relationship between category data, the key features and factors in the clustering process can be intuitively understood, which helps disease control decision makers make scientific decisions based on data, especially in the evaluation and adjustment of prevention and control measures, and can provide more reliable decision support.

[0086] 4. Adapt to the special needs of the disease control field: The data in the disease control field often have the characteristics of strong timeliness, complex correlation, and diverse data dimensions. The existing general clustering methods are difficult to optimize for these characteristics. The present invention can better adapt to this complexity by designing a clustering method specifically for disease control big data, especially when dealing with nonlinear and dynamically changing complex data such as the early stage of an epidemic outbreak, different stages of disease transmission, and policy implementation. It has significant advantages.

[0087] 5. Efficiency and scalability: The present invention not only has advantages in clustering accuracy, but also can effectively improve computing efficiency. By optimizing the data processing process and combining modern computing frameworks and algorithms, the system can quickly process large-scale data and adapt to the characteristics of large data volumes and fast updates in the field of disease control. In addition, the system is designed with good scalability, and can adjust and expand data sources and analysis models according to actual needs, providing flexible support for future application scenarios.

[0088] Example 2

[0089] Based on the above embodiment 1, combined with Figure 2-Figure 4 , this embodiment describes in detail the category data clustering capability of the present invention.

[0090] In this embodiment, the disease control big data analysis method proposed in the present invention is referred to as OCL. Three clustering methods, OCL, CBDM and ADC, are used to learn the VT data set, calculate the distance between attribute values, and map the processed data to a two-dimensional space for visualization through the t-distributed random neighbor embedding method (t-SNE). By comparing the clustering results of different methods, the significant advantages of the present invention in the category data clustering task are verified.

[0091] like Figure 2 As shown in Figure 1, we first preprocessed the VT dataset to ensure the integrity and validity of the data. Then, we used OCL, CBDM, and ADC methods for clustering, respectively, to obtain the distance structure between attribute values ​​learned by each method, and used these distance values ​​to convert the original category data into numerical form for visual analysis. Figure 2-Figure 4 The results of t-SNE visualization of the VT data set are shown respectively, corresponding to the clustering effects of the CBDM, ADC and OCL methods. It can be clearly seen from the visualization results that the OCL method of the present invention performs well in clustering effect. Compared with the CBDM and ADC methods, OCL can distinguish samples of different categories more clearly, the overlap between categories is significantly reduced, the clustering area is more compact, and it shows a stronger clustering distinction ability.

[0092] In particular, the clustering results of the OCL method show obvious category separation characteristics, indicating that OCL can effectively identify and optimize the potential category structure in the data. This performance can optimize the clustering effect through the sequential learning mechanism and the sequential relationship between category data when processing the VT dataset without a clear numerical relationship. In contrast, the clustering effects of the CBDM and ADC methods have a certain category mixing phenomenon, and the boundaries between different categories are not clear, resulting in reduced clustering accuracy.

[0093] Through this experiment, the method of the present invention demonstrates its effectiveness and advantages in clustering category data, especially in the disease control big data analysis system, where accurate cluster analysis can help better understand the category relationship behind the data, thereby providing accurate decision support for disease prevention, epidemic trend analysis, etc. This experimental result verifies the superiority of the technical solution of the present invention in practical applications, and provides a solid theoretical and empirical basis for the application of the present invention in the field of disease control big data.

[0094] Example 3

[0095] like Figure 5 As shown, the present invention also provides a disease control big data analysis system, which is used for the disease control big data analysis method, and includes: a data preprocessing module, an optimal sequence search module, a clustering module and a clustering update module.

[0096] The data preprocessing module obtains a data set X including multiple category attributes and performs preprocessing. The optimal order search module receives the preprocessed data set, performs an independent optimal order search for each category attribute in the data set, obtains the optimal order, and determines a distance structure suitable for disease control big data analysis. The clustering module calculates the distance metric between samples and clusters based on the obtained optimal order and constructs a clustering model. The clustering update module updates the sample attribution matrix and order by minimizing the objective function, and dynamically adjusts the clustering results.

[0097] In the embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above method embodiments; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media capable of storing program codes.

[0098] Alternatively, if the above-mentioned embodiments of the present invention are implemented in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media capable of storing program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0099] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. The icons describing the structural positional relationships in the accompanying drawings are only used for exemplary purposes and are not to be construed as limiting the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and is not possible to list all the embodiments here. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the claims of the present invention.

Claims

1. A disease control big data analysis method, characterized in that: The following steps are involved: S1, obtain a data set X including multiple category attributes and perform preprocessing; S2. Perform an independent optimal order search for each category attribute in the data set to obtain the optimal order and determine the distance structure suitable for disease control big data analysis; S3, according to the obtained optimal order, calculate the distance metric between samples and clusters and build a clustering model; S4. By minimizing the objective function, the sample attribution matrix and order are updated, and the clustering results are dynamically adjusted.

2. A disease control big data analysis method according to claim 1, characterized in that: The data set X consists of n data samples, each sample x i Represented as a d-dimensional row vector in the form of x i =[x i,1 ,x i,2 ,…,x i,d ] · , where d is the number of attributes of the sample, and each attribute a r The value of comes from a finite set of possible values l r Represents attribute a r The number of possible values, usually satisfying l r □n.

3. A disease control big data analysis method according to claim 2, characterized in that: In step S1, the preprocessing process includes: checking missing values ​​and taking filling or deletion operations, converting the values ​​of all category attributes into a unified format, encoding and converting category variables, converting category data into numerical data using label encoding, and constructing sequential data based on variables with sequential properties. After data reception and preprocessing are completed, data verification is performed and integrity check is performed through statistical description.

4. The disease control big data analysis method according to claim 1, characterized in that: In step S2, each category attribute a r Has an optimal order An independent sequential search is performed for each attribute, and the effect of different sequential combinations is evaluated using an objective function based on the clustering task. The objective function is specifically: Among them, Θ(x i ,C m ; O) represents sample x i With cluster C m The distance between them, Q is the sample belonging matrix, which is an n×k matrix, where n is the number of samples, k is the number of clusters, indicating which cluster the sample in the data set belongs to, q i,m Represents sample x i With cluster C m The belonging between i,m =1 means x i Belongs to cluster C m ,q i,m =0 means x i Does not belong to cluster C m .

5. A disease control big data analysis method according to claim 4, characterized in that: The step S2 adopts the strategy of performing optimal sequential search within each cluster, traverses the samples in each cluster, calculates the distances between the samples and each possible value, and evaluates the relative importance of the value based on these distances. By summarizing the performance of each attribute value in all clusters, a weighted average ranking of each attribute value is obtained, and the weighted average ranking of each attribute value in a specific cluster C is defined. m The optimal order within is By minimizing the internal component L r,m To obtain the optimal order, the form is: Among them, θ(x i,r ,p m,r ) represents the distance between the value of the sample on attribute r and the probability distribution of the cluster center. By sorting the weighted average of each attribute value, the algorithm finally obtains the global optimal order. And sort the obtained rankings.

6. A disease control big data analysis method according to claim 5, characterized in that: The step S3 includes the following specific steps: First, define the distance θ(x i ,C m ; O), this distance reflects the sample x i With cluster C m The similarity and distance between them are calculated as follows: Where d is the number of attributes, θ(x i,r ,p m,r ) represents the sample x under attribute r i,r With cluster center p m,r The distance between them, θ(x i,r ,p m,r ) can be expressed as: where d i,r is a sample x i,r is a vector of distances from all possible values, and p m,r is the description cluster C m A vector of probability distributions of the possible values ​​in .

7. A disease control big data analysis method according to claim 6, characterized in that: When calculating the specific distance between a sample and a cluster, we first need to construct a distance vector It is defined as the normalized ordinal distance between the value of a sample at attribute r and every possible value: Among them l r is the number of possible values ​​of attribute r, o(x i,r ) and o(v r,g ) represent the samples x i,r and possible values ​​v r,g After completing the distance definition, update the sample attribution matrix Q. The specific update rules are as follows: The update rule indicates that if the sample x i With cluster C y The distance between them is the smallest, then the sample belongs to cluster C y .

8. A disease control big data analysis method according to claim 7, characterized in that: The step S4 includes the following specific steps: optimizing the clustering result by minimizing the clustering objective function, wherein minimizing the clustering objective function includes two sub-steps of fixing sample attribution and updating order and fixing attribute order and updating sample attribution, and the two sub-steps are performed alternately until the clustering objective function completes convergence, and the clustering objective function is defined as: Where Q is the sample attribution matrix, O is the order of the attributes, Θ(x i ,C m ; O) represents sample x i With cluster C m The distance between.

9. A disease control big data analysis method according to claim 8, characterized in that: The fixed sample attribution and update order are specifically as follows: fix the current sample attribution matrix And by minimizing the objective function To optimize the attribute order O, the algorithm obtains a new attribute order O by selecting the order that minimizes L. new , and update the distance metric D; the fixed attribute order and updated sample attribution are specifically: fix the latest attribute order And by minimizing the objective function To update the sample belonging matrix Q, recalculate the distance between each sample and all clusters according to the updated distance metric, and decide which cluster each sample should belong to based on these distances.

10. A disease control big data analysis system, the system being used for a disease control big data analysis method according to any one of claims 1 to 9, characterized in that: It includes: data preprocessing module, optimal sequence search module, clustering module and clustering update module; The data preprocessing module obtains a data set X including multiple category attributes and performs preprocessing. The optimal order search module receives the preprocessed data set, performs an independent optimal order search for each category attribute in the data set, obtains the optimal order, and determines a distance structure suitable for disease control big data analysis. The clustering module calculates the distance metric between samples and clusters based on the obtained optimal order and constructs a clustering model. The clustering update module updates the sample attribution matrix and order by minimizing the objective function, and dynamically adjusts the clustering results.

Images