Disease control big data analysis method and system

Abstract

The invention discloses a disease control big data analysis method and system, and the method comprises the following steps: obtaining a multi-source heterogeneous disease control data set, carrying out the preprocessing of the multi-source heterogeneous disease control data set, carrying out the independent optimal sequence search of each class attribute in a cluster to which the class attribute belongs, obtaining the optimal sequence of the class attribute, and obtaining the optimal sequence of the class attribute; determining a distance structure suitable for disease control big data analysis based on the optimal sequence; according to the obtained optimal sequence, distance measurement between the samples and the clusters is calculated, and a clustering model is constructed; alternately updating a sample attribution matrix and a category attribute sequence by minimizing a target function, and dynamically adjusting a clustering result; and outputting a case group, a propagation chain identification result and a risk assessment report based on a clustering result. According to the method, the clustering result is adaptively adjusted through an attribute sequence learning and sample clustering optimization conjoint analysis method, and the problem that an epidemic prevention and control decision is seriously affected due to the fact that the clustering result in the traditional technology cannot reflect business logic such as a real propagation chain and risk population layering is solved.

Description

Technical Field

[0001] This invention relates to the field of data analysis and processing technology, and more specifically, to a method and system for big data analysis in disease control. Background Technology

[0002] In the field of disease control, data analysis, especially cluster analysis of categorical data, plays a crucial role. Disease control data typically originates from multiple complex monitoring systems and contains a large amount of categorical data, such as disease types, symptoms, and patient health status. These categorical data often have fuzzy or unknown value relationships, making traditional clustering methods difficult to apply directly. Especially in scenarios such as disease diagnosis and vaccine effectiveness evaluation, accurately clustering data to discover potential disease patterns or population characteristics has become a significant challenge.

[0003] To enable categorical data to be used in clustering algorithms, the most common approach is to convert it into numerical form, typically achieved through data encoding techniques. The most basic encoding method is one-hot encoding, which transforms each category value into a binary vector, facilitating distance calculations. However, this method overlooks the potentially varying degrees of "distance" or similarity that may exist between categorical data. For example, in disease symptom analysis, "high fever" and "low fever" might be more similar than "high fever" and "headache," but one-hot encoding fails to reflect this. Therefore, relying on simple distance metrics, such as Hamming distance, often leads to information loss. To compensate for this deficiency, increasing research has introduced statistically based distance metrics. These methods, by considering the frequency distribution of attribute values ​​and the distribution characteristics of samples, help to better distinguish the similarity between data samples.

[0004] As research progresses, novel coding methods have begun to consider the relationships between attributes. For example, certain attributes (such as "gender" and "age") may have inherent correlations, and simply encoding them independently cannot fully reflect their interactions in cluster analysis. Therefore, coupling-based coding methods have emerged, improving clustering effectiveness by incorporating the coupling relationships between attributes. This approach helps algorithms better identify the inherent structure of data, especially in multi-dimensional data, effectively capturing the potential correlations between different attributes.

[0005] In categorical data clustering, representation learning techniques are gaining increasing attention. Representation learning, by automatically learning high-dimensional feature representations of data, can reveal deeper structures within the data. For example, some methods dynamically adjust the data representation during the clustering process by learning the "distance" between different attribute values, resulting in more accurate clustering. Unlike traditional hand-designed features, this method can directly learn useful representations from the data, is highly adaptable, and demonstrates good performance on different datasets. However, representation learning often requires complex models and large amounts of training data, and the selection of hyperparameters significantly impacts clustering results; the workload of hyperparameter tuning is also considerable.

[0006] Furthermore, the order of values ​​in categorical data (i.e., the sequential relationship between attribute values) is particularly important in some application scenarios. For example, certain attributes (such as "temperature" or "severity of patient symptoms") have explicit sequential relationships, and this sequential information is significantly helpful for cluster analysis. Traditional categorical data clustering methods often overlook this, while some advanced clustering algorithms improve clustering results by introducing the concept of value order. These methods can better handle categorical data with natural order, such as disease stages or symptom severity. However, in many practical applications, the value order is not predetermined, especially when dealing with complex data with fuzzy value relationships. How to dynamically learn the most suitable value order has become a pressing problem to be solved.

[0007] A prior art medical image enhancement method based on clustering algorithms is disclosed, which includes: clustering the gray values ​​of each pixel in a lung CT image using different category values ​​to obtain lung regions of each cluster segmented image; calculating the foreground degree value of each category in the lung regions of each cluster segmented image to obtain the enhancement necessity of each pixel; obtaining the pixels to be enhanced based on the enhancement necessity of each pixel; calculating the gray value increment of each pixel to be enhanced based on the maximum gray value among all pixels to be enhanced and the enhancement necessity of each pixel to be enhanced, and each pixel to be enhanced and its gray value increment constitute a gray value increment image; adding the gray value increment image to the gray value of the corresponding pixel in the lung CT image to obtain the enhanced lung CT image. The drawback of this scheme is that it does not consider the value order of the category data during the clustering process, which leads to the inability to obtain the most suitable value order when dealing with complex data with fuzzy value relationships.

[0008] Therefore, in light of the above requirements and the shortcomings of existing technologies, this application proposes a disease control big data analysis method and system. Summary of the Invention

[0009] This invention provides a method and system for big data analysis in disease control. By simultaneously optimizing the tasks of data clustering and defining the relationships between data, it improves the accuracy of clustering and can effectively handle large-scale data in disease monitoring and epidemic early warning.

[0010] The primary objective of this invention is to solve the aforementioned technical problems. The technical solution of this invention is as follows: The first aspect of this invention provides a method for analyzing big data related to disease control, which includes the following steps: S1. Obtain the multi-source heterogeneous disease control dataset X reported from the hospital information system, laboratory information system and community system and preprocess it. The preprocessed multi-source heterogeneous disease control dataset X includes multiple category attributes with business semantics.

[0011] S2. For each category attribute in the multi-source heterogeneous disease control dataset, perform an independent optimal order search within its cluster to obtain the optimal order of the category attribute, and determine a distance structure suitable for disease control big data analysis based on the optimal order.

[0012] S3. Based on the optimal order, calculate the distance metric between the sample and the cluster, and construct a clustering model.

[0013] S4. By minimizing the objective function, the sample attribution matrix and the order of category attributes are updated alternately to dynamically adjust the clustering results.

[0014] S5. Output case clustering, transmission chain identification results, and risk assessment reports based on clustering results.

[0015] The second aspect of the present invention provides a disease control big data analysis system, which is used in the aforementioned disease control big data analysis method, and includes: a data preprocessing module, an optimal order search module, a clustering module, a clustering update module, and a disease control analysis module.

[0016] The data preprocessing module acquires and preprocesses a multi-source heterogeneous disease control dataset X reported from hospital information systems, laboratory information systems, and community systems. The preprocessed multi-source heterogeneous disease control dataset X includes multiple category attributes with business semantics. The optimal order search module receives the preprocessed dataset and performs an independent optimal order search within its cluster for each category attribute in the multi-source heterogeneous disease control dataset to obtain the optimal order of that category attribute. Based on the optimal order, it determines a distance structure suitable for disease control big data analysis. The clustering module calculates the distance metric between samples and clusters according to the obtained optimal order and constructs a clustering model. The clustering update module dynamically adjusts the clustering results by alternately updating the sample affiliation matrix and category attribute order by minimizing the objective function. The disease control analysis module outputs case clustering, transmission chain identification results, and risk assessment reports based on the clustering results.

[0017] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention provides a method and system for big data analysis in disease control. By independently performing optimal order searches on each category attribute in multi-source heterogeneous disease control datasets reported from hospital information systems, laboratory information systems, and community systems, the accuracy of clustering results can be significantly improved. Especially when dealing with complex and multidimensional data, it can better identify and optimize the relationships between categories, avoiding errors caused by simple category labels. By defining a highly adaptive order distance and joint optimization mechanism, it can handle the potential ambiguity and uncertainty in the data, making it particularly suitable for complex and diverse health data in the field of disease control. It can achieve good clustering results even when there are no clear relationships. Further analysis of the clustering results yields case clustering, transmission chain identification results, and risk assessment reports closely related to disease control decision-making, reflecting the actual transmission chain, risk population stratification, and other business logic. Attached Figure Description

[0018] Figure 1 This is a flowchart of a disease control big data analysis method according to the present invention.

[0019] Figure 2 This is a visualization of the categorical data clustering effect of the CBDM method in one embodiment of the present invention.

[0020] Figure 3 This is a visualization of the clustering effect of the ADC method for category data in one embodiment of the present invention.

[0021] Figure 4 This is a visualization of the categorical data clustering effect of the OCL method in one embodiment of the present invention.

[0022] Figure 5 This is a schematic diagram of a disease control big data analysis system according to the present invention. Detailed Implementation

[0023] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0024] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0025] Example 1 In core operations of disease control, such as disease surveillance, epidemic early warning, and transmission chain tracing, efficient clustering analysis of multi-source heterogeneous data is required. Specific data types may include: Key case data: patient symptoms, contact history, and nucleic acid test results; among which, patient symptoms include non-numerical categories such as asymptomatic, cough, and fever; contact history includes travel, family, and hospital; and nucleic acid test results include progressive categories such as negative, initial screening positive, and retest positive.

[0026] Decision support data: onset time and basic attributes; the onset time is segmented by period, and the basic attributes include discrete categories such as age and gender.

[0027] Multi-source collaborative data: cross-scenario data such as hospital medical records, laboratory test data, and community-reported information.

[0028] Traditional clustering methods (such as K-Means and hierarchical clustering) have significant limitations: they cannot directly process the aforementioned non-numerical category data and require conversion through One-Hot encoding, fixed-order encoding, etc., but such conversions lose the semantic order relationship between categories, such as the correlation between the severity of mild and severe cases, and the correlation between the diagnosis process of "positive initial screening and positive retesting"; the semantic distance characterization of fuzzy category values ​​such as unexplained contact history and suspected travel contact history is distorted, resulting in clustering results that cannot reflect the real transmission chain, risk population stratification, and other business logic, seriously affecting the accuracy of epidemic prevention and control decisions.

[0029] Given the above application background, the specific technical problems that this solution needs to solve are as follows: There is an engineering adaptation problem stemming from the lack of semantic order awareness in category attributes. Existing coding techniques cannot dynamically learn the semantic order of disease control category data, such as the symptom severity order of {asymptomatic, cough, fever, chest tightness} or the diagnosis process order of {negative, initial screening positive, retest positive}. This leads to a disconnect between clustering results and disease control operations, necessitating technical means to align "data-semantics-business".

[0030] Bottlenecks in distance measurement technology for multi-source fuzzy category data. Complex non-numerical categories such as "symptom combinations" and "contact history routes" in disease control data lack engineering-practical distance calculation methods. Traditional distance measures (such as Euclidean distance and Hamming distance) cannot reflect their business semantic relationships, resulting in fuzzy cluster boundaries and high overlap.

[0031] The clustering results are not well-suited to the technical needs of disease control decision-making. Existing solutions do not form a technical closed loop of "clustering results → transmission chain identification → at-risk population segmentation → decision support," and cannot handle dynamically updated data in scenarios involving daily new cases and cross-regional data linkage. They are also difficult to integrate into existing disease control data platforms and lack engineering practicality.

[0032] Based on the problems existing in the above-mentioned technologies, such as Figure 1 As shown, this invention provides a method for big data analysis in disease control, which includes the following steps: S1. Obtain the multi-source heterogeneous disease control dataset X reported from the hospital information system, laboratory information system and community system and preprocess it. The preprocessed multi-source heterogeneous disease control dataset X includes multiple category attributes with business semantics.

[0033] S2. For each category attribute in the multi-source heterogeneous disease control dataset, perform an independent optimal order search within its cluster to obtain the optimal order of the category attribute, and determine a distance structure suitable for disease control big data analysis based on the optimal order.

[0034] S3. Based on the optimal order, calculate the distance metric between the sample and the cluster, and construct a clustering model.

[0035] S4. By minimizing the objective function, the sample attribution matrix and the order of category attributes are updated alternately to dynamically adjust the clustering results.

[0036] S5. Output case clustering, transmission chain identification results, and risk assessment reports based on clustering results.

[0037] The category attributes with business semantics include at least one of the following: symptom attributes, diagnostic attributes, testing attributes, contact history attributes, vaccination attributes, and transmission association attributes. In a specific embodiment, the multiple category attributes come from disease control-related datasets from different data sources, covering the core dimensions of transmission prevention and control. Common data sources and category attributes are shown in Table 1, including: disease incidence rate, mortality rate, time and geographical information of outbreaks, environmental factors related to specific diseases, demographic characteristics, geographic information, clinical diagnosis and treatment data, laboratory test results, and the implementation status of policies and prevention and control measures.

[0038]

[0039] Table 1. Common data sources and category attributes of multi-source disease control category datasets Based on the above technical features, this invention addresses the clustering problem of complex data in the field of disease control and prevention by simultaneously optimizing the tasks of data clustering and defining the relationships between data, thereby improving the accuracy of clustering and effectively handling large-scale data in disease monitoring and epidemic early warning.

[0040] The preprocessed multi-source heterogeneous disease control dataset Depend on It consists of 10 data samples, each sample Represented as a 3D row vector, in the form of ,in The number of category attributes of the data samples is denoted as X. The preprocessed dataset X is used as the sample set of the clustering model in step S3. Together with the number of clusters to be divided K, it is used as the input of the clustering model. The dataset X is clustered into K clusters by the clustering model.

[0041] In step S1, the preprocessing process includes: checking for missing and ambiguous values ​​and performing fill or deletion operations based on the correlation between the disease control business ledger and the data; performing a unified format conversion on the values ​​of all category attributes; encoding conversion of category variables; using label encoding to convert the data values ​​of category attributes into numerical data; and constructing ordinal data based on variables with ordinal properties. After data reception and preprocessing are completed, data verification is performed, and integrity checks are conducted through statistical description. It should be noted that the system in this invention focuses on the processing of ordinal and category data. Therefore, the system encodes category variables and uses label encoding to convert category data into numerical data for further analysis. Simultaneously, the system constructs ordinal data based on variables with ordinal properties to ensure that this ordinal information is effectively reflected in the clustering process. The variables include different stages of the disease, the implementation sequence of policy measures, etc., and the statistical description includes the sample size and attribute value distribution.

[0042] In a specific implementation, missing items such as "unknown contact history" and "unrecorded vaccination status" are supplemented by combining the disease control business ledger and data correlation; vague qualitative descriptions such as "suspected fever" and "mild cough" are supplemented by combining the "Guidelines for Diagnosis and Treatment of Infectious Diseases" to avoid the problem of "fuzzy values ​​causing distortion of distance calculation"; non-uniform fields from different systems are converted into a unified format, such as "positive", "POS", and "retest positive" in the LIS system are unified as "negative / initial screening positive / retest positive".

[0043] Each category attribute Having an optimal order, the algorithm independently searches for the order of each category attribute and evaluates the effect of different order combinations using a clustering task-based objective function. In this embodiment, for each category attribute (symptoms, testing, contact history, etc.) in the disease control field, the algorithm uses intra-cluster statistics of OCL and the logic of OLO (Optimal LinearOrdering) to independently learn the optimal order and minimize the objective function.

[0044] The objective function is specifically:

[0045]

[0046]

[0047] in, Indicates sample with cluster The distance between them, where m represents the cluster index. This is the sample attribution matrix, indicating which cluster a sample in the dataset belongs to. Indicates the order of category attributes. Indicates the number of category attributes. This represents the ordinal distance between the sample category attribute value 'r' and other values. Indicates in cluster Category attributes value The probability of its occurrence, Represents a collection of category attributes.

[0048] In one specific embodiment, the probability of a "retest positive" result in the "high-transmission cluster of severe cases" is:

[0049] A strategy of optimal order search within each cluster is adopted, traversing the samples within each cluster, calculating the distance between a sample and each possible value, and evaluating the relative importance of the value based on these distances. A weighted average ranking of each attribute value is obtained by summing the performance of each attribute value across all clusters. Specifically, this includes defining the ranking of each attribute within a specific cluster. The optimal order within is By minimizing internal components To obtain the optimal order, the form is:

[0050] in, Indicates the value to be Category attributes Cluster The contribution of the objective function, i.e., the category attribute The sum of the ordinal distances of all possible values. Indicates the sample in the category attribute The distance between the value on the index and the probability distribution of the cluster center, in a specific embodiment, is represented by the total distance of "cough" in the "mild case family cluster" as:

[0051] The link strength for each category attribute value is then calculated using the following formula. :

[0052] Each category attribute value is sorted by weighted average based on its link strength. Category attribute values ​​with higher link strength are placed in the middle of the queue. Then, all local sorts are merged by cluster size, resulting in the globally optimal order. And sort the resulting rankings.

[0053] In one specific embodiment, the link strength of "positive retest" in the "severe cluster" is:

[0054] In step S3, when calculating the distance metric between a sample and a cluster, it is first necessary to construct a distance vector. It is defined as a sample in terms of attributes The normalized ordinal distance between the value and each possible value:

[0055] in It is an attribute The number of possible values, and Representing samples respectively and possible values The serial number.

[0056] In one specific embodiment, all samples are divided into 2 clusters, in cluster... There are 800 samples, and the symptom attributes are ranked as follows:

[0057] In cluster There are 200 samples, and the symptom attributes are ranked as follows:

[0058] Taking the symptom "fever" as an example, it is found in clusters Its sorting position is 2, in cluster The sorting position is 3, therefore the weighted sorting position of this symptom is .

[0059] Following this logic, the optimal ranking of symptom attributes is:

[0060] After processing by the sequential learning layer, the optimal order of each category attribute in dataset X is determined.

[0061] Step S4 includes the following specific steps: optimizing the clustering result by minimizing the clustering objective function, wherein minimizing the clustering objective function includes two sub-steps: fixing the sample assignment and update order, and fixing the attribute order and updating the sample assignment. The two sub-steps are performed alternately until the clustering objective function converges.

[0062] Specifically, the fixed attribute order and updated sample attribution involve updating the sample attribution matrix after defining the distance based on the actual epidemiological investigation or testing results from the disease control center. The specific update rules are as follows:

[0063] The update rule indicates that if the sample with cluster The sample belongs to the cluster with the smallest distance between them. ; The triggering condition for the fixed sample attribution and update order is: when an epidemic mutation occurs, i.e., the mutant strain causes a change in the optimal order, the link strength is recalculated. The order of category attributes is adjusted, the distance between all samples and clusters is recalculated, and the assignment of some samples is adjusted.

[0064] In one specific embodiment, this X was initially classified as a "sporadic suspected cluster" (unknown contact history). After epidemiological investigation, it was added that "he had ridden in the same elevator as a severe case", and the distance was recalculated, so he was classified as a "severe high-transmission cluster". This information was then pushed to the epidemiological investigation team to update the control list.

[0065] It's important to note that in this step, the algorithm's primary goal is to dynamically learn the optimal order for each category attribute in order to determine a distance structure suitable for a specific clustering task. This optimal order not only reflects the semantic relationships between attribute values ​​but is also crucial for improving the accuracy and interpretability of clustering. To improve search efficiency, the algorithm employs a strategy of searching for the optimal order within each cluster. During computation, the algorithm iterates through the samples within each cluster, calculates the distance between each sample and each possible value, and evaluates the relative importance of the value based on these distances. Then, by summarizing the performance of each attribute value across all clusters, the algorithm obtains a weighted average ranking for each attribute value. This ranking process takes into account the number of samples in each cluster, ensuring that attribute values ​​that perform better in larger clusters occupy a more significant position in the final order. Through the weighted average ranking of each attribute value, the algorithm ultimately obtains the globally optimal order. This order provides the foundation for defining the distance between samples and clusters in subsequent steps. It's important to note that the final optimal order is not necessarily an integer; the obtained rankings need to be sorted to ensure the generated order is valid. Through this dynamic learning mechanism, the algorithm not only improves the accuracy of clustering but also enhances the interpretability of the clustering results, making the final clustering results more targeted and valuable for application.

[0066] The fixed sample attribution and update order specifically refers to: fixing the current sample attribution matrix. And by minimizing the objective function To optimize the attribute order By selecting, it can be minimized The order in which new attributes are obtained. and update the distance metric. The fixed attribute order and updated sample attribution specifically refer to: fixing the latest attribute order. And by minimizing the objective function To update the sample attribution matrix Based on the updated distance metric, the distance between each sample and all clusters is recalculated, and each sample is assigned to a specific cluster based on these distances.

[0067] It's important to note that the algorithm performs clustering by defining the distance between samples and clusters, and updates the sample affiliations accordingly. This process is the core of cluster analysis, directly impacting the accuracy and effectiveness of the clustering results. Through these calculations, the algorithm updates sample affiliations in each iteration based on the latest distance metric, thereby continuously optimizing the clustering results. Ultimately, this process of defining distances and updating sample affiliations lays the foundation for subsequent objective function minimization and clustering result optimization.

[0068] Step S5 includes the following specific steps: Based on the sample affiliation matrix in the clustering results, the data samples are divided into different case groups. Based on the central features and attribute distribution of each cluster, a clinical feature description is generated for each case group. The clinical feature description includes the combination of main symptoms, the distribution of test results, and the pattern of contact history.

[0069] Analyze the attribute order correlation between different clusters, reconstruct the propagation path based on the contact history attributes and geographic time information of samples within the cluster, and mark the key nodes and diffusion direction in the propagation chain.

[0070] Based on the size, attribute distribution characteristics, and transmission chain identification results of each cluster, the risk level index of each cluster is calculated. Combined with the epidemiological model, a risk assessment report is generated, which includes the size of the high-risk population, the transmission risk index, and early warning recommendations.

[0071] The results of case clustering, transmission chain identification, and risk assessment reports are integrated into a structured output and converted into an instruction format that can be recognized by the disease control business system, automatically triggering the execution of corresponding prevention and control measures.

[0072] In a specific implementation, for clusters of severe cases with high transmission risk, the cluster analysis results output "prioritize the allocation of ICU resources," etc.; for clusters of mild cases with family transmission, the cluster analysis results output "community disinfection guidelines," etc.; and for sporadic suspected clusters, the cluster analysis results output "complete travel history" and "test every 2 days," etc.

[0073] The present invention has the following advantages and effects compared with the prior art: 1. Improved Clustering Accuracy: Traditional clustering methods in existing technologies are typically optimized for numerical data and lack effective processing for categorical data, especially ordinal categorical data. This invention, by introducing an ordinal-based categorical data clustering method combined with an ordinal learning mechanism, effectively solves the problem of low clustering accuracy for categorical data. By considering the ordinal relationships of categorical data simultaneously during the clustering process, the accuracy of the clustering results is significantly improved. Particularly when dealing with complex and multidimensional data, it can better identify and optimize the relationships between categories, avoiding errors caused by simple category labels.

[0074] 2. Enhanced Data Adaptability: Existing clustering methods often assume explicit numerical relationships or distance definitions between data points, failing to effectively handle fuzzy or uncertain data relationships. The system of this invention, by defining a highly adaptable sequential distance and joint optimization mechanism, can handle potential fuzziness and uncertainty in data, making it particularly suitable for complex and diverse health data in the field of disease control, achieving better clustering results even in the absence of explicit relationships.

[0075] 3. High interpretability: Unlike existing deep learning or black-box models, the joint optimization strategy based on sequential learning and categorical data in this invention has good interpretability. By analyzing the relationship between the sequential and categorical data in the clustering results, the key features and factors in the clustering process can be intuitively understood, which helps disease control decision-makers make data-driven scientific decisions, especially in the evaluation and adjustment of prevention and control measures, providing more evidence-based decision support.

[0076] 4. Adapting to the specific needs of the disease control field: Data in the disease control field is often characterized by high timeliness, complex correlations, and diverse data dimensions. Existing general clustering methods are difficult to optimize for these characteristics. However, this invention designs a clustering method specifically for big data in disease control, which can better adapt to this complexity, especially when dealing with nonlinear and dynamically changing complex data such as the early stages of an outbreak, different stages of disease transmission, and policy implementation.

[0077] 5. High Efficiency and Scalability: This invention not only excels in clustering accuracy but also effectively improves computational efficiency. By optimizing the data processing flow and combining modern computing frameworks and algorithms, the system can rapidly process large-scale data, adapting to the characteristics of large data volumes and rapid updates in the disease control field. Furthermore, the system design features good scalability, allowing for adjustments and expansion of data sources and analysis models according to actual needs, providing flexible support for future application scenarios.

[0078] Example 2 Based on the above embodiment 1, combined with Figures 2-4 This embodiment elaborates on the category data clustering capability of the present invention.

[0079] In this embodiment, the disease control big data analysis method proposed in this invention is abbreviated as OCL. It employs three clustering methods—OCL, CBDM, and ADC—to learn from the VT dataset, calculate the distances between attribute values, and visualize the processed data in a two-dimensional space using the t-distributed random neighborhood embedding method (t-SNE). By comparing the clustering results of different methods, the significant advantages of this invention in categorical data clustering tasks are verified.

[0080] like Figure 2 As shown, we first preprocessed the VT dataset to ensure its integrity and validity. Then, we used OCL, CBDM, and ADC methods for clustering to obtain the distance structure between attribute values ​​learned by each method. These distance values ​​were then used to transform the original categorical data into numerical form for visualization analysis. Figures 2-4 The results of t-SNE visualization of the VT dataset are presented, showing the clustering effects of CBDM, ADC, and OCL methods, respectively. The visualization results clearly demonstrate that the OCL method of this invention performs superiorly in clustering. Compared to CBDM and ADC methods, OCL can more clearly distinguish samples of different categories, significantly reduces overlap between categories, and produces more compact cluster regions, exhibiting stronger clustering discrimination ability.

[0081] In particular, the OCL method exhibits clear class separation characteristics in its clustering results, indicating that OCL can effectively identify and optimize the potential class structure in the data. This performance, when processing the VT dataset, can optimize clustering results through sequential learning mechanisms and the sequential relationships between class data, even without explicit numerical relationships. In contrast, the CBDM and ADC methods show some class mixing, with unclear boundaries between different classes, leading to reduced clustering accuracy.

[0082] This experiment demonstrates the effectiveness and advantages of the method of this invention in clustering categorical data. Particularly in disease control big data analysis systems, precise cluster analysis can help to better understand the categorical relationships behind the data, thereby providing accurate decision support for disease prevention, epidemic trend analysis, and other areas. The experimental results verify the superiority of the technical solution of this invention in practical applications, providing a solid theoretical and empirical foundation for its application in the field of disease control big data.

[0083] Example 3 like Figure 5 As shown, the present invention also provides a disease control big data analysis system, which is used in the aforementioned disease control big data analysis method, and includes: a data preprocessing module, an optimal order search module, a clustering module, a clustering update module, and a disease control analysis module.

[0084] The data preprocessing module acquires and preprocesses a multi-source heterogeneous disease control dataset X reported from hospital information systems, laboratory information systems, and community systems. The preprocessed multi-source heterogeneous disease control dataset X includes multiple category attributes with business semantics. The optimal order search module receives the preprocessed dataset and performs an independent optimal order search within its cluster for each category attribute in the multi-source heterogeneous disease control dataset to obtain the optimal order of that category attribute. Based on the optimal order, it determines a distance structure suitable for disease control big data analysis. The clustering module calculates the distance metric between samples and clusters according to the obtained optimal order and constructs a clustering model. The clustering update module dynamically adjusts the clustering results by alternately updating the sample affiliation matrix and category attribute order by minimizing the objective function. The disease control analysis module outputs case clustering, transmission chain identification results, and risk assessment reports based on the clustering results.

[0085] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] Alternatively, if the above embodiments of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0087] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. The icons depicting structural positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for analyzing big data in disease control, characterized in that, Includes the following steps: S1. Obtain multi-source heterogeneous disease control dataset X reported from hospital information system, laboratory information system and community system and preprocess it. The preprocessed multi-source heterogeneous disease control dataset X includes multiple category attributes with business semantics. S2. For each category attribute in the multi-source heterogeneous disease control dataset, perform an independent optimal order search within its cluster to obtain the optimal order of the category attribute, and determine a distance structure suitable for disease control big data analysis based on the optimal order. S3. Based on the optimal order, calculate the distance metric between the sample and the cluster, and construct a clustering model; S4. By minimizing the objective function, the sample attribution matrix and the order of category attributes are updated alternately to dynamically adjust the clustering results; S5. Output case clustering, transmission chain identification results, and risk assessment reports based on clustering results.

2. The disease control big data analysis method according to claim 1, characterized in that, The category attributes with business semantics include at least one of the following: symptom attributes, diagnosis attributes, testing attributes, contact history attributes, vaccination attributes, and transmission association attributes; The preprocessed multi-source heterogeneous disease control dataset Depend on It consists of 10 data samples, each sample Represented as a 3D row vector, in the form of ,in The number of category attributes of the data samples is denoted as X. The preprocessed dataset X is used as the sample set of the clustering model in step S3. Together with the number of clusters to be divided K, it is used as the input of the clustering model. The dataset X is clustered into K clusters by the clustering model.

3. The disease control big data analysis method according to claim 2, characterized in that, In step S1, the preprocessing process includes: checking for missing and ambiguous values ​​and taking filling or deletion operations based on the correlation between the disease control business ledger and the data; performing a unified format conversion on the values ​​of all category attributes and encoding conversion on category variables; using label encoding to convert the data values ​​of category attributes into numerical data; constructing ordinal data based on variables with ordinal properties; and performing data verification after data reception and preprocessing, and conducting integrity checks through statistical description.

4. The disease control big data analysis method according to claim 1, characterized in that, Each category attribute There is an optimal order. An independent sequential search is performed for each category attribute, and the effect of different order combinations is evaluated using an objective function based on the clustering task. Specifically, the objective function is: in, Indicates sample with cluster The distance between them, where m represents the cluster index. This is the sample attribution matrix, indicating which cluster a sample in the dataset belongs to. Indicates the order of category attributes. Indicates the number of category attributes. This represents the ordinal distance between the sample category attribute value 'r' and other values. Indicates in cluster Category attributes The probability of its occurrence, Indicates in cluster Category attributes value The probability of its occurrence, Represents a collection of category attributes.

5. The disease control big data analysis method according to claim 4, characterized in that, The strategy of optimal order search within each cluster is adopted. The samples in each cluster are traversed, the distance between the sample and each possible value is calculated, and the relative importance of the value is evaluated based on these distances. By summarizing the performance of each attribute value in all clusters, the weighted average ranking of each attribute value is obtained. The specific content includes defining each attribute in a specific cluster. The optimal order within is By minimizing internal components To obtain the optimal order, the form is: in, Indicates the value to be Category attributes Cluster The contribution of the objective function, i.e., the category attribute The sum of the ordinal distances of all possible values. Indicates the sample in the category attribute The distance between the value on the index and the probability distribution of the cluster center is used to calculate the link strength of each category attribute value using the following formula. : Each category attribute value is sorted by weighted average based on its link strength. Category attribute values ​​with higher link strength are placed in the middle of the queue. Then, all local sorts are merged by cluster size, resulting in the globally optimal order. And sort the resulting rankings.

6. The disease control big data analysis method according to claim 5, characterized in that, In step S3, when calculating the distance metric between a sample and a cluster, it is first necessary to construct a distance vector. It is defined as a sample in terms of attributes The normalized ordinal distance between the value and each possible value: in It is an attribute The number of possible values, and Representing samples respectively and possible values The serial number.

7. The disease control big data analysis method according to claim 6, characterized in that, Step S4 includes the following specific steps: optimizing the clustering result by minimizing the clustering objective function, wherein minimizing the clustering objective function includes two sub-steps: fixing the sample assignment and update order, and fixing the attribute order and updating the sample assignment. The two sub-steps are performed alternately until the clustering objective function converges. Specifically, the fixed attribute order and updated sample attribution involve updating the sample attribution matrix after defining the distance based on the actual epidemiological investigation or testing results from the disease control center. The specific update rules are as follows: The update rule indicates that if the sample with cluster The sample belongs to the cluster with the smallest distance between them. ; The triggering condition for the fixed sample attribution and update order is: when an epidemic mutation occurs, i.e., the mutant strain causes a change in the optimal order, the link strength is recalculated. The order of category attributes is adjusted, the distance between all samples and clusters is recalculated, and the assignment of some samples is adjusted.

8. The disease control big data analysis method according to claim 7, characterized in that, The fixed sample attribution and update order specifically refers to: fixing the current sample attribution matrix. And by minimizing the objective function To optimize the attribute order By selecting, it can be minimized The order in which new attributes are obtained. and update the distance metric. The fixed attribute order and updated sample attribution specifically refer to: fixing the latest attribute order. And by minimizing the objective function To update the sample attribution matrix Based on the updated distance metric, the distance between each sample and all clusters is recalculated, and each sample is assigned to a specific cluster based on these distances.

9. A disease control big data analysis method according to claim 8, characterized in that, Step S5 includes the following specific steps: Based on the sample attribution matrix in the clustering results, the data samples are divided into different case groups. Based on the central features and attribute distribution of each cluster, a clinical feature description is generated for each case group. The clinical feature description includes the combination of main symptoms, the distribution of test results, and the pattern of contact history. Analyze the attribute order correlation between different clusters, and reconstruct the propagation path based on the contact history attributes and geographic time information of samples within the cluster, and mark the key nodes and diffusion direction in the propagation chain; Based on the size, attribute distribution characteristics, and transmission chain identification results of each cluster, the risk level index of each cluster is calculated. Combined with the epidemiological model, a risk assessment report is generated, which includes the size of the high-risk population, the transmission risk index, and early warning recommendations. The results of case clustering, transmission chain identification, and risk assessment reports are integrated into a structured output and converted into an instruction format that can be recognized by the disease control business system, automatically triggering the execution of corresponding prevention and control measures.

10. A disease control big data analysis system, wherein the system is used in the disease control big data analysis method according to any one of claims 1-9, characterized in that, It includes: a data preprocessing module, an optimal order search module, a clustering module, a clustering update module, and a disease control analysis module; The data preprocessing module acquires and preprocesses a multi-source heterogeneous disease control dataset X reported from hospital information systems, laboratory information systems, and community systems. The preprocessed multi-source heterogeneous disease control dataset X includes multiple category attributes with business semantics. The optimal order search module receives the preprocessed dataset and performs an independent optimal order search within its cluster for each category attribute in the multi-source heterogeneous disease control dataset to obtain the optimal order of that category attribute. Based on the optimal order, it determines a distance structure suitable for disease control big data analysis. The clustering module calculates the distance metric between samples and clusters according to the obtained optimal order and constructs a clustering model. The clustering update module dynamically adjusts the clustering results by alternately updating the sample affiliation matrix and category attribute order by minimizing the objective function. The disease control analysis module outputs case clustering, transmission chain identification results, and risk assessment reports based on the clustering results.

Images