Multi-dimensional integration and intelligent management method for silkworm germplasm resources

Abstract

The invention belongs to the technical field of data processing and management, particularly relates to a multi-dimensional integration and intelligent management method for silkworm germplasm resources, and aims to solve the problems of data fragmentation, non-standard processing, low classification accuracy, insufficient intelligence and the like in existing management. The method comprises the steps of collecting multi-source original data according to four dimensions and binding a unique traceability identifier; forming a standardized data set through abnormal value processing, standardized conversion and integrity verification; integrating and generating a complete data file, and extracting a single-dimensional core feature and a cross-dimensional correlation feature; a dynamic database is constructed after K-means clustering classification is improved; and an intelligent management platform is built to realize full-process management and control, and the model is periodically iterated and optimized. The system is correspondingly provided with a linkage module, and the unique traceability identifier is used as core associated data. Deep integration and standardized management of multi-source data are realized, classification accuracy and management intelligence level are improved, and a closed-loop mechanism is formed.

Description

Technical Field

[0001] This invention belongs to the field of data processing and management technology, and in particular relates to a multi-dimensional integration and intelligent management method for silkworm germplasm resources. Background Technology

[0002] Silkworm germplasm resources are the core material foundation for the sustainable development of the silk industry, carrying the genetic diversity and superior traits of silkworm varieties. Their protection, integration, and efficient utilization are directly related to innovation in silkworm breeding, improvement of industry quality and efficiency, and biodiversity conservation. With the transformation of the silkworm industry towards large-scale, precise, and intelligent production, and the widespread application of molecular biology techniques in breeding, the research dimensions of silkworm germplasm resources are constantly expanding, extending from traditional phenotypic identification to multiple levels such as genotypic analysis, ecological adaptability assessment, and quantification of breeding characteristics, resulting in massive, multi-source, and heterogeneous germplasm resource data.

[0003] Currently, the management of silkworm germplasm resources still faces many technical bottlenecks, making it difficult to meet the needs of high-quality development of the industry. Specific problems are as follows: First, data fragmentation is severe. Existing management models mostly store single-dimensional data (such as phenotypic data or genotypic data) independently. Multi-dimensional data such as phenotypic, genotypic, ecological adaptability and breeding characteristics lack effective correlation and integration, forming "data silos". This makes it impossible to fully reflect the comprehensive characteristics of germplasm resources, making it difficult for researchers to explore the potential correlations between different dimensional traits and restricting breeding efficiency.

[0004] Second, insufficient standardization in data processing, the heterogeneity of multi-source data (continuous, discrete, hierarchical) leads to inconsistent data formats, lack of standardized procedures for handling outliers and missing values, poor data accuracy and comparability, which hinders subsequent data analysis and application.

[0005] Third, the classification and archiving methods are crude. Traditional classification relies heavily on manual division based on experience or single traits, which is highly subjective, inefficient, and difficult to adapt to the comprehensive evaluation needs of multi-dimensional core characteristics, thus failing to achieve accurate classification and rapid retrieval of germplasm resources.

[0006] Fourth, the management system suffers from low dynamism and intelligence. Existing databases are mostly statically stored, requiring manual repetitive processing and entry of new germplasm resource data, making automated synchronous updates and dynamic expansion difficult. Furthermore, the lack of an integrated intelligent management platform results in fragmented functions such as resource querying, dynamic monitoring, and precise matching, failing to form a closed-loop mechanism of "data collection-processing-integration-management-optimization," leading to low utilization efficiency of germplasm resources. Fifth, the models and algorithms lack adaptability. Traditional clustering and feature extraction algorithms do not consider the industry characteristics of silkworm germplasm resources (such as traditional strain classification standards), easily leading to problems such as a disconnect between classification results and actual application needs, and insufficient accuracy in core feature extraction, making it difficult to support precision breeding and efficient resource utilization.

[0007] Therefore, in response to the pain points of existing silkworm germplasm resource management, such as data fragmentation, non-standard processing, inaccurate classification, and insufficient intelligent management, this invention aims to construct a multi-dimensional integration and intelligent management method to achieve standardized processing, deep integration, accurate classification, and dynamic intelligent management of multi-source data, explore potential correlations between traits, and improve the efficiency of germplasm resource protection and utilization. Summary of the Invention

[0008] The purpose of this invention is to provide a multi-dimensional integration and intelligent management method for silkworm germplasm resources, in order to solve the technical problems in existing silkworm germplasm resource management such as data fragmentation, non-standard processing, inaccurate classification, and insufficient intelligent management.

[0009] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for multi-dimensional integration and intelligent management of silkworm germplasm resources includes the following steps: S1: For the silkworm germplasm resource population, collect multi-source raw data according to phenotypic, genotypic, ecological adaptability and breeding characteristics, record the unique traceability identifier of each silkworm germplasm resource, and form a raw data set and corresponding traceability information table; S2: Preprocess the original dataset and traceability information table to form a standardized dataset. The standardized dataset and the traceability information table are established in a one-to-one correspondence through a unique traceability identifier. S3: Integrate the standardized dataset and corresponding traceability information table into a complete data archive by multi-dimensional data integration of silkworm germplasm resources, and extract the potential correlation features between core features of different dimensions to obtain the core feature set of silkworm germplasm resources; S4: Based on the core feature set and complete data archive, the improved K-means clustering algorithm is used to classify silkworm germplasm resources, and then a dynamic database of silkworm germplasm resources is constructed. At the same time, a data update interface is set to realize the dynamic expansion of the database. S5: Based on the dynamic database built by S4, a smart management platform for silkworm germplasm resources is established to achieve multi-dimensional management and record management operation logs; S6: Based on management operation logs and expanded data from dynamic databases, regularly iterate and optimize the data processing and management model.

[0010] Preferably, the specific process of step S2 is as follows: S21: Using the unique traceability identifier as an index, the original dataset is divided into four data subsets according to the dimensions of phenotype, genotype, ecological adaptability, and reproductive characteristics. Each subset is further divided into subsets according to specific indicators. For the numerical data subsets of each indicator, Grubbs' test is used to identify outliers. For the discrete marker data of the genotype dimension, abnormal marker sites are identified by comparing with known marker patterns of the same strain of silkworm. S22: To address the heterogeneity of data across different dimensions, a targeted algorithm is employed for standardization transformation. At the same time, the original units and transformation rules of each dimension of data are preserved. A one-to-one correspondence is established between the unique traceability identifier and the traceability information table to form a standardized dataset. S23: Perform integrity verification on the standardized data. If there is missing data, fill it with the mean of the corresponding dimension of the same type of silkworm germplasm resources and mark the filling mark. Establish a one-to-one correspondence between the standardized data of each dimension and the traceability information table through the unique traceability mark, and integrate them to form a standardized dataset. Each record in the dataset contains a unique traceability mark, standardized data of each dimension, original unit and transformation rules.

[0011] Preferably, the specific process of identifying outliers using the Grubbs test in step S21 is as follows: S211: Calculate the mean of a subset of data. Sum of standard deviations s: ; ; in, n The sample size of the data subset. x i For the first i The values ​​of each sample; S212: Calculate the Grubbs statistic G i : ; S213: The calculated result G i Critical value of Grubbs test G n,α Comparison, αIf we take 0.05, we can look it up in the Grubbs critical value table. G i > G n,α Then determine x i This is an outlier.

[0012] Preferably, the specific process of step S3 is as follows: S31: Generate a complete data archive through data matching and data integration operations; S32: For the standardized data of each dimension in the complete data archive, principal component analysis is used for dimensionality reduction to extract the core features of each dimension. S33: Cross-dimensional association feature mining based on association rule mining algorithm: Based on the extracted core features of each dimension, the potential association features between core features of different dimensions are extracted through association rule mining algorithm; S34: Generate the core feature set of silkworm germplasm resources: Integrate the extracted single-dimensional core features with the mined cross-dimensional correlation features to generate the core feature set of silkworm germplasm resources.

[0013] Preferably, the specific process of step S32 is as follows: S321: Data subset division: Based on the complete data archive, the data is divided into four subsets according to four dimensions: phenotype, genotype, ecological adaptability, and breeding characteristics. Each data subset contains standardized data of all silkworm germplasm resources under that dimension. S322: For each dimension of the data subset, calculate its covariance matrix; S323: Solving the covariance matrix C The eigenvalues ​​and corresponding eigenvectors are then assigned, and the eigenvalues ​​are arranged in descending order. S324: Calculate the contribution rate and cumulative contribution rate for each feature value, and retain the top features with a cumulative contribution rate ≥ 85%. k Each principal component serves as the core feature of that dimension, and each principal component corresponds to a feature vector, representing the core information of that dimension. S325: Map the selected core features of each dimension to the corresponding records in the complete data archive, and label the original indicators and contribution rates of each core feature.

[0014] Preferably, the specific process of step S33 is as follows: S331: Using the unique traceability identifier in the complete data archive as an index, the core features of each dimension are integrated to construct an association rule dataset. Each record in the dataset corresponds to a multi-dimensional core feature combination of silkworm germplasm resources. S332: An association rule mining algorithm is used to perform frequent itemset mining on the association rule dataset. The minimum support is set to 5% and the minimum confidence is set to 80%, and the core feature combinations that appear simultaneously in multiple silkworm germplasm resource individuals are mined. S333: Extract cross-dimensional association rules from the frequent itemset as cross-dimensional association features. The form of the association rules is: core features of dimension A → core features of dimension B. S334: Validate the extracted cross-dimensional association features and filter out association features with a lift greater than 1 by calculating the lift of the association rules.

[0015] Preferably, the specific process of step S34 is as follows: S341: Summarize the single-dimensional core features and cross-dimensional related features of each dimension to form a preliminary version of the core feature set; S342: Establish a mapping relationship between the core feature set and the complete data archive, and label each core feature with a unique traceability identifier, original data indicator and contribution rate; S343: Output the final set of core features of silkworm germplasm resources. This set includes single-dimensional core features and cross-dimensional related features, and maintains a one-to-one mapping relationship with the complete data archive.

[0016] Preferably, the specific process of step S4 is as follows: S41: Accurate classification of silkworm germplasm resources based on improved K-means clustering algorithm: S42: Establish the overall architecture of the dynamic database for silkworm germplasm resources, including the basic archive module: used to store the complete data archive output by S3, with each record using a unique traceability identifier as the primary key; the core feature module: used to store the core feature set, with each record bound to a unique traceability identifier; the classification result module: used to store classification labels and classification criteria, with each record bound to a unique traceability identifier; the traceability tracking module: used to store traceability information throughout the entire process; and establish a unified database management engine. S43: Develop data update interfaces and dynamically expand the database.

[0017] Preferably, the specific process of step S41 is as follows: S411: Data Preprocessing and Algorithm Optimization Parameter Setting: Organize the core feature set into a clustering feature matrix; introduce prior rules and optimize the algorithm; S412: Germplasm resource classification is implemented based on an improved K-means clustering algorithm; S413: Assign a unique category code and category name to each cluster category to form a standardized classification label.

[0018] Preferably, the multidimensional management in step S5 includes: Resource Query: Quickly retrieve complete information on corresponding silkworm germplasm resources by using unique traceability identifiers, classification tags, or core feature keywords; Dynamic monitoring: Real-time collection of new and existing data changes in the database, dynamic updates to classification results, and automatic triggering of re-clustering and classification when the core feature set changes significantly; Precise matching: Based on user needs, extract the characteristics of the needs and perform similarity calculation with the core feature set in the dynamic database, and output the optimal matching list of silkworm germplasm resources and adaptation analysis report; The intelligent management platform generates operation logs to record the data entry, query, update, and matching processes, enabling full-process traceability.

[0019] The beneficial effects of this invention include: 1. This invention uses a unique traceability identifier as the core index to integrate multi-dimensional data on phenotype, genotype, ecological adaptability, and reproductive characteristics with traceability information, forming a complete data archive for each individual and breaking down "data silos." Simultaneously, it extracts potential cross-dimensional correlation features through association rule mining algorithms, clarifying the intrinsic relationships between different dimensional traits (such as genotype and silk production performance, ecological adaptability and reproductive characteristics), providing data support for precision breeding and filling the gap in existing technologies that lack multi-dimensional data integration and association mining.

[0020] 2. Outliers are accurately identified using the Grubbs test and verified by tracing the source information. Targeted standardization algorithms are employed to process heterogeneous data, while missing data is imputed and labeled, resulting in a standardized dataset. This unified data processing workflow ensures the accuracy, standardization, and comparability of the data, laying a solid data foundation for subsequent feature extraction, clustering, classification, and intelligent management, and addressing the lack of standardization in existing data processing technologies.

[0021] 3. An improved K-means clustering algorithm is adopted, incorporating traditional silkworm strain classification standards as prior information to optimize initial cluster centers. The clustering effect is verified using silhouette coefficients, achieving accurate classification of germplasm resources. The classification labels integrate algorithmic features with traditional strain names, ensuring clear and traceable classification criteria. This guarantees both objectivity and accuracy while aligning with industry understanding and practical application scenarios, significantly improving efficiency and applicability.

[0022] 4. The established dynamic database comprises four core modules. A unified management engine enables data linkage and efficient scheduling. The data update interface supports the automatic processing and synchronous insertion of newly added resources, achieving dynamic database expansion. The intelligent management platform integrates resource query, dynamic monitoring, and precise matching functions. Combined with operation logs, it achieves full-process traceability, forming a closed-loop mechanism of "collection-processing-integration-management-optimization." This significantly improves the intelligence level and utilization efficiency of germplasm resource management, overcoming the limitations of existing static management models.

[0023] 5. Based on management operation logs and expanded data, regularly optimize data processing and management models, adjust algorithm parameters and model structure to ensure that the model always adapts to the characteristics of newly added germplasm resources and changes in industry needs. The optimized model can continuously improve feature extraction accuracy, clustering accuracy, and matching rationality, providing stable technical support for long-term breeding innovation, germplasm resource protection, and industrial quality improvement and efficiency enhancement. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the multi-dimensional integration and intelligent management method for silkworm germplasm resources according to the present invention.

[0025] Figure 2 This is a flowchart illustrating step S3 in one embodiment of the present invention. Detailed Implementation

[0026] The following is in conjunction with the appendix Figures 1-2 The present invention will be further described in detail below: Example 1 See appendix Figure 1 As shown, a multi-dimensional integration and intelligent management method for silkworm germplasm resources includes the following steps: S1: For silkworm germplasm resource populations, collect multi-source raw data according to phenotypic, genotypic, ecological adaptability and breeding characteristics.

[0027] The original data for the phenotypic dimension includes cocoon shape parameters, body color characteristics, developmental stage, and silk production performance indicators. The original data for the genotypic dimension includes SSR molecular marker locus information, SNP site polymorphism data, and functional gene sequence fragments. The original data for the ecological adaptability dimension includes survival rate, feeding efficiency, and stress resistance score under different temperature and humidity gradients. The original data for the reproductive characteristics dimension includes mating success rate, egg production, and egg hatching rate. During the collection process, a unique traceability identifier for each silkworm germplasm resource is recorded simultaneously. The unique traceability identifier is bound to the collection location, collection time, and strain number of the silkworm germplasm resource to form an original data set and a corresponding traceability information table.

[0028] S2: Based on the original dataset and the traceability information table, outliers are first removed from the original data. The Grubbs test is used to identify and mark outliers in each dimension of the data. The authenticity of the outliers is verified in conjunction with the traceability information table. Outliers confirmed as collection errors are removed, and real outliers are retained separately and labeled with characteristics. Then, to address the heterogeneity of the data in different dimensions, a standardization transformation is performed. Continuous data in the phenotypic dimension is transformed into normalized data in the [0,1] interval. Discrete labeled data in the genotypic dimension is transformed into binary encoded data. The grade score data in the ecological adaptability and reproductive characteristics dimensions is transformed into quantitative values. At the same time, the original units and transformation rules of each dimension of the data are retained to form a standardized dataset. The standardized dataset and the traceability information table are established in a one-to-one correspondence through a unique traceability identifier.

[0029] S3: Multi-dimensional Data Integration and Feature Extraction: Based on a standardized dataset and corresponding traceability information table, a multi-dimensional data integration model for silkworm germplasm resources is constructed. First, using the unique traceability identifier as the core index, the data of each dimension in the standardized dataset and the traceability information in the traceability information table are linked and integrated to form a complete data archive of a single individual silkworm germplasm resource. Then, feature extraction is performed on the integrated complete data archive. Principal component analysis is used to reduce the dimensionality of redundant data in each dimension, retaining the principal components with a cumulative contribution rate of ≥85% as the core features of each dimension. At the same time, through association rule mining algorithms, potential association features between core features of different dimensions are extracted to obtain the core feature set of silkworm germplasm resources. The core feature set includes single-dimensional core features and cross-dimensional association features, and maintains a mapping relationship with the complete data archive.

[0030] S4: Intelligent Classification and Archiving and Dynamic Database Construction: Based on the core feature set and complete data archives, an improved K-means clustering algorithm is first used to classify silkworm germplasm resources. Using the core feature set as the clustering basis, combined with the traditional strain classification standards of silkworm germplasm resources, the initial value of the cluster centers is optimized to group silkworm germplasm resources with similar core features into the same category, generating classification labels and classification criteria. Then, a dynamic database of silkworm germplasm resources is constructed. The dynamic database includes a basic archive module, a core feature module, a classification result module, and a traceability module, which respectively store complete data archives, core feature sets, classification labels, and traceability information. At the same time, a data update interface is set up to support the synchronous entry of newly added silkworm germplasm resource data into the database after processing in steps S2 and S3, realizing the dynamic expansion of the database.

[0031] S5: Intelligent Management and Precision Services: Based on the dynamic database built by S4, an intelligent management platform for silkworm germplasm resources is established to achieve multi-dimensional management.

[0032] S6: Data Iteration Optimization and Model Upgrade: Based on the intelligent management operation logs and expanded data of the dynamic database in S5, the data processing and management model is iteratively optimized regularly. First, the accuracy and efficiency of data processing in each step are statistically analyzed, and the corresponding algorithm parameters are adjusted for deviations that occur in the standardization transformation and feature extraction processes. Then, based on the newly added silkworm germplasm resource data, the multi-dimensional data integration model and intelligent classification algorithm are trained and optimized, and the core feature extraction standards and cluster centers are updated to adapt the model to the characteristics of the new resources. Finally, the optimized parameters and model are synchronized to the intelligent management platform and dynamic database to form a closed-loop mechanism of "data collection-processing-integration-management-optimization".

[0033] Example 2 Based on Example 1, the specific process of step S2 is as follows: S21: Using the unique traceability identifier as an index, the original dataset is divided into four data subsets according to the phenotypic dimension, genotypic dimension, ecological adaptability dimension, and reproductive characteristic dimension. Each subset is further divided into subsets according to specific indicators, including cocoon shape parameters and SSR molecular marker site information. For the numerical data subset of each indicator, the Grubbs test is used to identify outliers.

[0034] For discrete marker data at the genotype level, such as SNP polymorphism data, aberrant marker sites can be identified by comparing them with known marker patterns of the same strain of silkworm.

[0035] By combining the collection location, collection time, strain number, and collection personnel information in the traceability information table, the authenticity of abnormal data is verified. If the traceability information of abnormal data is complete and does not conflict with other resources collected in the same batch, it is confirmed as real abnormal data. If the traceability information is missing or contradicts the collection record, it is confirmed as a collection error.

[0036] Outliers caused by collection errors are removed; for real abnormal data, they are retained separately and labeled with "special abnormal characteristics", and associated with a unique traceability identifier, and then incorporated into the subsequent standardized transformation process.

[0037] S22: Based on the original data subset after outlier removal, a targeted algorithm is used to standardize the data according to the heterogeneity of different dimensions, while retaining the original units and transformation rules of each dimension. Finally, a one-to-one correspondence is established between the unique traceability identifier and the traceability information table to form a standardized dataset.

[0038] Continuous phenotypic data, such as cocoon shape parameters, developmental duration, and silk production performance indicators, are converted into normalized data in the [0,1] interval. The specific formula is as follows: ; in,x norm,i For the first i Normalized values ​​of each sample, x i These are the original values ​​after outlier removal. x min The minimum value of this subset of data. x max This represents the maximum value of the subset of data.

[0039] Discrete marker data based on type dimension, such as SSR molecular marker site information and SNP site polymorphism data, are converted into binary encoded data. For SSR molecular marker site information: the site is encoded according to the number of alleles. If a site has two alleles, A and a, then the homozygous AA is encoded as (1,0), the homozygous aa is encoded as (0,1), and the heterozygous Aa is encoded as (1,1).

[0040] For SNP site polymorphism data: encoding is performed according to base type. If the base of a certain site is A, T, C, or G, it is encoded as 00, 01, 10, or 11, respectively. After encoding, a binary matrix is ​​formed. The rows of the matrix correspond to the unique traceability identifier of the silkworm germplasm resource, the columns correspond to the marked sites, and the elements are binary encoded values.

[0041] The rating data for ecological adaptability and reproductive characteristics, including stress resistance scores and mating success rate scores, were converted into quantitative values ​​using a linear interpolation algorithm. x quan,i = ( x i -x score,min ) / ( x score,max -x score,min )×10; in, x quan,i For the first i Quantified values ​​of a sample x i The original score after outlier removal. x score,min This is the lowest possible score for that rating indicator (e.g., 1 point). x score,max The highest score for this rating indicator (e.g., 10 points); S23: Perform integrity verification on the standardized data. If there is missing data, fill it with the mean of the corresponding dimension of the same type of silkworm germplasm resources and mark the filling mark. Establish a one-to-one correspondence between the standardized data of each dimension and the traceability information table through the unique traceability mark, and integrate them to form a standardized dataset. Each record in the dataset contains a unique traceability mark, standardized data of each dimension, original unit and transformation rules.

[0042] The specific process of identifying outliers using the Grubbs test in step S21 is as follows: S211: Calculate the mean of a subset of data. Sum of standard deviations s: ; ; in, n The sample size of the data subset. x i For the first i The values ​​of each sample; S212: Calculate the Grubbs statistic G i : ; S213: Perform outlier detection: The calculated G i Critical value of Grubbs test G n,α Comparison, α If we take 0.05, we can look it up in the Grubbs critical value table. G i > G n,α Then determine x i This is an outlier.

[0043] Example 3 Based on Example 1 or Example 2, see Figure 2 As shown, the specific process of steps S3 is as follows: S31: Construct a multi-dimensional data integration model and perform the following operations to generate a complete data archive: Data matching: Traverse all records in the standardized dataset and use the unique traceability identifier as the matching condition to find the corresponding traceability information record in the traceability information table.

[0044] Data integration: The matched traceability information (collection location, collection time, strain number) is integrated with the standardized data of four dimensions (phenotypic dimension [0,1] normalized data, genotype dimension binary encoded data, ecological adaptability dimension quantitative value, and breeding characteristic dimension quantitative value) to form a complete data record with a unique traceability identifier as the primary key.

[0045] Archive generation: The complete data records of all silkworm germplasm resources are summarized to generate a complete data archive of a single individual silkworm germplasm resource. Each record in the archive corresponds to a silkworm germplasm resource and contains all traceability information and multi-dimensional standardized data of the resource.

[0046] S32: For the standardized data of each dimension in the complete data archive, principal component analysis (PCA) is used to perform dimensionality reduction and extract the core features of each dimension.

[0047] S33: Cross-dimensional association feature mining based on association rule mining algorithm: Based on the core features of each dimension extracted in S32, the potential association features between core features of different dimensions are extracted through association rule mining algorithm.

[0048] S34: Generate the core feature set of silkworm germplasm resources: Integrate the single-dimensional core features extracted in S32 with the cross-dimensional correlation features mined in S33 to generate the core feature set of silkworm germplasm resources.

[0049] The specific process of step S32 is as follows: S321: Data subset division: Based on the complete data archive, the data is divided into four subsets according to four dimensions: phenotype, genotype, ecological adaptability, and breeding characteristics. Each data subset contains standardized data of all silkworm germplasm resources under that dimension. S322: Covariance Matrix Calculation: For each dimension of the data subset, calculate its covariance matrix C, using the following formula: ; Where X is a subset matrix of data for this dimension, with rows representing individual silkworm germplasm resources; and columns representing specific indicators for this dimension. Let be the mean vector of the data subset. n The number of individuals in the silkworm germplasm resource.

[0050] S323: Eigenvalue and Eigenvector Solving: Solving the Covariance Matrix C eigenvalues λ i and the corresponding feature vectors e i And arrange the eigenvalues ​​in descending order.

[0051] S324: Principal Component Screening: Calculate the contribution rate and cumulative contribution rate for each eigenvalue. The contribution rate formula is: ; in, m The number of indicators for this dimension. η i For the first i The contribution rate of each principal component is used to retain the top components with a cumulative contribution rate ≥ 85%. k Each principal component serves as the core feature of that dimension, and each principal component corresponds to a feature vector, representing the core information of that dimension.

[0052] S325: Core Feature Mapping: Map the selected core features of each dimension to the corresponding records in the complete data archive, and label the original indicators and contribution rates of each core feature to ensure that the core features can be traced back to the complete data archive.

[0053] The specific process of step S33 is as follows: S331: Construction of Association Rule Dataset: Using the unique traceability identifier in the complete data archive as an index, the core features of each dimension are integrated to construct an association rule dataset. Each record in the dataset corresponds to a multi-dimensional core feature combination of silkworm germplasm resources.

[0054] S332: Frequent itemset mining: The association rule mining algorithm, such as the Apriori algorithm, is used to mine frequent itemsets in the association rule dataset. The minimum support is set to 5% and the minimum confidence is set to 80%. The core feature combinations that appear simultaneously in multiple silkworm germplasm resource individuals are mined out, which are the frequent itemsets.

[0055] S333: Association Feature Extraction: Extract cross-dimensional association rules from frequent itemsets as cross-dimensional association features. The form of the association rules is "core feature of dimension A → core feature of dimension B", such as "core feature of SSR molecular marker in genotype dimension → core feature of silk production performance in phenotype dimension" and "core feature of stress resistance in ecological adaptability dimension → core feature of egg production in reproductive characteristics dimension".

[0056] S334: Validation of Association Features: Validate the extracted cross-dimensional association features. By calculating the lift of the association rules, select association features with a lift > 1 to ensure that the association features are statistically significant and not random associations.

[0057] The specific process of step S34 is as follows: S341: Feature Integration: Summarize the single-dimensional core features of each dimension (which may be the first 3 principal components of the tabular dimension or the first 2 principal components of the genotype dimension) and cross-dimensional association features (which may be genotype-phenotype association features or ecological adaptability-breeding characteristic association features) to form a preliminary version of the core feature set.

[0058] S342: Feature Mapping Relationship Establishment: Establish a mapping relationship between the core feature set and the complete data archive, and label each core feature with a unique traceability identifier, original data indicators and contribution rate to ensure that the core feature set can be traced back to the complete data archive.

[0059] S343: Core Feature Set Output: Outputs the final core feature set of silkworm germplasm resources. This set includes single-dimensional core features and cross-dimensional related features, and maintains a one-to-one mapping relationship with the complete data archive, providing a data foundation for the intelligent classification and archiving in the subsequent S4 step.

[0060] Example 4 Based on Example 1, Example 2, or Example 3, the specific process of step S4 is as follows: S41: Accurate classification of silkworm germplasm resources based on improved K-means clustering algorithm: S42: Establish the overall architecture for a dynamic database of silkworm germplasm resources, including: Basic Archive Module: Used to store complete data archives output by S3. Each record uses a unique traceability identifier as the primary key. The stored content includes: germplasm resource traceability information (collection location, collection time, strain number), phenotypic dimension [0,1] normalized data, genotype dimension binary encoded data, ecological adaptability dimension quantitative value, and breeding characteristic dimension quantitative value. At the same time, the original data units and data transformation rules are retained to realize the full storage of basic germplasm resource information.

[0061] Core Feature Module: Used to store the core feature set output by S3. Each record is bound to a unique traceability identifier. The stored content includes: single-dimensional core features (principal component feature values, cumulative contribution rate) and cross-dimensional correlation features (association rules, lift). It also establishes a mapping relationship between core features and the original indicators of the basic archive module, and supports the reverse tracing of core features back to the original data.

[0062] The classification results module stores the classification labels and classification criteria output by S4. Each record is bound to a unique traceability identifier. The stored content includes: the category code of the germplasm resource, the category name, and the full text of the classification criteria. It also reserves a "classification result update field" to support the synchronous overwriting of classification labels after subsequent dynamic clustering updates.

[0063] The traceability module stores the entire traceability information process, integrating the traceability information table of S1, the preprocessing operation log of S2, the feature extraction log of S3, and the clustering and classification log of S4. Each record is bound to a unique traceability identifier, realizing full-link traceability from raw data collection → standardization transformation → feature extraction → classification and archiving, and meeting the traceability compliance requirements of germplasm resource management.

[0064] Build a unified database management engine: A unified data management engine is built for the four core modules, enabling global data scheduling, access control, and data verification functions for the database. Core configuration: ① Establish data consistency verification rules to ensure that there are no data conflicts for the same unique traceability identifier in the four major modules; ② Configure a data indexing system, and build a multi-dimensional index based on unique traceability identifiers, category tags, and core feature keywords to improve the query efficiency of S5 in the future; ③ Reserved module expansion interfaces to support seamless integration of newly added functional modules (such as breeding evaluation modules and data analysis modules).

[0065] S43: Development of Data Update Interface and Implementation of Dynamic Database Expansion: S431: Develop standardized data update interfaces: Develop a dedicated data update interface adapted to this method. The interface is in the form of a standardized API and has two core functions to adapt to the data access requirements of newly added germplasm resources: Data access function: Supports batch uploading of raw data (phenotype, genotype, ecological adaptability, breeding characteristics) and traceability information tables of newly added silkworm germplasm resources. The interface automatically generates a unique traceability identifier for each newly added germplasm resource (using the combination of QR code and numerical code from S1) and completes data format verification, rejecting data with incorrect format.

[0066] Data flow function: The interface has built-in process call rules. After new data is added, the S2 "raw data preprocessing and standardization transformation" process and the S3 "multi-dimensional data integration and feature extraction" process will be automatically triggered without manual intervention, ensuring that the processing standards of new data are completely consistent with those of existing data.

[0067] S432: Dynamic Database Expansion New data automated processing: After new silkworm germplasm resource data is uploaded through the data update interface, the interface automatically executes step S2 (outlier removal, standardization transformation, and integrity verification) to generate a standardized dataset, and then automatically executes step S3 (multi-dimensional integration, PCA feature extraction, and association rule mining) to generate a core feature set and a complete data archive.

[0068] Synchronous execution of classification and archiving: After the new data has been processed by S2 and S3, the interface automatically pushes it to the improved K-means clustering module of S4.1, completes incremental clustering based on the existing cluster centers, matches the corresponding classification labels for the new germplasm resources, and generates classification basis.

[0069] Full data synchronization into the database: The complete data archive, core feature set, and classification tags of newly added data will be synchronously entered into the corresponding module of the dynamic database through the data update interface. The traceability module automatically records the entire process log of the newly added data, realizing the database's dynamic expansion without any awareness.

[0070] Dynamic updates to existing data: When the amount of new data reaches a preset threshold (e.g., a cumulative increase of ≥50 data entries), the interface will automatically trigger a full re-clustering. Based on the core feature set of existing and newly added data, the cluster centers and classification labels of all germplasm resources will be updated to ensure that the classification results always match the latest germplasm resource characteristics.

[0071] The specific process of step S41 is as follows: S411: Data Preprocessing and Algorithm Optimization Parameter Settings: The core feature set is organized into a clustering feature matrix, with each row of the matrix corresponding to a single silkworm germplasm resource (bound to a unique traceability identifier), and each column corresponding to a dimension of the core feature. The complete data archive of S3 is retrieved simultaneously as a supplementary basis for verifying the clustering results. Introducing prior rules: Extracting traditional classification standards for silkworm germplasm resources (industry-standardized, such as high fiber density lines and long filament lines based on silk production performance; high temperature resistant lines and disease resistant lines based on ecological adaptability; and high egg production lines and high hatching rate lines based on reproductive characteristics), and converting these standards into prior feature vectors of categories that can be recognized by the algorithm; Core optimization of the method: Addressing the shortcomings of traditional K-means clustering (random initial cluster centers, susceptibility to local optima, and significant discrepancies between clustering results and industry consensus), two key optimizations were implemented: The prior feature vectors of traditional strain classification are used as the initial cluster centers, replacing the random initialization method; Set clustering constraints: The clustering results must match the traditional strain classification by ≥80% to ensure that the algorithm classification meets the actual application requirements; S412: Accurate classification of germplasm resources based on an improved K-means clustering algorithm. Determining the optimal number of clusters K: Based on the total number of categories in the traditional classification of silkworm germplasm resources, and with the silhouette coefficient method used for verification, the optimal number of clusters K was finally determined (a common value K = 6~10, suitable for conventional silkworm strain classification); the silhouette coefficient calculation formula is as follows: ; The clustering effect is considered valid if the average silhouette coefficient of the final clustering result is ≥0.7. Core clustering iterative calculation: Step 1: Using the optimized initial cluster centers as a benchmark, calculate the Euclidean distance from the core feature vector of each germplasm resource in the cluster feature matrix to each cluster center. The formula is: ; in x im For the first i Germplasm resources m One core feature value, c jm For the first j The first cluster center m 1 eigenvalue, n The total number of core features; Step 2: Assign each germplasm resource to the category of the nearest cluster center to complete the initial clustering.

[0072] Step 3: Recalculate the mean value of each cluster feature and update the cluster centers; repeat the "distance calculation - category division - center update" iterative process until the cluster center change rate is ≤0.001, or the preset number of iterations is reached (default 500 times), then stop the iteration and output the final clustering result.

[0073] S413: Generation of Classification Labels and Classification Basis: Forming Standardized Classification Results Generate classification labels: Assign a unique category code and category name to each cluster category to form a standardized classification label; the label naming combines the clustering characteristics of the algorithm with the traditional strain name, for example: 01-high fiber length, high temperature resistance, high egg production strain, 02-high fiber density, disease resistance, high hatching strain. The classification label is bound to the unique traceability identifier of each germplasm resource.

[0074] Generate classification criteria: Write standardized classification criteria for each cluster category, including: ① the threshold range of the core characteristics of the category (e.g., core characteristic value of silk production performance ≥ 0.85, quantitative characteristic value of stress resistance ≥ 8.0); ② the characteristics that differentiate it from other categories; ③ the classification of the corresponding traditional strains; the classification criteria and classification labels correspond one-to-one to form a complete classification result file, which serves as the core data for the dynamic database.

[0075] The multidimensional management of step S5 includes: Resource Query: Quickly retrieve complete information on corresponding silkworm germplasm resources by using unique traceability identifiers, classification tags, or core feature keywords; Dynamic monitoring: Real-time collection of new and existing data changes in the database, dynamic updates to classification results, and automatic triggering of re-clustering and classification when the core feature set changes significantly; Precise matching: Based on user needs (including breeding goals, ecological adaptation scenarios and breeding requirements), extract the characteristics of the needs and perform similarity calculation with the core feature set in the dynamic database, and output the optimal matching list of silkworm germplasm resources and adaptation analysis report. The intelligent management platform generates operation logs to record the data entry, query, update, and matching processes, enabling full-process traceability.

[0076] This invention also provides a system for realizing multi-dimensional integration and intelligent management of silkworm germplasm resources, including: a multi-source data acquisition module, a data preprocessing and standardization module, a multi-dimensional data integration and feature extraction module, an intelligent classification and dynamic database module, an intelligent management platform module, and a model iteration and optimization module. Each module is linked in sequence, and a unique traceability identifier is used as the core of global data association to realize full-process management of silkworm germplasm resources. The multi-source data acquisition module is used to collect multi-source raw data for silkworm germplasm resource populations according to phenotypic, genotypic, ecological adaptability and breeding characteristics, and synchronously record the unique traceability identifier of each silkworm germplasm resource. The unique traceability identifier is bound to the collection location, collection time and strain number, and outputs the raw data set and the corresponding traceability information table. The data preprocessing and standardization module is communicatively connected to the multi-source data acquisition module. It is used to receive the raw data set and the traceability information table, complete the outlier identification and processing, heterogeneous data standardization transformation and data integrity verification according to the preset process, and output a standardized dataset. The standardized dataset and the traceability information table are established in a one-to-one correspondence through a unique traceability identifier. The multi-dimensional data integration and feature extraction module is connected to the data preprocessing and standardization module. It is used to receive standardized datasets and traceability information tables, construct a multi-dimensional data integration model, and generate a complete data archive of a single individual silkworm germplasm resource. Principal component analysis is used to extract core features of each dimension, and association rule mining algorithm is used to extract potential cross-dimensional association features. The core feature set is output, and the core feature set maintains a one-to-one mapping relationship with the complete data archive. The intelligent classification and dynamic database module communicates with the multi-dimensional data integration and feature extraction module to receive the core feature set and complete data archive. It uses an improved K-means clustering algorithm to accurately classify silkworm germplasm resources, generate classification labels and classification criteria, construct a dynamic database including a basic archive module, a core feature module, a classification result module and a traceability module, build a unified data management engine, develop a data update interface, and realize dynamic expansion of the database. The intelligent management platform module communicates with the intelligent classification and dynamic database module, and is used to realize resource query, dynamic monitoring and accurate matching functions based on the dynamic database. It synchronously generates management operation logs to record the entire process of data entry, query, update and matching, so as to achieve full traceability. The model iteration and optimization module is connected to the intelligent management platform module and the intelligent classification and dynamic database module, respectively. It is used to receive management operation logs and dynamic database expansion data, periodically adjust the parameters and optimize the structure of the data processing algorithm and management model, and synchronize the optimized model and parameters to the corresponding modules to form a closed-loop optimization mechanism.

Claims

1. A method for multi-dimensional integration and intelligent management of silkworm germplasm resources, characterized in that, Includes the following steps: S1: For the silkworm germplasm resource population, collect multi-source raw data according to phenotypic, genotypic, ecological adaptability and breeding characteristics, record the unique traceability identifier of each silkworm germplasm resource, and form a raw data set and corresponding traceability information table; S2: Preprocess the original dataset and traceability information table to form a standardized dataset. The standardized dataset and the traceability information table are established in a one-to-one correspondence through a unique traceability identifier. S3: Integrate the standardized dataset and corresponding traceability information table into a complete data archive by multi-dimensional data integration of silkworm germplasm resources, and extract the potential correlation features between core features of different dimensions to obtain the core feature set of silkworm germplasm resources; S4: Based on the core feature set and complete data archive, the improved K-means clustering algorithm is used to classify silkworm germplasm resources, and then a dynamic database of silkworm germplasm resources is constructed. At the same time, a data update interface is set to realize the dynamic expansion of the database. S5: Based on the dynamic database built by S4, a smart management platform for silkworm germplasm resources is established to achieve multi-dimensional management and record management operation logs; S6: Based on management operation logs and expanded data from dynamic databases, regularly iterate and optimize the data processing and management model.

2. The method for multi-dimensional integration and intelligent management of silkworm germplasm resources according to claim 1, characterized in that, The specific process of step S2 is as follows: S21: Using the unique traceability identifier as an index, the original dataset is divided into four data subsets according to the dimensions of phenotype, genotype, ecological adaptability, and reproductive characteristics. Each subset is further divided into subsets according to specific indicators. For the numerical data subsets of each indicator, Grubbs' test is used to identify outliers. For the discrete marker data of the genotype dimension, abnormal marker sites are identified by comparing with known marker patterns of the same strain of silkworm. S22: To address the heterogeneity of data across different dimensions, a targeted algorithm is employed for standardization transformation. At the same time, the original units and transformation rules of each dimension of data are preserved. A one-to-one correspondence is established between the unique traceability identifier and the traceability information table to form a standardized dataset. S23: Perform integrity verification on the standardized data. If there is missing data, fill it with the mean of the corresponding dimension of the same type of silkworm germplasm resources and mark the filling mark. Establish a one-to-one correspondence between the standardized data of each dimension and the traceability information table through the unique traceability mark, and integrate them to form a standardized dataset. Each record in the dataset contains a unique traceability mark, standardized data of each dimension, original unit and transformation rules.

3. The method for multi-dimensional integration and intelligent management of silkworm germplasm resources according to claim 2, characterized in that, The specific process of identifying outliers using the Grubbs test in step S21 is as follows: S211: Calculate the mean of a subset of data. Sum of standard deviations s: ； ； in, n The sample size of the data subset. x i For the first i The values ​​of each sample; S212: Calculate the Grubbs statistic G i : ； S213: The calculated result G i Critical value of Grubbs test G n,α Comparison, α If we take 0.05, we can look it up in the Grubbs critical value table. G i > G n,α Then determine x i This is an outlier.

4. The method for multi-dimensional integration and intelligent management of silkworm germplasm resources according to claim 1, characterized in that, The specific steps and S3 are as follows: S31: Generate a complete data archive through data matching and data integration operations; S32: For the standardized data of each dimension in the complete data archive, principal component analysis is used for dimensionality reduction to extract the core features of each dimension. S33: Cross-dimensional association feature mining based on association rule mining algorithm: Based on the extracted core features of each dimension, the potential association features between core features of different dimensions are extracted through association rule mining algorithm; S34: Generate the core feature set of silkworm germplasm resources: Integrate the extracted single-dimensional core features with the mined cross-dimensional correlation features to generate the core feature set of silkworm germplasm resources.

5. The method for multi-dimensional integration and intelligent management of silkworm germplasm resources according to claim 4, characterized in that, The specific process of step S32 is as follows: S321: Data subset division: Based on the complete data archive, the data is divided into four subsets according to four dimensions: phenotype, genotype, ecological adaptability, and breeding characteristics. Each data subset contains standardized data of all silkworm germplasm resources under that dimension. S322: For each dimension of the data subset, calculate its covariance matrix; S323: Solving the covariance matrix C The eigenvalues ​​and corresponding eigenvectors are then assigned, and the eigenvalues ​​are arranged in descending order. S324: Calculate the contribution rate and cumulative contribution rate for each feature value, and retain the top features with a cumulative contribution rate ≥ 85%. k Each principal component serves as the core feature of that dimension, and each principal component corresponds to a feature vector, representing the core information of that dimension. S325: Map the selected core features of each dimension to the corresponding records in the complete data archive, and label the original indicators and contribution rates of each core feature.

6. The method for multi-dimensional integration and intelligent management of silkworm germplasm resources according to claim 4, characterized in that, The specific process of step S33 is as follows: S331: Using the unique traceability identifier in the complete data archive as an index, the core features of each dimension are integrated to construct an association rule dataset. Each record in the dataset corresponds to a multi-dimensional core feature combination of silkworm germplasm resources. S332: An association rule mining algorithm is used to perform frequent itemset mining on the association rule dataset. The minimum support is set to 5% and the minimum confidence is set to 80%, and the core feature combinations that appear simultaneously in multiple silkworm germplasm resource individuals are mined. S333: Extract cross-dimensional association rules from the frequent itemset as cross-dimensional association features. The form of the association rules is: core features of dimension A → core features of dimension B. S334: Validate the extracted cross-dimensional association features and filter out association features with a lift greater than 1 by calculating the lift of the association rules.

7. The method for multi-dimensional integration and intelligent management of silkworm germplasm resources according to claim 4, characterized in that, The specific process of step S34 is as follows: S341: Summarize the single-dimensional core features and cross-dimensional related features of each dimension to form a preliminary version of the core feature set; S342: Establish a mapping relationship between the core feature set and the complete data archive, and label each core feature with a unique traceability identifier, original data indicator and contribution rate; S343: Output the final set of core features of silkworm germplasm resources. This set includes single-dimensional core features and cross-dimensional related features, and maintains a one-to-one mapping relationship with the complete data archive.

8. The method for multi-dimensional integration and intelligent management of silkworm germplasm resources according to claim 7, characterized in that, The specific process of step S4 is as follows: S41: Accurate classification of silkworm germplasm resources based on improved K-means clustering algorithm: S42: The overall architecture for building a dynamic database of silkworm germplasm resources, including a basic archive module: used to store complete data archives output by S3, with each record using a unique traceability identifier as the primary key; Core Feature Module: Stores the core feature set, with each record bound to a unique traceability identifier; Classification Result Module: Stores classification labels and classification criteria, with each record bound to a unique traceability identifier; Traceability Module: Stores traceability information throughout the entire process; and establishes a unified database management engine. S43: Develop data update interfaces and dynamically expand the database.

9. The method for multi-dimensional integration and intelligent management of silkworm germplasm resources according to claim 1, characterized in that, The specific process of step S41 is as follows: S411: Data Preprocessing and Algorithm Optimization Parameter Setting: Organize the core feature set into a clustering feature matrix; introduce prior rules and optimize the algorithm; S412: Germplasm resource classification is implemented based on an improved K-means clustering algorithm; S413: Assign a unique category code and category name to each cluster category to form a standardized classification label.

10. A method for multi-dimensional integration and intelligent management of silkworm germplasm resources according to claim 1, characterized in that, The multidimensional management of step S5 includes: Resource Query: Quickly retrieve complete information on corresponding silkworm germplasm resources by using unique traceability identifiers, classification tags, or core feature keywords; Dynamic monitoring: Real-time collection of new and existing data changes in the database, dynamic updates to classification results, and automatic triggering of re-clustering and classification when the core feature set changes significantly; Precise matching: Based on user needs, extract the characteristics of the needs and perform similarity calculation with the core feature set in the dynamic database, and output the optimal matching list of silkworm germplasm resources and adaptation analysis report; The intelligent management platform generates operation logs to record the data entry, query, update, and matching processes, enabling full-process traceability.

Images