A data management system and method for electronic engineering projects based on big data

Through the multi-source data adaptive consistency integration algorithm and multi-dimensional hybrid feature decomposition and reconstruction algorithm, the problems of multi-source data consistency and feature extraction in electronic engineering projects are solved, and efficient and accurate data integration and analysis are achieved.

CN118760885BActive Publication Date: 2024-11-19YANTAI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411237171.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-05
Publication Date
2024-11-19
Estimated Expiration
2044-09-05

AI Technical Summary

Technical Problem

The prior art is difficult to ensure the consistency, completeness and accuracy of multi-source data in electronic engineering projects, and it is difficult to accurately extract key features during large-scale data processing, resulting in inefficient data analysis and insufficient robustness.

Method used

Adoptional consistency integration algorithm of multi-source data and multi-dimensional hybrid feature decomposition and reconstruction algorithm are used to generate eigenvalue decomposition and multi-dimensional hybrid feature decomposition and reconstruction algorithm. Intelligently processed data ensures the consistency of data and the accuracy of feature extraction.

Benefits of technology

It improves the quality and efficiency of data integration, ensures the accuracy and robustness of data analysis, can accurately capture key features in complex data, and improves the accuracy and efficiency of data processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118760885B_ABST
    Figure CN118760885B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of data processing technology, and in particular to an electronic engineering project data management system and method based on big data. The content includes: acquiring data from data sources of different electronic engineering projects to obtain original multi-source data; integrating and processing the original multi-source data through a multi-source data adaptive consistency integration algorithm to obtain integrated multi-source data; based on the integrated multi-source data, intelligent processing is performed through a multi-dimensional hybrid feature decomposition and reconstruction algorithm, and the intelligently processed data is analyzed and mined to obtain analysis results; the intelligently processed data and analysis results are stored, and automated data management is performed. The problem that the consistency, integrity and accuracy of multi-source electronic engineering project data cannot be guaranteed in electronic engineering project data management, and the problem that key features cannot be accurately extracted to maintain data analysis accuracy and robustness in large-scale data processing is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an electronic engineering project data management system and method based on big data. Background Art

[0002] As the scale and complexity of electronic engineering projects continue to expand, the data sources involved in electronic engineering projects are becoming more diverse, including design tools (such as CAD software), sensors, monitoring equipment, project document libraries, etc. The data formats generated by these data sources vary, covering structured data (such as relational databases), semi-structured data (such as XML, JSON) and unstructured data (such as text files, images, etc.), which brings huge challenges to data integration and management. Traditional data management systems often find it difficult to ensure the consistency, integrity and accuracy of data when processing multi-source and multi-format data, resulting in a complex and inefficient data integration process. In addition, as the project progresses, the amount of project data grows rapidly. How to efficiently extract key features and maintain the accuracy and robustness of data analysis has become an urgent problem to be solved.

[0003] Existing intelligent data analysis methods often rely on a single data processing algorithm when processing large-scale electronic engineering project data, and are difficult to adapt to diverse data features and complex project requirements. Although these methods can provide certain analytical capabilities in pattern recognition and predictive model generation, they are prone to overfitting or failure to accurately capture deep patterns in project data when faced with high-dimensional and multi-scale features of project data. In addition, existing machine learning algorithms often face problems with computing efficiency and resource consumption when processing project data with high real-time requirements, making it difficult to meet the needs of rapid decision-making.

[0004] In summary, in electronic engineering project data management, there are problems such as the inability to ensure the consistency, completeness and accuracy of multi-source electronic engineering project data, and the inability to accurately extract key features to maintain data analysis accuracy and robustness during large-scale data processing. Summary of the invention

[0005] The present invention provides an electronic engineering project data management system and method based on big data to solve the problems that the consistency, integrity and accuracy of multi-source electronic engineering project data cannot be guaranteed in electronic engineering project data management, and key features cannot be accurately extracted to maintain data analysis accuracy and robustness during large-scale data processing.

[0006] The present invention provides an electronic engineering project data management system and method based on big data, which specifically includes the following technical solutions:

[0007] A data management method for electronic engineering projects based on big data comprises the following steps:

[0008] S1. Obtain data from data sources of different electronic engineering projects to obtain original multi-source data; integrate the original multi-source data through a multi-source data adaptive consistency integration algorithm to obtain integrated multi-source data;

[0009] S2. Based on the integrated multi-source data, intelligent processing is performed through a multi-dimensional hybrid feature decomposition and reconstruction algorithm to obtain intelligently processed data;

[0010] S3. Analyze and mine the intelligently processed data to obtain analysis results; store the intelligently processed data and the analysis results to obtain stored data, and perform automated data management on the stored data.

[0011] Preferably, the S1 specifically includes:

[0012] The multi-source data adaptive consistency integration algorithm preprocesses the original multi-source data to obtain preprocessed multi-source data; based on the preprocessed multi-source data, consistency check, nonlinear data conversion and mapping and adaptive integration fusion are performed to obtain integrated multi-source data.

[0013] Preferably, the S1 specifically includes:

[0014] In the process of consistency check of preprocessed multi-source data, a consistency measurement function is introduced to evaluate the similarity between the same data features from different data sources and obtain the consistency measurement value; the specific formula of the consistency measurement function is as follows:

[0015] ;

[0016] in, is the consistency measurement function; is a measure of consistency; After preprocessing and A collection of data sources; It is Data source The data value after preprocessing; It is Data source The value of the data after preprocessing; is a tuning parameter used to control the weight of the Euclidean distance term in the consistency measure; is a tuning parameter used to control the influence of the nonlinear term in the exponential function; is a tuning parameter used to control the sensitivity of the logistic function term to data differences; It is and The standard deviation of the data source; is the total number of data in the data source collection; Used to avoid the denominator being 0;

[0017] Based on the consistency metric, the fusion weight of the data source is obtained.

[0018] Preferably, the S1 specifically includes:

[0019] A measurement threshold is set, and nonlinear data conversion and mapping is performed on the preprocessed multi-source data whose consistency measurement value is higher than the measurement threshold to obtain the data value after nonlinear mapping.

[0020] Preferably, the S1 specifically includes:

[0021] Based on the data values ​​after nonlinear mapping and the fusion weights of the data sources, adaptive integrated fusion processing is performed to obtain integrated multi-source data.

[0022] Preferably, the S2 specifically includes:

[0023] In the process of implementing the multi-dimensional mixed eigendecomposition and reconstruction algorithm, the integrated multi-source data is centralized to obtain centralized data; the covariance matrix of the centralized data is calculated, and the eigenvalue decomposition is performed based on the covariance matrix of the centralized data to obtain the eigenvalues ​​and corresponding eigenvectors.

[0024] Preferably, the S2 specifically includes:

[0025] Based on the combination and transformation of the corresponding eigenvectors, a mixed eigenvector is generated. The specific formula is as follows:

[0026] ;

[0027] in, is the first of the mixed features finally generated feature vectors; is the first value obtained by performing eigenvalue decomposition on the covariance matrix of the centralized data. The eigenvector Quantity; is the first value obtained by performing eigenvalue decomposition on the covariance matrix of the centralized data. The eigenvector Quantity; It is an element-wise operation; is the first value obtained by performing eigenvalue decomposition on the covariance matrix of the centralized data. eigenvalues; Used to avoid the occurrence of 0 in logarithmic operations; It is the first The mean of the dimension feature; It is the first The standard deviation of the dimension feature; It is the characteristic dimension of the centralized data after eigenvalue decomposition.

[0028] Preferably, the S2 specifically includes:

[0029] Based on the mixed feature vector, weighted and nonlinear transformation is performed to obtain the reconstructed feature vector; based on the reconstructed feature vector, multi-scale transformation is performed to obtain the multi-scale transformed feature vector, and feature optimization and extraction are performed on the multi-scale transformed feature vector to obtain intelligently processed data.

[0030] An electronic engineering project data management system based on big data includes the following parts:

[0031] Multi-source data acquisition module, multi-source data integration module, intelligent data processing module, intelligent data analysis module, distributed storage module, and automated data management module;

[0032] The multi-source data acquisition module acquires data from data sources of different electronic engineering projects to obtain original multi-source data; and sends the original multi-source data to the multi-source data integration module;

[0033] The multi-source data integration module integrates the original multi-source data through a multi-source data adaptive consistency integration algorithm to obtain integrated multi-source data; and sends the integrated multi-source data to the intelligent data processing module;

[0034] The intelligent data processing module performs intelligent processing on the integrated multi-source data through a multi-dimensional hybrid feature decomposition and reconstruction algorithm to obtain intelligently processed data; the intelligently processed data is sent to the intelligent data analysis module and the distributed storage module;

[0035] The intelligent data analysis module performs pattern recognition on the intelligently processed data to obtain pattern recognition results; generates a prediction model based on the pattern recognition results to obtain prediction results; obtains analysis results based on the pattern recognition results and the prediction results; and sends the analysis results to the distributed storage module;

[0036] The distributed storage module stores the intelligently processed data and analysis results to obtain stored data; and sends the stored data to the automated data management module;

[0037] The automated data management module performs automated data management on the storage data in the distributed storage module.

[0038] The beneficial effects of the technical solution of the present invention are:

[0039] 1. By adopting a multi-source data adaptive consistency integration algorithm, it is possible to obtain project data in different formats from multiple data sources, and ensure the integrity, consistency and accuracy of project data through preprocessing, consistency checking, nonlinear data conversion and mapping, adaptive integration fusion processing and other steps. At the same time, the consistency measurement function is introduced, and nonlinear data conversion and mapping are performed, which can effectively handle the differences between multi-source data, so that project data from different sources maintain high consistency during the integration process, thereby greatly improving the quality and efficiency of data integration.

[0040] 2. Through the multi-dimensional mixed feature decomposition and reconstruction algorithm, the integrated multi-source data can be deeply processed intelligently. The integrated multi-source data is centrally processed to eliminate the mean difference between dimensions, and then the covariance matrix of the centralized data is decomposed by eigenvalues ​​to extract the main features of the project data, and the corresponding eigenvectors in the results of eigenvalue decomposition are combined and transformed at multiple levels to generate mixed feature vectors. Subsequently, the mixed feature vectors are reconstructed and multi-scale transformed, and the eigenvectors after multi-scale transformation are weighted and combined to obtain the intelligently processed data, so that the intelligently processed data has a high degree of representativeness and generalization ability at different scales. This process ensures that key features can be accurately captured when processing complex data, and improves the accuracy and robustness of data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a structural diagram of an electronic engineering project data management system based on big data according to the present invention;

[0042] Figure 2 This is a flow chart of an electronic engineering project data management method based on big data described in the present invention. DETAILED DESCRIPTION

[0043] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0045] The following is a detailed description of a specific solution of an electronic engineering project data management system and method based on big data provided by the present invention in conjunction with the accompanying drawings.

[0046] See attached Figure 1 , which shows a structure diagram of an electronic engineering project data management system based on big data provided by an embodiment of the present invention, the system includes the following parts:

[0047] Multi-source data acquisition module, multi-source data integration module, intelligent data processing module, intelligent data analysis module, distributed storage module, and automated data management module;

[0048] The multi-source data acquisition module acquires data from data sources of different electronic engineering projects to obtain original multi-source data; and sends the original multi-source data to the multi-source data integration module;

[0049] The original multi-source data includes design tool data, sensor data, monitoring equipment data, project document data, etc., and the data formats include structured data (such as relational database), semi-structured data (such as XML, JSON), and unstructured data (such as text files, images, etc.);

[0050] The multi-source data integration module integrates the original multi-source data through a multi-source data adaptive consistency integration algorithm to obtain integrated multi-source data; and sends the integrated multi-source data to the intelligent data processing module;

[0051] The intelligent data processing module performs intelligent processing on the integrated multi-source data through a multi-dimensional hybrid feature decomposition and reconstruction algorithm to obtain intelligently processed data; the intelligently processed data is sent to the intelligent data analysis module and the distributed storage module; the intelligent data analysis module performs pattern recognition on the intelligently processed data to obtain pattern recognition results; based on the pattern recognition results, a prediction model is generated to obtain prediction results; based on the pattern recognition results and the prediction results, an analysis result is obtained, i.e., a report that can be used for project decision-making; the analysis result is sent to the distributed storage module;

[0052] The distributed storage module stores the intelligently processed data and analysis results to obtain stored data for subsequent use such as query operations; and sends the stored data to the automated data management module;

[0053] The automated data management module automatically manages the stored data in the distributed storage module according to predefined rules such as data generation time, access frequency, and data importance.

[0054] See attached Figure 2 , which shows a flow chart of an electronic engineering project data management method based on big data provided by an embodiment of the present invention, the method comprising the following steps:

[0055] S1. Acquire data from data sources of different electronic engineering projects to obtain original multi-source data; integrate the original multi-source data through a multi-source data adaptive consistency integration algorithm to obtain integrated multi-source data;

[0056] Select the data source of the electronic engineering project according to the expert experience method, the data source includes design tools (such as CAD software), sensors, monitoring equipment, project document library, etc.; obtain data from various data sources to obtain original multi-source data, the original multi-source data format includes structured data (such as relational database), semi-structured data (such as XML, JSON), and unstructured data (such as text files, images, etc.);

[0057] The original multi-source data is integrated and processed using a multi-source data adaptive consistency integration algorithm; the multi-source data adaptive consistency integration algorithm integrates data from different data sources to ensure the consistency, integrity and accuracy of project data. Data integration is achieved by preprocessing, consistency checking, nonlinear data conversion and mapping, and adaptive integration fusion processing of the original multi-source data. The specific implementation process is as follows:

[0058] First, for the original multi-source data in different formats (structured, semi-structured, unstructured), different preprocessing methods and format conversion strategies are used to uniformly convert the original multi-source data into a processable standardized format to obtain the preprocessed multi-source data;

[0059] Among them, standardization processing is used for preprocessing of structured data; nested mapping functions are used for preprocessing of semi-structured data; natural language processing and graph structure data processing methods are used for preprocessing of unstructured data; specifically, standardization processing, nested mapping functions, natural language processing and graph structure data processing methods are selected according to the specific scenario and will not be elaborated here.

[0060] Secondly, the consistency check of the preprocessed multi-source data is carried out, and a consistency measurement function is introduced to evaluate the similarity between the same data features from different data sources, so as to determine the weight of the data source. The consistency measurement value obtained based on the consistency measurement function is used in the subsequent adaptive integration and fusion processing process to ensure that data sources with higher consistency are given priority during data integration. The consistency measurement function is as follows:

[0061]

[0062] in, It is a consistency metric used to measure the consistency of different data sources; After preprocessing and A collection of data sources; It is Data source The data value after preprocessing; It is Data source The value of the data after preprocessing; It is a tuning parameter used to control the weight of the Euclidean distance term in the consistency measure and determines the sensitivity of the consistency measure to the differences between different data sources; It is a tuning parameter used to control the influence of the nonlinear term in the exponential function and determine the response strength of the nonlinear term to data differences in the consistency measure; It is a tuning parameter used to control the sensitivity of the logistic function term to data differences, especially the impact when dealing with small differences; It is and The standard deviation of the data source; is the total number of data in the data source collection; is a small constant to avoid the denominator being zero.

[0063] Furthermore, the measurement threshold is set according to the expert experience method, and nonlinear data conversion and mapping are performed on the preprocessed multi-source data whose consistency measurement value is higher than the measurement threshold to obtain the nonlinear mapped data to improve the consistency between the data. The specific formula is as follows:

[0064]

[0065] in, It is Data source The data value after nonlinear mapping; It is Data source The data value after preprocessing; It is Data source The data value after preprocessing; It is the adjustment parameter in nonlinear mapping, controlling the amplitude of data conversion; It is the power of the exponential term, which controls the sensitivity of the absolute value of the data during the data conversion process; is the angle adjustment parameter, which controls the frequency and phase of the data in the sine function; It is the adjustment parameter of the cosine function in nonlinear mapping, which controls the amplitude change in data mapping;

[0066] Finally, after completing the nonlinear data transformation and mapping, weighted fusion is performed. Define the fusion weight to achieve adaptive integrated fusion processing. The specific formula is as follows:

[0067]

[0068] in, It is the final fused data value, which represents the integrated multi-source data obtained by weighted fusion and nonlinear integration of the same category data from multiple data sources; Is the data source after nonlinear mapping The fusion weight reflects the influence of the data source on the overall data integration and fusion during the integration and fusion process; is the number of data sources, indicating the total number of all different data sources involved in the fusion calculation; the calculation formula is as follows:

[0069]

[0070] in, It is a tuning parameter used to control the influence of the consistency measure on the fusion weight.

[0071] S2. Based on the integrated multi-source data, intelligent processing is performed through a multi-dimensional hybrid feature decomposition and reconstruction algorithm to obtain intelligently processed data;

[0072] The integrated multi-source data is intelligently processed using a multi-dimensional hybrid feature decomposition and reconstruction algorithm to obtain intelligently processed data; the specific implementation process is as follows:

[0073] First, the integrated multi-source data is centralized to eliminate the mean differences between different dimensions; the integrated multi-source data is more suitable for subsequent feature extraction and analysis through centralization. The specific process of the centralization is to subtract the mean of each dimension from the data of each dimension of the integrated multi-source data to obtain the centralized data;

[0074] Furthermore, in order to capture the important features in the centralized data, the covariance matrix of the centralized data is calculated to understand the linear relationship between the different feature dimensions of the centralized data. Each element of the covariance matrix represents the correlation between two features of the centralized data. In order to further analyze the main directions and change patterns in the centralized data, the covariance matrix of the centralized data is subjected to eigenvalue decomposition. The technical purpose of eigenvalue decomposition is to project the centralized data into the feature space to obtain the projection weights and importance of the centralized data in different feature dimensions. The result of the eigenvalue decomposition gives a set of eigenvalues ​​and corresponding eigenvectors; the eigenvalues ​​reflect the variance of the centralized data in the direction of the corresponding eigenvectors; the corresponding eigenvectors represent the distribution pattern of the centralized data in the direction of the corresponding eigenvectors;

[0075] Furthermore, in order to combine the above eigenvalue decomposition results into useful eigenvectors, a mixed eigenvector is generated by performing multi-level combination and transformation on the corresponding eigenvectors after eigenvalue decomposition; the specific formula is as follows:

[0076]

[0077] in, is the first of the mixed features finally generated feature vectors; is the first value obtained by performing eigenvalue decomposition on the covariance matrix of the centralized data. The eigenvector Quantity; is the first value obtained by performing eigenvalue decomposition on the covariance matrix of the centralized data. The eigenvector Quantity; It is an element-wise operation (Hadamard product), that is, each element of the eigenvector is squared; is the first value obtained by performing eigenvalue decomposition on the covariance matrix of the centralized data. eigenvalues; It is the first The mean of the dimension feature; It is the first The standard deviation of the dimension feature; It is the dimension of the data after centralization, that is, the characteristic dimension of the data after eigenvalue decomposition; It is a small constant used to avoid the occurrence of 0 in logarithmic operations; It is used to adjust the amplitude of the eigenvector. By combining the square sum with the logarithmic function, the amplitude of the eigenvalue is stabilized and the extreme change of the eigenvalue is avoided. The periodic changes of the eigenvector are captured by the sine function; Increased sensitivity to the asymmetry and skewness of the eigenvalue distribution, and enhanced the capture of skewness information through cubic terms and exponential functions; The inverse tangent function is introduced to compress the extreme values ​​in the feature space;

[0078] Furthermore, feature reconstruction is performed based on the mixed feature vector. The purpose of reconstruction is to recombine the mixed feature vector by introducing weights to obtain a reconstructed feature vector, thereby increasing the richness and expression ability of the reconstructed features. The specific formula is as follows:

[0079]

[0080] in, is the reconstructed feature A vector, which represents the feature vector obtained after the mixed feature vector is weighted and nonlinearly transformed; is the first in the mixed feature The weight of the feature vector;

[0081] Furthermore, the reconstructed feature vector is subjected to multi-scale transformation to obtain the feature vector after multi-scale transformation, so as to capture the changing characteristics of the data after centralized processing at different scales. Multi-scale transformation re-encodes and integrates the features at different time points and frequencies through wavelet transformation to ensure that the obtained feature vector after multi-scale transformation has high representativeness and generalization ability. The specific formula is as follows:

[0082]

[0083] in, is the first feature after multi-scale transformation feature vectors; It is the coefficient in wavelet transform, which is used to adjust the reconstructed features at different scales. The weight on is the reconstructed feature Vector In the Feature vectors at scales; is the feature vector used for historical reconstruction The weight coefficient of The influence of the reconstructed features at each time point; is the reconstructed eigenvector in the past The component at a time point, namely the historical reconstruction feature vector, reflects the contribution of the historical reconstruction feature vector to the current feature vector; Indicates the total number of scales; Indicates the total number of time points;

[0084] Finally, in order to extract features with practical application value, the feature vectors after multi-scale transformation are optimized and extracted. The optimization objective function is defined by expert experience method, and the Lagrange multiplier method and gradient descent method are introduced to optimize the optimization objective function and find the optimal feature weights. The optimal feature weights are used to perform weighted combination of the feature vectors after multi-scale transformation to obtain intelligently processed data.

[0085] S3. Analyze and mine the intelligently processed data to obtain analysis results; store the intelligently processed data and the analysis results to obtain stored data, and perform automated data management on the stored data.

[0086] Intelligent analysis and mining are performed on the data after intelligent processing, specifically including using existing machine learning algorithms (such as K-Means clustering, support vector machine (SVM)), etc., selected based on the usage scenario according to expert experience, to perform pattern recognition on the data after intelligent processing, identify specific patterns or trends in the data after intelligent processing, and obtain pattern recognition results; based on the pattern recognition results, use existing algorithms such as regression analysis and time series analysis to generate prediction models to obtain prediction results, such as project progress, cost overrun risks, etc.; based on the pattern recognition results and prediction results, generate analysis results, that is, reports that can be used for project decision-making, and the report content includes data patterns, trend analysis, and risk prediction and early warning.

[0087] The analysis results and the data after intelligent processing are stored to obtain storage data.

[0088] First, the analysis results and the data after intelligent processing need to be classified and layered. According to the type, access frequency and importance of the analysis results and the data after intelligent processing, the analysis results and the data after intelligent processing are divided into hot data, cold data and archived data; hot data is data with high access frequency and high importance, which is stored on high-performance storage devices, such as SSD, to ensure fast access; cold data is accessed less frequently and stored on low-cost storage media, such as HDD or tape; archived data is data that has been analyzed and is no longer frequently used, and is stored in long-term storage media to reduce the burden on the electronic engineering project data management system;

[0089] After the classification is completed, the existing distributed storage technology is used for storage. Distributed file systems (such as HDFS) use data sharding and replication technology to ensure high availability and reliability of data; data sharding is to divide data into multiple small blocks and store them on different nodes to improve access speed and fault tolerance; data replication creates multiple copies of each data block and distributes them on different storage nodes to prevent single point failures. In addition, in order to improve the utilization of storage space and transmission efficiency, the stored data is compressed, and indexes are established based on common query conditions (such as time, project ID, data type), and query optimization technology (such as partition table, parallel query) is used to improve the efficiency of data query.

[0090] Furthermore, automated data management is performed on the stored data.

[0091] First, configure the data management strategy through predefined rules. Based on factors such as the generation time, access frequency and importance of the stored data, define management strategies such as backup frequency, archiving strategy and deletion rules. The frequency of data backup is determined by the importance and real-time requirements of the stored data. Common data backup frequencies include daily backup or weekly backup. The data archiving strategy is to move the stored data that is no longer frequently used to the archiving storage system according to the life cycle of the stored data. The data deletion rule automatically deletes expired data according to project requirements or regulations. Then, the electronic engineering project data management system automatically manages the life cycle of data according to predefined rules. Through data migration, hot data that is no longer frequently accessed is migrated from high-performance storage devices to cold data storage media; for data that needs to be stored for a long time but is not frequently accessed, perform data archiving operations and store it in low-cost long-term storage media; regularly scan the storage and automatically clean up expired or useless data according to the deletion rules to free up storage space. To ensure the flexibility and adaptability of the electronic engineering project data management system, real-time monitoring of data usage is carried out. By monitoring the access frequency of data, the electronic engineering project data management system can dynamically adjust the storage level of data; according to the progress of the project, the importance of data is regularly evaluated and the management strategy is adjusted. In addition, existing machine learning or data analysis technologies are used to optimize management rules, and through continuous learning and adjustment, the efficiency and accuracy of storage data management are improved, thereby ensuring the efficient operation of the entire electronic engineering project data management system and the optimal use of storage resources.

[0092] In summary, an electronic engineering project data management system and method based on big data has been completed.

[0093] The order of the embodiments of the invention is for description only and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0094] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0095] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A data management method for electronic engineering projects based on big data, characterized in that: The following steps are involved: S1. Acquire data from data sources of different electronic engineering projects to obtain original multi-source data; pre-process the original multi-source data through a multi-source data adaptive consistency integration algorithm to obtain pre-processed multi-source data; The consistency measurement function is introduced to perform consistency check on the preprocessed multi-source data, evaluate the similarity between the same data features from different data sources, and obtain the consistency measurement value; the specific formula of the consistency measurement function is as follows: ; in, is the consistency measurement function; is a measure of consistency; After preprocessing and A collection of data sources; It is Data source The data value after preprocessing; It is Data source The value of the data after preprocessing; is a tuning parameter used to control the weight of the Euclidean distance term in the consistency measure; is a tuning parameter used to control the influence of the nonlinear term in the exponential function; is a tuning parameter used to control the sensitivity of the logistic function term to data differences; It is and The standard deviation of the data source; is the total number of data in the data source collection; Used to avoid the denominator being 0; The measurement threshold is set, and nonlinear data conversion and mapping is performed on the preprocessed multi-source data whose consistency measurement value is higher than the measurement threshold to obtain the data value after nonlinear mapping; the specific formula is as follows: ; in, It is Data source The data value after nonlinear mapping; It is Data source The data value after preprocessing; It is Data source The data value after preprocessing; is the adjustment parameter in the nonlinear mapping; is the power of the exponential term; is the angle adjustment parameter; is the adjustment parameter of the cosine function in the nonlinear mapping; is the total number of data in the data source collection; Used to avoid the denominator being 0; Based on the data values ​​after nonlinear mapping, adaptive integration and fusion processing is performed to obtain integrated multi-source data. The specific formula is as follows: ; in, is the final fused data value, representing the integrated multi-source data; Is the data source after nonlinear mapping The fusion weight of is the number of data sources; Used to avoid the denominator being 0; the fusion weight of the data source after nonlinear mapping The calculation formula is as follows: ; in, is the adjustment parameter; S2. Centralize the integrated multi-source data through a multi-dimensional hybrid eigendecomposition and reconstruction algorithm to obtain centralized data; calculate the covariance matrix of the centralized data, perform eigenvalue decomposition based on the covariance matrix of the centralized data, and obtain eigenvalues ​​and corresponding eigenvectors; Based on the combination and transformation of the corresponding eigenvectors, a mixed eigenvector is generated. The specific formula is as follows: ; in, is the first of the mixed features finally generated feature vectors; is the first value obtained by performing eigenvalue decomposition on the covariance matrix of the centralized data. The eigenvector Quantity; is the eigenvector of the eigenvalue decomposition of the covariance matrix of the centered data. Quantity; It is an element-wise operation; is the first value obtained by performing eigenvalue decomposition on the covariance matrix of the centralized data. eigenvalues; Used to avoid the occurrence of 0 in logarithmic operations; It is the first The mean of the dimension feature; It is the first The standard deviation of the dimension feature; It is the characteristic dimension of the centralized data after eigenvalue decomposition; Based on the mixed feature vector, weighted and nonlinear transformation is performed to obtain the reconstructed feature vector; based on the reconstructed feature vector, multi-scale transformation is performed to obtain the feature vector after multi-scale transformation, and feature optimization and extraction are performed on the feature vector after multi-scale transformation to obtain the data after intelligent processing; S3. Analyze and mine the intelligently processed data to obtain analysis results; store the intelligently processed data and the analysis results to obtain stored data, and perform automated data management on the stored data.

2. An electronic engineering project data management system based on big data, applied to an electronic engineering project data management method based on big data as claimed in claim 1, characterized in that: Includes the following sections: Multi-source data acquisition module, multi-source data integration module, intelligent data processing module, intelligent data analysis module, distributed storage module, and automated data management module; The multi-source data acquisition module acquires data from data sources of different electronic engineering projects to obtain original multi-source data; and sends the original multi-source data to the multi-source data integration module; The multi-source data integration module integrates the original multi-source data through a multi-source data adaptive consistency integration algorithm to obtain integrated multi-source data; and sends the integrated multi-source data to the intelligent data processing module; The intelligent data processing module performs intelligent processing on the integrated multi-source data through a multi-dimensional hybrid feature decomposition and reconstruction algorithm to obtain intelligently processed data; the intelligently processed data is sent to the intelligent data analysis module and the distributed storage module; The intelligent data analysis module performs pattern recognition on the intelligently processed data to obtain pattern recognition results; based on the pattern recognition results, a prediction model is generated to obtain prediction results; Based on the pattern recognition results and prediction results, the analysis results are obtained; Send the analysis results to the distributed storage module; The distributed storage module stores the intelligently processed data and analysis results to obtain stored data; Sending stored data to an automated data management module; The automated data management module performs automated data management on the storage data in the distributed storage module.