Engineering data pattern processing method and system based on machine learning

By performing feature extraction and vectorization of engineering data, it is divided into the first vector and the second vector, and using supervised learning model training, the problem of redundant data and correlation in the existing technology is solved, and the problem of redundant data and correlation is improved Analytical accuracy and effect are obtained.

CN118568505BActive Publication Date: 2024-11-01SHENZHEN HOPESEN TECH CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411037353.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-01
Estimated Expiration
2044-07-31

AI Technical Summary

Technical Problem

When processing engineering data, existing machine learning models are prone to problems such as redundant data and correlation not being considered, resulting in poor analysis results.

Method used

Through feature extraction, preprocessing, serialization and vectorization operations, the engineering data set is divided into the first vector and the second vector, and a supervised learning neural network model is used for training to ensure the uncorrelation and correlation between vectors. to improve analysis accuracy.

Benefits of technology

The analysis accuracy and effectiveness of the engineering data analysis model are improved, ensuring the accuracy of sample data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118568505B_ABST
    Figure CN118568505B_ABST
Patent Text Reader

Abstract

The present invention provides a method and system for processing engineering data patterns based on machine learning, wherein the method includes: extracting features from an engineering data set, performing preprocessing, serialization, separation and other operations to divide the data set into a first vector and a second vector, training a preset neural network model using a supervised learning method based on each first training data and second training data to obtain an engineering data analysis model, and analyzing the engineering data set to be tested through the engineering data analysis model. The beneficial effects of the present invention are: the engineering data analysis model has accurate sample data for training, thereby improving the analysis accuracy of the engineering data analysis model and improving the analysis effect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method and system for processing engineering data patterns based on machine learning. Background Art

[0002] Data processing in the engineering field has always been an important project, especially with the development of big data and Internet of Things technology, engineering data has become more complex and huge. Traditional methods are often difficult to effectively process this data.

[0003] Machine learning is an application branch of artificial intelligence. It is dedicated to developing technologies and algorithms that enable computers to learn from data and make predictions or take actions. However, the existing machine learning method mainly inputs engineering data directly into the machine learning model. Since engineering data is relatively complex, in the existing technology, engineering data is either directly input into the machine learning model for analysis, or engineering data is classified and then input into the machine learning model for analysis. The former adds redundant data during the analysis process, and the latter does not consider the correlation between engineering data, resulting in poor analysis results after the machine learning model training is completed. Summary of the invention

[0004] The main purpose of the present invention is to provide an engineering data pattern processing method and system based on machine learning, aiming to solve the problem of poor analysis effect after the existing machine learning model training is completed.

[0005] The present invention provides an engineering data pattern processing method based on machine learning, comprising:

[0006] Collecting multiple sets of engineering data sets, and using a preset feature extraction model to extract feature information in each set of the engineering data sets to obtain an engineering information set; wherein the engineering data sets include sensor data, process parameters and / or equipment status;

[0007] Preprocessing each of the engineering information sets to obtain a standard engineering information set;

[0008] Serializing the characteristic information in the standard engineering information set according to a preset serialization operation to obtain text information in a text form corresponding to each characteristic information;

[0009] Arrange the text information in a preset order and separate them with delimiters to obtain an engineering text sequence, select a target delimiter from the engineering text sequence, record the subsequence of the engineering text sequence before the target delimiter as a first sequence, and record the subsequence of the engineering text sequence after the target delimiter as a second sequence;

[0010] Vectorize the first sequence and the second sequence respectively to obtain a first vector and a second vector, calculate the similarity between the first vector and the second vector, perform sigmoid nonlinear mapping on the obtained similarity value to the (0,1) interval, and calculate the linear distance from 1;

[0011] When the linear distance is greater than a set threshold, the first vector and the second vector are used as features of the corresponding engineering data set;

[0012] Obtain indicator data corresponding to each engineering data set, and annotate the indicator data with a first vector and a second vector of the corresponding engineering data set to obtain first training data and second training data;

[0013] According to each of the first training data and the second training data, a preset neural network model is trained in a supervised learning manner to obtain an engineering data analysis model;

[0014] Performing the same processing operation on the engineering data set to be tested as that on the engineering data set to obtain a corresponding first target vector and a second target vector;

[0015] The target first vector and the target second vector are input into the engineering data analysis model to obtain target indicator data of the engineering data set to be tested.

[0016] Furthermore, the step of calculating the similarity between the first vector and the second vector, performing sigmoid nonlinear mapping on the obtained similarity value to the (0, 1) interval, and calculating the linear distance from 1 includes:

[0017] By formula similarity a ⃗ , b ⃗ = ∑ i= 1 n a ⃗ [ i ]∗ b ⃗ [ i ] ∑ i= 1 n a ⃗ [ i ]∗ a ⃗ [ i ] ∑ i= 1 n b ⃗ [ i ]∗ b ⃗ [ i ] The similarity value is calculated, wherein, is the similarity value, represents the first vector, represents the second vector, a ⃗ [ i ] represents the i-th dimension of the first vector, b ⃗ [ i ] represents the i-th dimension of the second vector;

[0018] By formula Calculate the mapping value of the nonlinear mapping to the (0,1) interval; where, Represents a mapping value;

[0019] The linear distance from 1 is calculated according to the mapped value.

[0020] Furthermore, after the step of selecting the target delimiter from the engineering text sequence, the method further includes:

[0021] Obtaining a first sequence distance from each target delimiter to the beginning of the engineering text sequence, and a second sequence distance from each target delimiter to the end of the engineering text sequence;

[0022] According to the formula Calculate the distance score of each target separator, wherein K is the distance score, X is the first sequence distance, and Y is the second sequence distance;

[0023] Determining whether the distance score is less than a preset distance score;

[0024] If it is less than the preset distance score, it is determined that the selected target separation is reasonable.

[0025] Furthermore, after the step of arranging the text information in a preset order and separating them with separators to obtain the engineering text sequence, the method further includes:

[0026] Detecting the sequence length of the engineering text sequence;

[0027] When the sequence length is greater than a preset sequence length, multiple target separators are selected.

[0028] Furthermore, the step of training the preset neural network model in a supervised learning manner according to each of the first training data and the second training data to obtain the engineering data analysis model includes:

[0029] Dividing the plurality of the first training data and the plurality of the second training data into a training data set and a validation data set according to a preset ratio;

[0030] Inputting the training data in the training data set into a preset neural network model, and performing training processing in a supervised learning manner to obtain an intermediate data analysis model;

[0031] Using the verification data in the verification data set to perform verification processing on the intermediate data analysis model to obtain a verification result;

[0032] Determine whether the verification result is passed;

[0033] If the verification result passes, the intermediate data analysis model is recorded as the engineering data analysis model.

[0034] The present invention also provides an engineering data pattern processing system based on machine learning, comprising:

[0035] A collection module, used to collect multiple groups of engineering data sets, and use a preset feature extraction model to extract feature information in each group of the engineering data sets to obtain an engineering information set; wherein the engineering data sets include sensor data, process parameters and / or equipment status;

[0036] A preprocessing module, used for preprocessing each of the engineering information sets to obtain a standard engineering information set;

[0037] A serialization module, used to serialize the characteristic information in the standard engineering information set according to a preset serialization operation to obtain text information in a text form corresponding to each characteristic information;

[0038] An arrangement module is used to arrange each text information in a preset order and separate them with separators to obtain an engineering text sequence, and select a target separator from the engineering text sequence, record the subsequence of the engineering text sequence before the target separator as a first sequence, and record the subsequence of the engineering text sequence after the target separator as a second sequence;

[0039] A vectorization module, used for performing vectorization operations on the first sequence and the second sequence respectively to obtain a first vector and a second vector, performing similarity calculation on the first vector and the second vector, and performing sigmoid nonlinear mapping on the obtained similarity value to the (0,1) interval to obtain a linear distance from 1;

[0040] A setting module, configured to use the first vector and the second vector as features of a corresponding engineering data set when the linear distance is greater than a set threshold;

[0041] An acquisition module is used to acquire the indicator data corresponding to each engineering data set, and annotate the indicator data with the first vector and the second vector of the corresponding engineering data set to obtain the first training data and the second training data;

[0042] A training module, used to train a preset neural network model in a supervised learning manner according to each of the first training data and the second training data to obtain an engineering data analysis model;

[0043] An execution module, used for performing the same processing operation on the engineering data set to be tested as that on the engineering data set, so as to obtain a corresponding target first vector and a target second vector;

[0044] An input module is used to input the target first vector and the target second vector into the engineering data analysis model to obtain target indicator data of the engineering data set to be tested.

[0045] Furthermore, the vectorization module includes:

[0046] The first similarity value calculation submodule is used to calculate the similarity value through the formula similarity a ⃗ , b ⃗ = ∑ i= 1 n a ⃗ [ i ]∗ b ⃗ [ i ] ∑ i= 1 n a ⃗ [ i ]∗ a ⃗ [ i ] ∑ i= 1 n b ⃗ [ i ]∗ b ⃗ [ i ] The similarity value is calculated, wherein, is the similarity value, represents the first vector, represents the second vector, a ⃗ [ i ] represents the i-th dimension of the first vector, b ⃗ [ i ] represents the i-th dimension of the second vector;

[0047] The second similarity value calculation submodule is used to calculate the similarity value through the formula Calculate the mapping value of the nonlinear mapping to the (0,1) interval; where, Represents a mapping value;

[0048] The similarity value third calculation submodule is used to calculate the linear distance from 1 according to the mapping value.

[0049] Furthermore, the engineering data pattern processing system based on machine learning further includes:

[0050] A distance acquisition module, used for acquiring a first sequence distance from each target delimiter to the beginning of the engineering text sequence, and a second sequence distance from each target delimiter to the end of the engineering text sequence;

[0051] Distance score calculation module, used to calculate the distance score according to the formula Calculate the distance score of each target separator, wherein K is the distance score, X is the first sequence distance, and Y is the second sequence distance;

[0052] A distance score judgment module, used to judge whether the distance score is less than a preset distance score;

[0053] The determination module is used to determine that the selected target separation is reasonable if the distance is less than a preset distance score.

[0054] Furthermore, the engineering data pattern processing system based on machine learning further includes:

[0055] A sequence length detection module, used to detect the sequence length of the engineering text sequence;

[0056] The target separator selection module is used to select multiple target separators when the sequence length is greater than a preset sequence length.

[0057] Furthermore, the training module includes:

[0058] A division submodule, used for dividing the plurality of the first training data and the plurality of the second training data into a training data set and a verification data set according to a preset ratio;

[0059] An input submodule, used to input the training data in the training data set into a preset neural network model, and perform training processing in a supervised learning manner to obtain an intermediate data analysis model;

[0060] An adopting submodule is used to use the verification data in the verification data set to perform verification processing on the intermediate data analysis model to obtain a verification result;

[0061] The judgment submodule is used to judge whether the verification result is passed;

[0062] The marking submodule is used to record the intermediate data analysis model as the engineering data analysis model if the verification result passes.

[0063] The beneficial effects of the present invention are as follows: by performing feature extraction, preprocessing, serialization, separation and other operations on the engineering data set, the engineering data set is divided into a first vector and a second vector, which not only ensures the independence of the first vector and the second vector, but also ensures the correlation of the engineering data contained in any one of the first vector and the second vector, so that the engineering data analysis model has accurate sample data for training, thereby improving the analysis accuracy of the engineering data analysis model and improving the analysis effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a flowchart of a method for processing engineering data patterns based on machine learning according to an embodiment of the present invention;

[0065] Figure 2 It is a schematic block diagram of the structure of an engineering data pattern processing system based on machine learning according to an embodiment of the present invention;

[0066] Figure 3 A schematic block diagram of the structure of a computer device according to an embodiment of the present application.

[0067] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly. The connection can be a direct connection or an indirect connection.

[0070] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone.

[0071] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0072] Reference Figure 1 The present invention proposes an engineering data pattern processing method based on machine learning, comprising:

[0073] S1: Collect multiple groups of engineering data sets, and use a preset feature extraction model to extract feature information in each group of the engineering data sets to obtain an engineering information set; wherein the engineering data sets include sensor data, process parameters and / or equipment status;

[0074] S2: preprocessing each of the engineering information sets to obtain a standard engineering information set;

[0075] S3: Serializing the characteristic information in the standard engineering information set according to a preset serialization operation to obtain text information in a text form corresponding to each characteristic information;

[0076] S4: Arrange the text information in a preset order and separate them with delimiters to obtain an engineering text sequence, select a target delimiter from the engineering text sequence, record the subsequence of the engineering text sequence before the target delimiter as a first sequence, and record the subsequence of the engineering text sequence after the target delimiter as a second sequence;

[0077] S5: performing vectorization operations on the first sequence and the second sequence respectively to obtain a first vector and a second vector, performing similarity calculation on the first vector and the second vector, and performing sigmoid nonlinear mapping on the obtained similarity value to the (0,1) interval to calculate the linear distance from 1;

[0078] S6: when the linear distance is greater than a set threshold, taking the first vector and the second vector as features of the corresponding engineering data set;

[0079] S7: Obtain indicator data corresponding to each engineering data set, and annotate the indicator data with a first vector and a second vector of the corresponding engineering data set to obtain first training data and second training data;

[0080] S8: training the preset neural network model in a supervised learning manner according to each of the first training data and the second training data to obtain an engineering data analysis model;

[0081] S9: performing the same processing operation on the engineering data set to be tested as that on the engineering data set to obtain a corresponding target first vector and a target second vector;

[0082] S10: Inputting the target first vector and the target second vector into the engineering data analysis model to obtain target indicator data of the engineering data set to be tested.

[0083] As described in the above step S1, multiple groups of engineering data sets are collected, and a preset feature extraction model is used to extract feature information from each group of the engineering data sets to obtain an engineering information set; wherein the engineering data set includes sensor data, process parameters and / or equipment status; wherein, multiple groups of engineering data sets are collected, including sensor data, process parameters and equipment status, etc. Since these data may come from different engineering equipment, sensor systems or monitoring systems, they can be obtained by manually summarizing and uploading, or from the storage address of the corresponding data, or on the platform, to establish a preset feature extraction model, which can define a set of appropriate feature extraction algorithms or methods according to the characteristics and needs of the engineering field, and is used to extract important feature information from engineering data. The preset feature model can be a feature extraction model based on deep learning, or a custom feature extraction model, which can be specifically identified and extracted according to the characteristics of the data, or keywords, etc.

[0084] As described in step S2 above, each of the engineering information sets is preprocessed to obtain a standard engineering information set. The preprocessing method specifically includes: Data cleaning: Check the data in the engineering information set, process missing values, outliers or erroneous data to ensure the integrity and accuracy of the data. Data normalization or standardization: Normalize or standardize the numerical data in the engineering information set so that the values ​​of different features are within a similar numerical range, which is helpful for model training and result comparison. Feature selection: For a large number of features, feature selection can be performed to eliminate features that have little contribution to model prediction and reduce redundant information in the data set.

[0085] As described in the above step S3, the feature information in the standard engineering information set is serialized according to the preset serialization operation to obtain text information in text form corresponding to each feature information. Specifically, the serialization operation can be to first determine the specific rules and formats for serializing the feature information into text information, including the order of features, separators, identifiers, etc., which can be planned according to the meaning and importance of the specific features. According to the serialization rules, each feature information is serialized and converted into information in text form. Numerical features can be directly converted into text form, or specific encoding, mapping and other operations can be performed to generate corresponding text information, and the various feature information text information obtained after serialization processing are assembled into a complete text form information set. It involves operations such as adding feature names and inserting separators between information to ensure that the generated text information meets the preset format requirements.

[0086] As described in the above-mentioned step S4, each text information is arranged in a pre-set order and separated by a separator to obtain an engineering text sequence, and a target separator is selected from the engineering text sequence, and the subsequence of the engineering text sequence before the target separator is recorded as the first sequence, and the subsequence of the engineering text sequence after the target separator is recorded as the second sequence. That is, based on the target separator, the engineering text sequence is divided into two subsequences, namely the first sequence and the second sequence. The first sequence contains all the text information before the target separator, and the second sequence contains all the text information after the target separator. In addition, the pre-set order is the order set by the relevant personnel according to the corresponding analysis results, or the data correlation.

[0087] As described in step S5 above, the first sequence and the second sequence are vectorized respectively to obtain the first vector and the second vector, and the similarity of the first vector and the second vector is calculated, and the obtained similarity value is sigmoid nonlinearly mapped to the interval (0,1), and the linear distance from 1 is calculated, wherein the formula for similarity calculation can be simhash algorithm, WMD algorithm (word mover's distance), calculation based on SVM (Support Vector Machine) vector model, algorithm based on cosine similarity, etc., which can calculate the similarity of the first vector and the second vector. In some embodiments, it is necessary to ensure that the dimensions of the first vector and the second vector are the same. If the dimensions are different, they can be subjected to dimensionality increase or dimensionality reduction calculations, etc., so that their dimensions are consistent, so as to facilitate the calculation of similarity. Then the calculated similarity value is mapped to the interval (0,1), and the similarity value can be reflected in the linear distance from 1, which is convenient for subsequent comparison with the set threshold.

[0088] As described in the above step S6, when the linear distance is greater than the set threshold, the first vector and the second vector are used as features of the corresponding engineering data set. That is, if the linear distance is greater than the set threshold, it means that the first vector and the second vector are not related, so the two can be trained separately, that is, it is not certain whether the result in the engineering data set corresponds to the first vector or the second vector, so they can be trained separately. When the linear distance is not greater than the set threshold, it means that the first vector and the second vector are related. At this time, the first vector and the second vector can have the same representation of the indicator data, but their data are similar, which will lead to repeated calculations. The data is not well separated. Therefore, it is necessary to re-determine the target separator until the calculated linear distance is greater than the set threshold. Specifically, since the target separator cannot be determined, all separators can be selected from the middle to both ends in sequence until the calculated linear distance corresponding to the separator is greater than the set threshold, and the separator is recorded as the target separator.

[0089] As described in step S7 above, the indicator data corresponding to each engineering data set is obtained, and the indicator data is labeled with the first vector and the second vector of the corresponding engineering data set to obtain the first training data and the second training data. Each engineering data set has corresponding indicator data, and the indicator data is the key data as the analysis result, so the obtained indicator data is labeled as the first vector and the second vector respectively to facilitate the subsequent model training.

[0090] As described in step S8, the preset neural network model is trained in a supervised learning manner according to each of the first training data and the second training data to obtain an engineering data analysis model. The neural network model can adopt any feasible model, such as a perceptron neural network model, a BP neural network model, a radial basis function neural network model, etc.

[0091] As described in step S9 above, the engineering data set to be tested is subjected to the same processing operation as that performed on the engineering data set to obtain the corresponding target first vector and target second vector. That is, the engineering data set to be tested adopts a preset feature extraction model to extract feature information in the engineering data set to be tested, and then the feature information is preprocessed to obtain a standard engineering information set, and the feature information in the standard engineering information set is serialized according to a preset serialization operation to obtain text information in text form corresponding to each feature information, and each text information is arranged in a preset order and separated by a separator to obtain an engineering text sequence, and a target separator is selected from the engineering text sequence, and the subsequence of the engineering text sequence before the target separator is recorded as the first sequence, and the subsequence of the engineering text sequence after the separator is recorded as the second sequence, and the first sequence and the second sequence are respectively vectorized to obtain the target first vector and the target second vector.

[0092] As described in step S10 above, the target first vector and the target second vector are input into the engineering data analysis model to obtain the target indicator data of the engineering data set to be tested. The target indicator data is the analysis result of the engineering data set to be tested, and relevant personnel can use it to evaluate the engineering data set to be tested. In this way, the engineering data is divided into the first vector and the second vector, and the independence between the first vector and the second vector is ensured, and the correlation of the engineering data contained in any one of the first vector and the second vector is ensured, so that the analysis accuracy of the machine learning model can be improved, thereby improving the analysis effect.

[0093] In one embodiment, the step S5 of calculating the similarity between the first vector and the second vector, performing sigmoid nonlinear mapping on the obtained similarity value to the interval (0, 1), and calculating the linear distance from 1 includes:

[0094] S501: By formula similarity a ⃗ , b ⃗ = ∑ i= 1 n a ⃗ [ i ]∗ b ⃗ [ i ] ∑ i= 1 n a ⃗ [ i ]∗ a ⃗ [ i ] ∑ i= 1 n b ⃗ [ i ]∗ b ⃗ [ i ] The similarity value is calculated, wherein, is the similarity value, represents the first vector, represents the second vector, a ⃗ [ i ] represents the i-th dimension of the first vector, b ⃗ [ i ] represents the i-th dimension of the second vector;

[0095] S502: By formula Calculate the mapping value of the nonlinear mapping to the (0,1) interval; where, Represents a mapping value;

[0096] S503: Calculate the linear distance from 1 according to the mapping value.

[0097] As described in the above steps S501-S503, the calculation of the correlation between the first vector and the second vector is realized. Since the first vector and the second vector can be processed by dimensionality increase or dimensionality reduction to ensure that their dimensions are the same, each dimension can be calculated separately and then summed to obtain a similarity value, so that the calculation of the similarity value uses as many original data values ​​as possible, reducing calculation losses and ensuring the calculation effect. Then, the mapping value of each similarity value in the interval (0,1) is calculated by the sigmoid function, and the linear distance from 1 is calculated based on the mapping value. The method of calculating is to subtract the mapping value from 1, wherein the smaller the linear distance is, the less correlated the first vector is with the second vector, and vice versa, the more correlated they are.

[0098] In one embodiment, after the step S4 of selecting a target delimiter from the engineering text sequence, the method further includes:

[0099] S511: Obtain a first sequence distance from each target delimiter to the beginning of the engineering text sequence, and a second sequence distance from each target delimiter to the end of the engineering text sequence;

[0100] S512: According to the formula Calculate the distance score of each target separator, wherein K is the distance score, X is the first sequence distance, and Y is the second sequence distance;

[0101] S513: Determine whether the distance score is less than a preset distance score;

[0102] S514: If the distance is less than the preset distance score, it is determined that the selected target separation is reasonable.

[0103] As described in the above steps S511-S514, in the actual calculation process, there may be multiple target delimiters whose calculated linear distances are greater than the set threshold. Therefore, the position of each target delimiter in the engineering text sequence can be calculated according to the formula The distance score of each target separator is calculated, and a final value is obtained by weighted summation of the distance score and the calculated similarity score. The target separator corresponding to the largest final value is selected as the target separator, thereby ensuring a balance between the data volume of the first vector and the second vector.

[0104] In one embodiment, after the step S4 of arranging the text information in a preset order and separating them with separators to obtain the engineering text sequence, the method further includes:

[0105] S521: Detecting the sequence length of the engineering text sequence;

[0106] S522: When the sequence length is greater than a preset sequence length, multiple target delimiters are selected.

[0107] As described in the above steps S521-S522, the sequence length of the engineering text sequence may be relatively long, that is, it contains more engineering data. At this time, multiple target delimiters can be set, and the ones between the target delimiters are used as additional vectors. For example, if two delimiters need to be set, the subsequence from the beginning of the engineering text sequence to the first delimiter is recorded as the first subsequence, the subsequence between the first delimiter and the second delimiter is recorded as the second subsequence, and the subsequence between the second delimiter and the end of the engineering text sequence is recorded as the third subsequence, thereby deriving the first vector, the second vector and the third vector. It should be noted that the linear distances of the first vector, the second vector and the third vector need to be calculated two by two, and it is necessary to ensure that the calculated linear distances are greater than the set thresholds. Subsequently, the indicator data is used for the first vector, the second vector and the third vector to form three training data.

[0108] In one embodiment, the step S8 of training the preset neural network model in a supervised learning manner according to each of the first training data and the second training data to obtain the engineering data analysis model includes:

[0109] S801: Divide the plurality of first training data and the plurality of second training data into a training data set and a validation data set according to a preset ratio;

[0110] S802: Inputting the training data in the training data set into a preset neural network model, and performing training processing in a supervised learning manner to obtain an intermediate data analysis model;

[0111] S803: Using the verification data in the verification data set to perform verification processing on the intermediate data analysis model to obtain a verification result;

[0112] S804: Determine whether the verification result is passed;

[0113] S805: If the verification result passes, the intermediate data analysis model is recorded as the engineering data analysis model.

[0114] As described in the above steps S801-S805, the data used in the training process and the verification process are all sample data from the same source, and the engineering data analysis model obtained by training has a certain reliability. And each data is labeled, and the training process can adopt a supervised learning method. Specifically, the model training can be carried out using a stochastic gradient descent algorithm, and the parameters of the neural network of each layer of the model are updated by the back propagation algorithm. If the verification result is passed, it means that the final engineering data analysis model can analyze the engineering data well.

[0115] Reference Figure 2 The present invention also provides an engineering data pattern processing system based on machine learning, comprising:

[0116] The acquisition module 10 is used to acquire multiple sets of engineering data sets, and use a preset feature extraction model to extract feature information in each set of the engineering data sets to obtain an engineering information set; wherein the engineering data sets include sensor data, process parameters and / or equipment status;

[0117] A preprocessing module 20, used for preprocessing each of the engineering information sets to obtain a standard engineering information set;

[0118] A serialization module 30, used to serialize the characteristic information in the standard engineering information set according to a preset serialization operation to obtain text information in a text form corresponding to each characteristic information;

[0119] The arrangement module 40 is used to arrange the text information in a preset order and separate them with separators to obtain an engineering text sequence, and select a target separator from the engineering text sequence, record the subsequence of the engineering text sequence before the target separator as a first sequence, and record the subsequence of the engineering text sequence after the target separator as a second sequence;

[0120] A vectorization module 50 is used to perform vectorization operations on the first sequence and the second sequence respectively to obtain a first vector and a second vector, and perform similarity calculation on the first vector and the second vector, and perform sigmoid nonlinear mapping on the obtained similarity value to the (0, 1) interval to calculate the linear distance from 1;

[0121] A setting module 60, configured to use the first vector and the second vector as features of a corresponding engineering data set when the linear distance is greater than a set threshold;

[0122] An acquisition module 70 is used to acquire the indicator data corresponding to each engineering data set, and annotate the indicator data with the first vector and the second vector of the corresponding engineering data set to obtain the first training data and the second training data;

[0123] A training module 80, configured to train a preset neural network model in a supervised learning manner according to each of the first training data and the second training data, so as to obtain an engineering data analysis model;

[0124] An execution module 90 is used to perform the same processing operation on the engineering data set to be tested as that on the engineering data set to obtain a corresponding target first vector and a target second vector;

[0125] The input module 100 is used to input the target first vector and the target second vector into the engineering data analysis model to obtain the target indicator data of the engineering data set to be tested.

[0126] In one embodiment, the vectorization module 50 includes:

[0127] The first similarity value calculation submodule is used to calculate the similarity value through the formula similarity a ⃗ , b ⃗ = ∑ i= 1 n a ⃗ [ i ]∗ b ⃗ [ i ] ∑ i= 1 n a ⃗ [ i ]∗ a ⃗ [ i ] ∑ i= 1 n b ⃗ [ i ]∗ b ⃗ [ i ] The similarity value is calculated, wherein, is the similarity value, represents the first vector, represents the second vector, a ⃗ [ i ] represents the i-th dimension of the first vector, b ⃗ [ i ] represents the i-th dimension of the second vector;

[0128] The second similarity value calculation submodule is used to calculate the similarity value through the formula Calculate the mapping value of the nonlinear mapping to the (0,1) interval; where, Represents a mapping value;

[0129] The similarity value third calculation submodule is used to calculate the linear distance from 1 according to the mapping value.

[0130] In one embodiment, the engineering data pattern processing system based on machine learning further includes:

[0131] A distance acquisition module, used for acquiring a first sequence distance from each target delimiter to the beginning of the engineering text sequence, and a second sequence distance from each target delimiter to the end of the engineering text sequence;

[0132] Distance score calculation module, used to calculate the distance score according to the formula Calculate the distance score of each target separator, wherein K is the distance score, X is the first sequence distance, and Y is the second sequence distance;

[0133] A distance score judgment module, used to judge whether the distance score is less than a preset distance score;

[0134] The determination module is used to determine that the selected target separation is reasonable if the distance is less than a preset distance score.

[0135] In one embodiment, the engineering data pattern processing system based on machine learning further includes:

[0136] A sequence length detection module, used to detect the sequence length of the engineering text sequence;

[0137] The target separator selection module is used to select multiple target separators when the sequence length is greater than a preset sequence length.

[0138] In one embodiment, the training module 80 includes:

[0139] A division submodule, used for dividing the plurality of the first training data and the plurality of the second training data into a training data set and a verification data set according to a preset ratio;

[0140] An input submodule, used to input the training data in the training data set into a preset neural network model, and perform training processing in a supervised learning manner to obtain an intermediate data analysis model;

[0141] An adopting submodule is used to use the verification data in the verification data set to perform verification processing on the intermediate data analysis model to obtain a verification result;

[0142] The judgment submodule is used to judge whether the verification result is passed;

[0143] The marking submodule is used to record the intermediate data analysis model as the engineering data analysis model if the verification result passes.

[0144] The beneficial effects of the present invention are as follows: by performing feature extraction, preprocessing, serialization, separation and other operations on the engineering data set, the engineering data set is divided into a first vector and a second vector, thereby ensuring the independence of the first vector and the second vector, and ensuring the correlation of the engineering data contained in any one of the first vector and the second vector, so that the engineering data analysis model has accurate sample data for training, thereby improving the analysis accuracy of the engineering data analysis model and improving the analysis effect.

[0145] Reference Figure 3 In an embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface, a database, a display screen and an input device connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store various engineering data, etc. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the engineering data pattern processing method based on machine learning described in any of the above embodiments can be implemented.

[0146] Those skilled in the art will understand that Figure 3 The structure shown in is merely a block diagram of a portion of the structure related to the present application solution and does not constitute a limitation on the computer device to which the present application solution is applied.

[0147] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the engineering data pattern processing method based on machine learning described in any of the above embodiments can be implemented.

[0148] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0149] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, device, article or method including the element.

[0150] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0151] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0152] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of the claims of the present invention.

Claims

1. A method for processing engineering data patterns based on machine learning, characterized in that: include: Collect multiple sets of engineering data sets, and use a preset feature extraction model to extract feature information from each set of the engineering data sets to obtain an engineering information set; wherein the engineering data sets include sensor data, process parameters, and equipment status; Preprocessing each of the engineering information sets to obtain a standard engineering information set; Serializing the characteristic information in the standard engineering information set according to a preset serialization operation to obtain text information in a text form corresponding to each characteristic information; Arrange the text information in a preset order and separate them with delimiters to obtain an engineering text sequence, select a target delimiter from the engineering text sequence, record the subsequence of the engineering text sequence before the target delimiter as a first sequence, and record the subsequence of the engineering text sequence after the target delimiter as a second sequence; Vectorize the first sequence and the second sequence respectively to obtain a first vector and a second vector, calculate the similarity between the first vector and the second vector, perform sigmoid nonlinear mapping on the obtained similarity value to the (0,1) interval, and calculate the linear distance from 1; When the linear distance is greater than a set threshold, the first vector and the second vector are used as features of the corresponding engineering data set; Obtain indicator data corresponding to each engineering data set, and annotate a first vector and a second vector of the corresponding engineering data set based on the indicator data to obtain first training data and second training data; According to each of the first training data and each of the second training data, a preset neural network model is trained in a supervised learning manner to obtain an engineering data analysis model; Performing the same processing operation on the engineering data set to be tested as that on the engineering data set to obtain a corresponding first target vector and a second target vector; Inputting the target first vector and the target second vector into the engineering data analysis model to obtain target indicator data of the engineering data set to be tested; After the step of selecting the target delimiter from the engineering text sequence, the method further includes: Obtaining a first sequence distance from each target delimiter to the beginning of the engineering text sequence, and a second sequence distance from each target delimiter to the end of the engineering text sequence; According to the formula Calculate the distance score of each target separator, wherein K is the distance score, X is the first sequence distance, and Y is the second sequence distance; Determining whether the distance score is less than a preset distance score; If it is less than the preset distance score, it is determined that the selected target separation is reasonable.

2. The method for processing engineering data patterns based on machine learning according to claim 1, characterized in that: The step of calculating the similarity between the first vector and the second vector, performing sigmoid nonlinear mapping on the obtained similarity value to the interval (0, 1), and calculating the linear distance from 1 includes: By formula The similarity value is calculated, wherein, is the similarity value, represents the first vector, represents the second vector, represents the i-th dimension of the first vector, represents the i-th dimension of the second vector; By formula Calculate the mapping value of the nonlinear mapping to the (0,1) interval; where, Represents a mapping value; The linear distance from 1 is calculated according to the mapped value.

3. The method for processing engineering data patterns based on machine learning according to claim 1, characterized in that: The step of training the preset neural network model in a supervised learning manner according to each of the first training data and the second training data to obtain the engineering data analysis model includes: Dividing the plurality of the first training data and the plurality of the second training data into a training data set and a validation data set according to a preset ratio; Inputting the training data in the training data set into a preset neural network model, and performing training processing in a supervised learning manner to obtain an intermediate data analysis model; Using the verification data in the verification data set to perform verification processing on the intermediate data analysis model to obtain a verification result; Determine whether the verification result is passed; If the verification result passes, the intermediate data analysis model is recorded as the engineering data analysis model.

4. An engineering data pattern processing system based on machine learning, characterized in that: include: The acquisition module is used to acquire multiple sets of engineering data sets, and use a preset feature extraction model to extract feature information from each set of the engineering data sets to obtain an engineering information set; wherein the engineering data sets include sensor data, process parameters, and equipment status; A preprocessing module, used for preprocessing each of the engineering information sets to obtain a standard engineering information set; A serialization module, used to serialize the characteristic information in the standard engineering information set according to a preset serialization operation to obtain text information in a text form corresponding to each characteristic information; An arrangement module is used to arrange each text information in a preset order and separate them with separators to obtain an engineering text sequence, and select a target separator from the engineering text sequence, record the subsequence of the engineering text sequence before the target separator as a first sequence, and record the subsequence of the engineering text sequence after the target separator as a second sequence; A vectorization module, used for performing vectorization operations on the first sequence and the second sequence respectively to obtain a first vector and a second vector, performing similarity calculation on the first vector and the second vector, and performing sigmoid nonlinear mapping on the obtained similarity value to the (0,1) interval to obtain a linear distance from 1; A setting module, configured to use the first vector and the second vector as features of a corresponding engineering data set when the linear distance is greater than a set threshold; An acquisition module, used to acquire indicator data corresponding to each engineering data set, and annotate the first vector and the second vector of the corresponding engineering data set based on the indicator data to obtain the first training data and the second training data; A training module, used to train a preset neural network model in a supervised learning manner according to each first training data and each second training data, so as to obtain an engineering data analysis model; An execution module, used for performing the same processing operation on the engineering data set to be tested as that on the engineering data set, so as to obtain a corresponding target first vector and a target second vector; An input module, used for inputting the target first vector and the target second vector into the engineering data analysis model to obtain target indicator data of the engineering data set to be tested; A distance acquisition module, used for acquiring a first sequence distance from each target delimiter to the beginning of the engineering text sequence, and a second sequence distance from each target delimiter to the end of the engineering text sequence; Distance score calculation module, used to calculate the distance score according to the formula Calculate the distance score of each target separator, wherein K is the distance score, X is the first sequence distance, and Y is the second sequence distance; A distance score judgment module, used to judge whether the distance score is less than a preset distance score; The determination module is used to determine that the selected target separation is reasonable if the distance is less than a preset distance score.

5. The engineering data pattern processing system based on machine learning according to claim 4, characterized in that: The vectorization module comprises: The first similarity value calculation submodule is used to calculate the similarity value through the formula The similarity value is calculated, wherein, is the similarity value, represents the first vector, represents the second vector, represents the i-th dimension of the first vector, represents the i-th dimension of the second vector; The second similarity value calculation submodule is used to calculate the similarity value through the formula Calculate the mapping value of the nonlinear mapping to the (0,1) interval; where, Represents a mapping value; The similarity value third calculation submodule is used to calculate the linear distance from 1 according to the mapping value.

6. The engineering data pattern processing system based on machine learning according to claim 4, characterized in that: The training module comprises: A division submodule, used for dividing the plurality of the first training data and the plurality of the second training data into a training data set and a verification data set according to a preset ratio; An input submodule, used to input the training data in the training data set into a preset neural network model, and perform training processing in a supervised learning manner to obtain an intermediate data analysis model; An adopting submodule is used to use the verification data in the verification data set to perform verification processing on the intermediate data analysis model to obtain a verification result; The judgment submodule is used to judge whether the verification result is passed; The marking submodule is used to record the intermediate data analysis model as the engineering data analysis model if the verification result passes.

Citation Information

Patent Citations

  • Sample collection method and system based on feature space

    CN110147845A

  • Mark query method and system for similar paragraphs of document, equipment and storage medium

    CN113139374A

  • Text feature extraction method and device, computer equipment and storage medium

    CN113449081A

  • Query processing model generation method and device and electronic equipment

    CN114117183A

  • BERT and pooling-free convolutional neural network-based text classification method

    CN116340506A