Data processing method, device and computer-readable medium based on machine learning

Through the correlation analysis of data characteristics and deep learning network fusion, combined with overfitting parameters to adjust the data enhancement method and learning rate, the problems of information loss and overfitting in data processing are solved, and the generalization ability and processing of the model are improved efficiency.

CN118228795BActive Publication Date: 2024-11-05JINGWAN TECH (NANTONG) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410027249.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-08
Publication Date
2024-11-05
Estimated Expiration
2044-01-08

AI Technical Summary

Technical Problem

Existing data processing methods may lead to information loss during feature selection and dimensionality reduction, and may lead to model overfitting, reducing generalization performance on new data, and domain dependence affects poor data processing results.

Method used

By performing correlation analysis on data features, dynamically identify high-correlation feature groups, and reconstruct and fuse unselected features through deep learning networks, calculate the overfit parameters of the learning model, adjust the data enhancement method and learning rate, Control model complexity and generalization capabilities.

Benefits of technology

Effectively prevent training data from being overfitted, improve the generalization ability and stability of data processing, and improve data processing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118228795B_ABST
    Figure CN118228795B_ABST
Patent Text Reader

Abstract

The method of the embodiment of the present invention proposes a data processing method, device and computer-readable medium based on machine learning. The method first performs correlation analysis on the features of the data, dynamically identifies and selects the feature group with high correlation, and records the information of the unselected features; then reconstructs the unselected features through a deep learning network and merges them with the feature group with high correlation; then calculates the performance difference and model complexity of the learning model, and calculates the current overfitting parameters of the learning model; then replaces the data enhancement method through preset rules based on the current overfitting parameters to adjust the data enhancement strength and type; finally, after post-updating the learning rate, adjust the learning model. The method of the present invention controls the selection of data enhancement types and strengths by calculating overfitting parameters, effectively preventing overfitting of training data, having better generalization ability and stability, and improving data processing efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to the field of information technology, and in particular to a data processing method, device and computer-readable medium based on machine learning. [Background technology]

[0002] Data processing is a very important field in modern information technology. Existing data processing methods may cause information loss in some data processing steps, such as feature selection or dimensionality reduction. In addition, excessive data processing may cause the model to overfit the training data, reducing its generalization performance on new data. Furthermore, domain dependence is also an important reason for unsatisfactory data processing results. [Summary of the invention]

[0003] In view of this, an embodiment of the present invention provides a data processing method, device and computer-readable medium based on machine learning.

[0004] In a first aspect, an embodiment of the present invention provides a data processing method based on machine learning, the method comprising:

[0005] S1. Perform correlation analysis on the features of the data, dynamically identify and select highly correlated feature groups, and record information on unselected features;

[0006] S2, after reconstructing the unselected features through the deep learning network, it is fused with the highly correlated feature group;

[0007] S3, after calculating the performance difference and model complexity of the learning model, calculate the current overfitting parameters of the learning model;

[0008] S4. Based on the current overfitting parameter, the data enhancement method is replaced by a preset rule to adjust the data enhancement strength and type;

[0009] S5. Post-update the learning rate and adjust the learning model.

[0010] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S1 includes:

[0011] S11, through the formula Calculate the first correlation coefficient between two features, where r xy is the first correlation coefficient between features x and y, x i is the i-th observation value of feature x, y i is the i-th observation value of feature y, is the average value of all observed values ​​of feature x, is the average value of all observed values ​​of feature y;

[0012] S12, through the formula Calculate the second correlation coefficient between two features, where s xy is the second correlation coefficient between features x and y, d i is the ranking difference between features x and y on the i-th observation, and n is the total number of observations;

[0013] S13, based on the first correlation coefficient, the second correlation coefficient, the mutual information and the cosine similarity, by the formula CSM = ω1r xy +ω2s xy +ω3I xy +ω4cos xy Calculate the comprehensive correlation coefficient of the two features, CSM is the comprehensive correlation coefficient; ω1 is the first weight value, ω2 is the second weight value, ω3 is the third weight value, ω4 is the fourth weight value, ω1+ω2+ω3+ω4=1; mutual information is calculated by the formula Calculate, where p(x i ,y i ) is the joint probability of features x and y taking the i-th observation value at the same time, p(x i ) is the marginal probability of feature x taking the i-th observation value, p(y i ) is the marginal probability of feature y taking the i-th observation value; cos xy is the cosine similarity of features x and y;

[0014] S14. An adaptive feature graph is constructed with features as nodes and the comprehensive correlation coefficients between features as edges. A detection algorithm is used to identify tightly connected node groups as high-correlation feature groups, and information on unselected features is recorded.

[0015] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S2 includes:

[0016] S21, using the unselected features as the data set as input, constructing an autoencoder network including an input layer, an encoding layer, a latent space, a decoding layer, and an output layer;

[0017] Among them, the number of neurons in the input layer is the same as the number of features of the data set, which is N;

[0018] The coding layer is provided with m sub-coding layers. The number of neurons in the first sub-coding layer is N1=N-θ, the number of neurons in the second sub-coding layer is N2=N1-θ, ..., the number of neurons in the mth sub-coding layer is N m =N m-1 -θ, θ is a random positive integer within a preset interval;

[0019] The number of neurons in the latent space is For the preset positive integer, the latent space maps high-dimensional data to low-dimensional space;

[0020] The number of sub-decoding layers of the decoding layer is also m, and the number of neurons in each decoding layer is the same as the number of neurons in the corresponding encoding layer; the calculation formula for the output of the mth decoding layer is h′ m =RELU(W m z+b m ), h′ m is the output of the mth decoding layer, RELU is the activation function, W m The weight matrix of the mth decoding layer, z is the output of the latent space, b m The bias term of the mth decoding layer;

[0021] The number of neurons in the output layer is the same as that in the input layer, which is also N. The reconstruction loss function of the output layer is x i is the i-th input data, For the i-th reconstructed data, when the result of the loss function reaches the preset threshold, the output To reconstruct features;

[0022] S22, initialize weights {w1,w2,...,w n} and the feature set {X1,X1,...,X n}, and use accuracy and mean square error as performance indicators of the training task;

[0023] S23, after each training cycle, through the formula Update the weights, is the updated weight, is the current weight, ΔP is the change in performance index, α is the learning rate, is the performance index relative to the weight gradient;

[0024] S24, based on the adjusted weights through formula X fused =w1·X1+w2·X2+...+w n ·X n Perform feature fusion, X fused The fused features.

[0025] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S3 includes:

[0026] S31. Divide the data set into K subsets and perform K-fold cross validation to obtain multiple validation set errors and calculate the average of the validation set errors. Then the performance index change is calculated by the formula Calculate, Etrain is the training set error;

[0027] S32, through formula C total =β1C param +β2C compute +β3C struct +β4C data Calculate the model complexity, where β1, β2, β3 and β4 are the weight coefficients corresponding to parameter complexity, computational complexity, structural complexity and data processing complexity, and parameter complexity C param =C1, computational complexity C compute =C2, structural complexity C struct =C3+C4, data processing complexity C data =C5+C6, where C1, C2, C3, C4, C5, and C6 are the total number of parameters, the total number of operations per forward pass, the number of model layers, the connection complexity, the number of data preprocessing steps, and the complexity of data transformation;

[0028] S33, through the formula O = γ·C total ·ΔP·CR calculates the current overfitting parameters of the learning model, γ is the fitting weight, and CR is the convergence speed of the learning curve of the learning model.

[0029] According to the above aspects and any possible implementation, an implementation is further provided, where the convergence speed of the learning curve is calculated by the following method:

[0030] At the end of each training cycle, the loss function value of the model is recorded. The loss function value of the model is calculated by the formula Perform calculations;

[0031] By formula Calculate the convergence rate;

[0032] Where ΔL=L t -L t-1 , L t is the loss function value of the tth period, L t-1 is the loss function value of the t-1th cycle, ΔT is the time interval between two cycles;

[0033] γ1, γ2, and γ3 represent the corresponding weights of the first loss, the second loss, and the third loss. Second loss The third loss g p ′ is the current task gradient, μ0 is the prior parameter, n′ is the number of input parameter types; G is the generator model, D is the discriminator model, z is a random noise vector and obeys the probability distribution p z, G(z) is the image generated by the generator model, D(G(z)) is the discrimination result of the discriminator model on the image generated by the generator model; X i is the feature representation of the real image in the i-th layer of the convolutional neural network, G i To generate the feature representation of the image in the i-th layer of the convolutional neural network; is the correlation of the style image in the feature representation of the i-th layer in the convolutional neural network, To generate the correlation of the image in the feature representation of the i-th layer in the convolutional neural network, u and v are the spatial positions of the feature representation.

[0034] According to the above aspects and any possible implementation, an implementation is further provided, wherein S4 includes:

[0035] Construct a configuration set A including data enhancement types and data enhancement strengths = {a1, a2, ..., a i ,...,a n}, where each element of the configuration set is represented as a tuple containing the data enhancement type and the data enhancement strength, that is, a i =(t i ,s i ), t i Represented as the i-th data enhancement type, s i Represented as the i-th data enhancement strength;

[0036] The data enhancement types are divided into four categories: geometric transformation, color adjustment, noise injection and filtering. Geometric transformation includes rotation, scaling, flipping and cropping. Color adjustment includes brightness, contrast, saturation and hue. Noise injection is adding random noise. Filtering includes blur filter and sharpening filter. The data enhancement intensity is divided into several levels.

[0037] Determine the parameter range to which the current overfitting parameter O belongs, and select the data enhancement type and data enhancement strength corresponding to the parameter range.

[0038] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S5 includes:

[0039] By formula Update the learning rate, where a n is the updated learning rate, a min is the minimum learning rate, a max is the maximum learning rate, D j is the current iteration number, D J is the total number of iterations;

[0040] A n Substitute into the learning model In the above example, the learning model is adjusted, Ψ(θ) is the output of the learning model, N is the number of input parameters of the learning model, and F Mix m Output ω under training parameters θ m The corresponding fusion vector, F mix n″ Output ω under training parameters θ n″ The corresponding fusion vector, μ0 is the prior parameter of the learning model, λ is the correction factor of the learning model, ω n″ is the generator output corresponding to the image at the n″th moment, ω m is the generator output corresponding to the image at the mth moment.

[0041] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the fusion vector is constructed based on the fused features.

[0042] In a second aspect, an embodiment of the present invention provides a data processing device based on machine learning, the device comprising:

[0043] The analysis module is used to perform correlation analysis on the features of the data, dynamically identify and select the feature groups with high correlation, and record the information of the unselected features;

[0044] The fusion module is used to reconstruct the unselected features through the deep learning network and then fuse them with the highly correlated feature group;

[0045] A calculation module is used to calculate the performance difference and model complexity of the learning model and calculate the current overfitting parameters of the learning model;

[0046] An adjustment module, used to replace the data enhancement method through preset rules based on the current overfitting parameters to adjust the data enhancement strength and type;

[0047] A processing module is used to adjust the learning model after post-updating the learning rate.

[0048] In a third aspect, an embodiment of the present invention provides a computer-readable medium. The computer-readable medium stores a program code, and when the computer program code is run on a computer, the computer executes the data processing method based on machine learning provided by an embodiment of the present invention.

[0049] One of the above technical solutions has the following beneficial effects:

[0050] The method of the embodiment of the present invention proposes a data processing method, device and computer-readable medium based on machine learning. The method first performs correlation analysis on the features of the data, dynamically identifies and selects the feature group with high correlation, and records the information of the unselected features; then reconstructs the unselected features through a deep learning network and merges them with the feature group with high correlation; then calculates the performance difference and model complexity of the learning model, and calculates the current overfitting parameters of the learning model; then replaces the data enhancement method through preset rules based on the current overfitting parameters to adjust the data enhancement strength and type; finally, after post-updating the learning rate, adjusts the learning model. The method of the present invention controls the selection of data enhancement types and strengths by calculating overfitting parameters, effectively prevents overfitting of training data, has better generalization ability and stability, and improves data processing efficiency.

Brief Description of the Drawings

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A flowchart of a data processing method based on machine learning provided by an embodiment of the present invention;

[0053] Figure 2 A schematic block diagram of a data processing device based on machine learning provided by an embodiment of the present invention;

[0054] Figure 3 A schematic diagram of the hardware structure of a data processing device based on machine learning provided in an embodiment of the present invention. [Specific implementation method]

[0055] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Please refer to Figure 1 , which is a flow chart of a data processing method based on machine learning provided by an embodiment of the present invention. As shown in the figure, the method includes the following steps:

[0057] S1. Perform correlation analysis on the features of the data, dynamically identify and select highly correlated feature groups, and record information on unselected features;

[0058] S2, after reconstructing the unselected features through the deep learning network, it is fused with the highly correlated feature group;

[0059] S3, after calculating the performance difference and model complexity of the learning model, calculate the current overfitting parameters of the learning model;

[0060] S4. Based on the current overfitting parameter, the data enhancement method is replaced by a preset rule to adjust the data enhancement strength and type;

[0061] S5. Post-update the learning rate and adjust the learning model.

[0062] Specifically, the S1 includes:

[0063] S11, through the formula Calculate the first correlation coefficient between two features, where r xy is the first correlation coefficient between features x and y, x i is the i-th observation value of feature x, y i is the i-th observation value of feature y, is the average value of all observed values ​​of feature x, is the average value of all observed values ​​of feature y;

[0064] S12, through the formula Calculate the second correlation coefficient between two features, where s xy is the second correlation coefficient between features x and y, d i is the ranking difference between features x and y on the i-th observation, and n is the total number of observations;

[0065] S13, based on the first correlation coefficient, the second correlation coefficient, the mutual information and the cosine similarity, by the formula CSM = ω1r xy +ω2s xy +ω3I xy +ω4cos xy Calculate the comprehensive correlation coefficient of the two features, CSM is the comprehensive correlation coefficient; ω1 is the first weight value, ω2 is the second weight value, ω3 is the third weight value, ω4 is the fourth weight value, ω1+ω2+ω3+ω4=1; mutual information is calculated by the formula Calculate, where p(x i ,y i ) is the joint probability of features x and y taking the i-th observation value at the same time, p(x i ) is the marginal probability of feature x taking the i-th observation value, p(yi ) is the marginal probability of feature y taking the i-th observation value; cos xy is the cosine similarity of features x and y;

[0066] S14. An adaptive feature graph is constructed with features as nodes and the comprehensive correlation coefficients between features as edges. A detection algorithm is used to identify tightly connected node groups as high-correlation feature groups, and information on unselected features is recorded.

[0067] Specifically, the S2 includes:

[0068] S21, using the unselected features as the data set as input, constructing an autoencoder network including an input layer, an encoding layer, a latent space, a decoding layer, and an output layer;

[0069] Among them, the number of neurons in the input layer is the same as the number of features of the data set, which is N;

[0070] The coding layer is provided with m sub-coding layers. The number of neurons in the first sub-coding layer is N1=N-θ, the number of neurons in the second sub-coding layer is N2=N1-θ, ..., the number of neurons in the mth sub-coding layer is N m =N m-1 -θ, θ is a random positive integer within a preset interval;

[0071] The number of neurons in the latent space is For the preset positive integer, the latent space maps high-dimensional data to low-dimensional space;

[0072] The number of sub-decoding layers of the decoding layer is also m, and the number of neurons in each decoding layer is the same as the number of neurons in the corresponding encoding layer; the calculation formula for the output of the mth decoding layer is h′ m =RELU(W m z+b m ), h′ m is the output of the mth decoding layer, RELU is the activation function, W m The weight matrix of the mth decoding layer, z is the output of the latent space, b m The bias term of the mth decoding layer;

[0073] The number of neurons in the output layer is the same as that in the input layer, which is also N. The reconstruction loss function of the output layer is x i is the i-th input data, For the i-th reconstructed data, when the result of the loss function reaches the preset threshold, the output To reconstruct features;

[0074] S22, initialize weights {w1,w2,...,wn} and the feature set {X1,X1,...,X n}, and use accuracy and mean square error as performance indicators of the training task;

[0075] S23, after each training cycle, through the formula Update the weights, is the updated weight, is the current weight, ΔP is the change in performance index, α is the learning rate, is the performance index relative to the weight gradient;

[0076] S24, based on the adjusted weights through formula X fused =w1·X1+w2·X2+...+w n ·X n Perform feature fusion, X fused The fused features.

[0077] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S3 includes:

[0078] S31. Divide the data set into K subsets and perform K-fold cross validation to obtain multiple validation set errors and calculate the average of the validation set errors. Then the performance index change is calculated by the formula Calculate, E train is the training set error;

[0079] S32, through formula C total =β1C param +β2C compute +β3C struct +β4C data Calculate the model complexity, where β1, β2, β3 and β4 are the weight coefficients corresponding to parameter complexity, computational complexity, structural complexity and data processing complexity, and parameter complexity C param =C1, computational complexity C compute =C2, structural complexity C struct =C3+C4, data processing complexity C data =C5+C6, where C1, C2, C3, C4, C5, and C6 are the total number of parameters, the total number of operations per forward pass, the number of model layers, the connection complexity, the number of data preprocessing steps, and the complexity of data transformation;

[0080] S33, through the formula O = γ·C total ·ΔP·CR calculates the current overfitting parameters of the learning model, γ is the fitting weight, and CR is the convergence speed of the learning curve of the learning model;

[0081] The convergence rate of the learning curve is calculated as follows:

[0082] At the end of each training cycle, the loss function value of the model is recorded. The loss function value of the model is calculated by the formula Perform calculations;

[0083] By formula Calculate the convergence rate;

[0084] Where ΔL=L t -L t-1 , L t is the loss function value of the tth period, L t-1 is the loss function value of the t-1th cycle, ΔT is the time interval between two cycles;

[0085] γ1, γ2, and γ3 represent the corresponding weights of the first loss, the second loss, and the third loss. Second loss The third loss g p ′ is the current task gradient, μ0 is the prior parameter, n′ is the number of input parameter types; G is the generator model, D is the discriminator model, z is a random noise vector and obeys the probability distribution p z , G(z) is the image generated by the generator model, D(G(z)) is the discrimination result of the discriminator model on the image generated by the generator model; X i is the feature representation of the real image in the i-th layer of the convolutional neural network, G i To generate the feature representation of the image in the i-th layer of the convolutional neural network; is the correlation of the style image in the feature representation of the i-th layer in the convolutional neural network, To generate the correlation of the image in the feature representation of the i-th layer in the convolutional neural network, u and v are the spatial positions of the feature representation.

[0086] Specifically, the S4 includes:

[0087] Construct a configuration set A including data enhancement types and data enhancement strengths = {a1, a2, ..., a i ,...,a n}, where each element of the configuration set is represented as a tuple containing the data enhancement type and the data enhancement strength, that is, a i =(t i ,s i ), t i Represented as the i-th data enhancement type, s i Represented as the i-th data enhancement strength;

[0088] The data enhancement types are divided into four categories: geometric transformation, color adjustment, noise injection and filtering. Geometric transformation includes rotation, scaling, flipping and cropping. Color adjustment includes brightness, contrast, saturation and hue. Noise injection is adding random noise. Filtering includes blur filter and sharpening filter. The data enhancement intensity is divided into several levels.

[0089] Determine the parameter range to which the current overfitting parameter O belongs, and select the data enhancement type and data enhancement strength corresponding to the parameter range.

[0090] Specifically, the S5 includes:

[0091] By formula Update the learning rate, where a n is the updated learning rate, a min is the minimum learning rate, a max is the maximum learning rate, D j is the current iteration number, D J is the total number of iterations;

[0092] A n Substitute into the learning model In the above example, the learning model is adjusted, Ψ(θ) is the output of the learning model, N is the number of input parameters of the learning model, and F Mix m Output ω under training parameters θ m The corresponding fusion vector, F mix n″ Output ω under training parameters θ n″ The corresponding fusion vector, μ0 is the prior parameter of the learning model, λ is the correction factor of the learning model, ω n″ is the generator output corresponding to the image at the n″th moment, ω m is the generator output corresponding to the image at the mth moment.

[0093] Specifically, the fusion vector is constructed based on the fused features.

[0094] The method of the embodiment of the present invention proposes a data processing method, device and computer-readable medium based on machine learning. The method first performs correlation analysis on the features of the data, dynamically identifies and selects the feature group with high correlation, and records the information of the unselected features; then reconstructs the unselected features through a deep learning network and merges them with the feature group with high correlation; then calculates the performance difference and model complexity of the learning model, and calculates the current overfitting parameters of the learning model; then replaces the data enhancement method through preset rules based on the current overfitting parameters to adjust the data enhancement strength and type; finally, after post-updating the learning rate, adjusts the learning model. The method of the present invention controls the selection of data enhancement types and strengths by calculating overfitting parameters, effectively prevents overfitting of training data, has better generalization ability and stability, and improves data processing efficiency.

[0095] The embodiments of the present invention further provide device embodiments for implementing the steps and methods in the above method embodiments.

[0096] Please refer to Figure 2 , which is Figure 2 A schematic block diagram of a data processing device based on machine learning provided by an embodiment of the present invention. The device comprises:

[0097] Analysis module 210, used to perform correlation analysis on the features of the data, dynamically identify and select highly correlated feature groups, and record information on unselected features;

[0098] A fusion module 220 is used to reconstruct the unselected features through a deep learning network and then fuse them with a high-correlation feature group;

[0099] A calculation module 230 is used to calculate the performance difference and model complexity of the learning model and calculate the current overfitting parameters of the learning model;

[0100] An adjustment module 240, configured to replace the data enhancement method according to a preset rule based on the current overfitting parameter to adjust the data enhancement strength and type;

[0101] The processing module 250 is used to adjust the learning model after post-updating the learning rate.

[0102] Since each unit module in this embodiment can execute Figure 1 For the method shown in the embodiment, the part not described in detail in this embodiment can be referred to Figure 1 Related instructions.

[0103] Please refer to Figure 3, which is a hardware structure diagram of a data processing device based on machine learning provided in an embodiment of the present invention. The data prediction device includes at least one processor and a memory, wherein the at least one processor is coupled to the memory and is used to read and execute instructions in the memory to execute the data prediction method provided in an embodiment of the present invention.

[0104] In a third aspect, an embodiment of the present invention provides a computer-readable medium. The computer-readable medium stores program code, and when the computer program code is executed on a computer, the computer executes the data processing method based on machine learning provided by an embodiment of the present invention.

[0105] At the hardware level, the device may include a processor, and optionally an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. Of course, the device may also include hardware required for other services.

[0106] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0107] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0108] The steps of the method disclosed in the embodiment of the present invention can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0109] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0110] For the convenience of description, the above device is described as various units or modules according to their functions. Of course, when implementing the present invention, the functions of each unit or module can be implemented in the same or multiple software and / or hardware.

[0111] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0113] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0115] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0116] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0117] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0118] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0119] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0120] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0121] Each embodiment of the present invention is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0122] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A data processing method based on machine learning, characterized in that: The method comprises: S1. Perform correlation analysis on the features of the data, dynamically identify and select highly correlated feature groups, and record information on unselected features; S2, after reconstructing the unselected features through the deep learning network, it is fused with the highly correlated feature group; S3, after calculating the performance difference and model complexity of the learning model, calculate the current overfitting parameters of the learning model; S4. Based on the current overfitting parameter, the data enhancement method is replaced by a preset rule to adjust the data enhancement strength and type; S5, after updating the learning rate, adjust the learning model; The S1 includes: S11, through the formula Calculate the first correlation coefficient between two features, where r xy is the first correlation coefficient between features x and y, x i is the i-th observation value of feature x, y i is the i-th observation value of feature y, is the average value of all observed values ​​of feature x, is the average value of all observed values ​​of feature y; S12, through the formula Calculate the second correlation coefficient between two features, where s xy is the second correlation coefficient between features x and y, d i is the ranking difference between features x and y on the i-th observation, and n is the total number of observations; S13, based on the first correlation coefficient, the second correlation coefficient, the mutual information and the cosine similarity, by the formula CSM = ω1r xy +ω2s xy +ω3I xy +ω4cos xy Calculate the comprehensive correlation coefficient of two features, CSM is the comprehensive correlation coefficient; ω1 is the first weight value, ω2 is the second weight value, ω3 is the third weight value, ω4 is the fourth weight value, ω1+ω2+ω3+ω4=1; mutual information is calculated by the formula Calculate, where p(x i ,y i ) is the joint probability of features x and y taking the i-th observation value at the same time, p(x i ) is the marginal probability of feature x taking the i-th observation value, p(y i ) is the marginal probability of feature y taking the i-th observation value; cos xy is the cosine similarity of features x and y; S14, constructing an adaptive feature graph with features as nodes and comprehensive correlation coefficients between features as edges, identifying closely connected node groups as high-correlation feature groups through a detection algorithm, and recording information of unselected features; The S2 includes: S21, using the unselected features as the data set as input, constructing an autoencoder network including an input layer, an encoding layer, a latent space, a decoding layer, and an output layer; Among them, the number of neurons in the input layer is the same as the number of features of the data set, which is N; The coding layer is provided with m sub-coding layers. The number of neurons in the first sub-coding layer is N1=N-θ, the number of neurons in the second sub-coding layer is N2=N1-θ, ..., the number of neurons in the mth sub-coding layer is N m =N m-1 -θ, θ is a random positive integer within a preset interval; The number of neurons in the latent space is For the preset positive integer, the latent space maps high-dimensional data to low-dimensional space; The number of sub-decoding layers of the decoding layer is also m, and the number of neurons in each decoding layer is the same as the number of neurons in the corresponding encoding layer; the calculation formula for the output of the mth decoding layer is h′ m =RELU(W m z+b m ), h′ m is the output of the mth decoding layer, RELU is the activation function, W m The weight matrix of the mth decoding layer, z is the output of the latent space, b m The bias term of the mth decoding layer; The number of neurons in the output layer is the same as that in the input layer, which is also N. The reconstruction loss function of the output layer is x i is the i-th input data, For the i-th reconstructed data, when the result of the loss function reaches the preset threshold, the output To reconstruct features; S22, initialize weights {w1,w2,...,w n } and the feature set {X1,X1,...,X n }, and use accuracy and mean square error as performance indicators of the training task; S23, after each training cycle, through the formula Update the weights, is the updated weight, is the current weight, ΔP is the change in performance index, α is the learning rate, is the performance index relative to the weight gradient; S24, based on the adjusted weights through formula X fused =w1·X1+w2·X2+…+w n ·X n Perform feature fusion, X fused is the fused feature; The S3 includes: S31. Divide the data set into K subsets and perform K-fold cross validation to obtain multiple validation set errors and calculate the average of the validation set errors. Then the performance index change is calculated by the formula Calculate, E train is the training set error; S32, through formula C total =β1C param +β2C compute +β3C struct +β4C data Calculate the model complexity, where β1, β2, β3 and β4 are the weight coefficients corresponding to parameter complexity, computational complexity, structural complexity and data processing complexity, and parameter complexity C param =C1, computational complexity C compute =C2, structural complexity C struct =C3+C4, data processing complexity C data =C5+C6, where C1, C2, C3, C4, C5, and C6 are the total number of parameters, the total number of operations per forward pass, the number of model layers, the connection complexity, the number of data preprocessing steps, and the complexity of data transformation; S33, through the formula O = γ·C total ·ΔP·CR calculates the current overfitting parameters of the learning model, γ is the fitting weight, and CR is the convergence speed of the learning curve of the learning model; The convergence rate of the learning curve is calculated as follows: At the end of each training cycle, the loss function value of the model is recorded. The loss function value of the model is calculated by the formula Perform calculations; By formula Calculate the convergence rate; Where ΔL=L t -L t-1 , L t is the loss function value of the tth period, L t-1 is the loss function value of the t-1th cycle, ΔT is the time interval between two cycles; γ1, γ2, and γ3 represent the corresponding weights of the first loss, the second loss, and the third loss. Second loss The third loss g p ′ is the current task gradient, μ0 is the prior parameter, n′ is the number of input parameter types; G is the generator model, D is the discriminator model, z is a random noise vector and obeys the probability distribution p z , G(z) is the image generated by the generator model, D(G(z)) is the discrimination result of the discriminator model on the image generated by the generator model; X i is the feature representation of the real image in the i-th layer of the convolutional neural network, G i To generate the feature representation of the image in the i-th layer of the convolutional neural network; is the correlation of the style image in the feature representation of the i-th layer in the convolutional neural network, To generate the correlation of the image in the feature representation of the i-th layer in the convolutional neural network, u and v are the spatial positions of the feature representation; The S4 includes: Construct a configuration set A including data enhancement types and data enhancement strengths = {a1, a2, ..., a i ,...,a n }, where each element of the configuration set is represented as a tuple containing the data enhancement type and the data enhancement strength, that is, a i =(t i ,s i ), t i Represented as the i-th data enhancement type, s i Represented as the i-th data enhancement strength; The data enhancement types are divided into four categories: geometric transformation, color adjustment, noise injection and filtering. Geometric transformation includes rotation, scaling, flipping and cropping. Color adjustment includes brightness, contrast, saturation and hue. Noise injection is adding random noise. Filtering includes blur filter and sharpening filter. The data enhancement intensity is divided into several levels. Determine the parameter range to which the current overfitting parameter O belongs, and select the data enhancement type and data enhancement strength corresponding to the parameter range; The S5 includes: By formula Update the learning rate, where a n is the updated learning rate, a min is the minimum learning rate, a max is the maximum learning rate, D j is the current iteration number, D J is the total number of iterations; A n Substitute into the learning model In the above example, the learning model is adjusted, Ψ(θ) is the output of the learning model, N is the number of input parameters of the learning model, and F Mix m Output ω under training parameters θ m The corresponding fusion vector, F mix n″ Output ω under training parameters θ n″ The corresponding fusion vector, μ0 is the prior parameter of the learning model, λ is the correction factor of the learning model, ω n″ is the generator output corresponding to the image at the n″th moment, ω m For the m The generator output corresponding to the image at that moment.

2. The data processing method based on machine learning according to claim 1, characterized in that: The fusion vector is constructed based on the fused features.

3. A data processing device based on machine learning, characterized in that: The device comprises: The analysis module is used to perform correlation analysis on the features of the data, dynamically identify and select the feature groups with high correlation, and record the information of the unselected features; The fusion module is used to reconstruct the unselected features through the deep learning network and then fuse them with the highly correlated feature group; A calculation module is used to calculate the performance difference and model complexity of the learning model and calculate the current overfitting parameters of the learning model; An adjustment module, used to replace the data enhancement method through preset rules based on the current overfitting parameters to adjust the data enhancement strength and type; A processing module is used to adjust the learning model after updating the learning rate; The analysis module is specifically used to perform: S11, through the formula Calculate the first correlation coefficient between two features, where r xy is the first correlation coefficient between features x and y, x i is the i-th observation value of feature x, y i is the i-th observation value of feature y, is the average value of all observed values ​​of feature x, is the average value of all observed values ​​of feature y; S12, through the formula Calculate the second correlation coefficient between two features, where s xy is the second correlation coefficient between features x and y, d i is the ranking difference between features x and y on the i-th observation, and n is the total number of observations; S13, based on the first correlation coefficient, the second correlation coefficient, the mutual information and the cosine similarity, by the formula CSM = ω1r xy +ω2s xy +ω3I xy +ω4cos xy Calculate the comprehensive correlation coefficient of two features, CSM is the comprehensive correlation coefficient; ω1 is the first weight value, ω2 is the second weight value, ω3 is the third weight value, ω4 is the fourth weight value, ω1+ω2+ω3+ω4=1; mutual information is calculated by the formula Calculate, where p(x i ,y i ) is the joint probability of features x and y taking the i-th observation value at the same time, p(x i ) is the marginal probability of feature x taking the i-th observation value, p(y i ) is the marginal probability of feature y taking the i-th observation value; cos xy is the cosine similarity of features x and y; S14, constructing an adaptive feature graph with features as nodes and comprehensive correlation coefficients between features as edges, identifying closely connected node groups as high-correlation feature groups through a detection algorithm, and recording information of unselected features; The fusion module is specifically used to perform: S21, using the unselected features as the data set as input, constructing an autoencoder network including an input layer, an encoding layer, a latent space, a decoding layer, and an output layer; Among them, the number of neurons in the input layer is the same as the number of features of the data set, which is N; The coding layer is provided with m sub-coding layers. The number of neurons in the first sub-coding layer is N1=N-θ, the number of neurons in the second sub-coding layer is N2=N1-θ, ..., the number of neurons in the mth sub-coding layer is N m =N m-1 -θ, θ is a random positive integer within a preset interval; The number of neurons in the latent space is For the preset positive integer, the latent space maps high-dimensional data to low-dimensional space; The number of sub-decoding layers of the decoding layer is also m, and the number of neurons in each decoding layer is the same as the number of neurons in the corresponding encoding layer; the calculation formula for the output of the mth decoding layer is h′ m =RELU(W m z+b m ), h′ m is the output of the mth decoding layer, RELU is the activation function, W m The weight matrix of the mth decoding layer, z is the output of the latent space, b m The bias term of the mth decoding layer; The number of neurons in the output layer is the same as that in the input layer, which is also N. The reconstruction loss function of the output layer is x i is the i-th input data, For the i-th reconstructed data, when the result of the loss function reaches the preset threshold, the output To reconstruct features; S22, initialize weights {w1,w2,...,w n } and the feature set {X1,X1,...,X n }, and use accuracy and mean square error as performance indicators of the training task; S23, after each training cycle, through the formula Update the weights, is the updated weight, is the current weight, ΔP is the change in performance index, α is the learning rate, is the performance index relative to the weight gradient; S24, based on the adjusted weights through formula X fused =w1·X1+w2·X2+…+w n ·X n Perform feature fusion, X fused is the fused feature; The computing module is specifically used to execute: S31. Divide the data set into K subsets and perform K-fold cross validation to obtain multiple validation set errors and calculate the average of the validation set errors. Then the performance index change is calculated by the formula Calculate, E train is the training set error; S32, through formula C total =β1C param +β2C compute +β3C struct +β4C data Calculate the model complexity, where β1, β2, β3 and β4 are the weight coefficients corresponding to parameter complexity, computational complexity, structural complexity and data processing complexity, and parameter complexity C param =C1, computational complexity C compute =C2, structural complexity C struct =C3+C4, data processing complexity C data =C5+C6, where C1, C2, C3, C4, C5, and C6 are the total number of parameters, the total number of operations per forward pass, the number of model layers, the connection complexity, the number of data preprocessing steps, and the complexity of data transformation; S33, through the formula O = γ·C total ·ΔP·CR calculates the current overfitting parameters of the learning model, γ is the fitting weight, and CR is the convergence speed of the learning curve of the learning model; The convergence rate of the learning curve is calculated as follows: At the end of each training cycle, the loss function value of the model is recorded. The loss function value of the model is calculated by the formula Perform calculations; By formula Calculate the convergence rate; Where ΔL=L t -L t-1 , L t is the loss function value of the tth period, L t-1 is the loss function value of the t-1th cycle, ΔT is the time interval between two cycles; γ1, γ2, and γ3 represent the corresponding weights of the first loss, the second loss, and the third loss. Second loss The third loss g p ′ is the current task gradient, μ0 is the prior parameter, n′ is the number of input parameter types; G is the generator model, D is the discriminator model, z is a random noise vector and obeys the probability distribution p z , G(z) is the image generated by the generator model, D(G(z)) is the discrimination result of the discriminator model on the image generated by the generator model; X i is the feature representation of the real image in the i-th layer of the convolutional neural network, G i To generate the feature representation of the image in the i-th layer of the convolutional neural network; is the correlation of the style image in the feature representation of the i-th layer in the convolutional neural network, To generate the correlation of the image in the feature representation of the i-th layer in the convolutional neural network, u and v are the spatial positions of the feature representation; The adjustment module is specifically used to perform: Construct a configuration set A including data enhancement types and data enhancement strengths = {a1, a2, ..., a i ,...,a n }, where each element of the configuration set is represented as a tuple containing the data enhancement type and the data enhancement strength, that is, a i =(t i ,s i ), t i Represented as the i-th data enhancement type, s i Represented as the i-th data enhancement strength; The data enhancement types are divided into four categories: geometric transformation, color adjustment, noise injection and filtering. Geometric transformation includes rotation, scaling, flipping and cropping. Color adjustment includes brightness, contrast, saturation and hue. Noise injection is adding random noise. Filtering includes blur filter and sharpening filter. The data enhancement intensity is divided into several levels. Determine the parameter range to which the current overfitting parameter O belongs, and select the data enhancement type and data enhancement strength corresponding to the parameter range; The processing module is specifically used to execute: By formula Update the learning rate, where a n is the updated learning rate, a min is the minimum learning rate, a max is the maximum learning rate, D j is the current iteration number, D J is the total number of iterations; A n Substitute into the learning model In the above example, the learning model is adjusted, Ψ(θ) is the output of the learning model, N is the number of input parameters of the learning model, and F Mix m Output ω under training parameters θ m The corresponding fusion vector, F mix n″ Output ω under training parameters θ n″ The corresponding fusion vector, μ0 is the prior parameter of the learning model, λ is the correction factor of the learning model, ω n″ is the generator output corresponding to the image at the n″th moment, ω m For the m The generator output corresponding to the image at that moment.

4. A computer-readable medium, characterized in that The computer-readable medium stores a program code, and when the program code is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 2.