Data processing method and device, equipment and storage medium
By using the trained data classification model to identify and filter effective data in the communication data to be processed, the problem of low data preprocessing accuracy in the prior art is solved, and efficient resource utilization and data processing efficiency are achieved.
Patent Information
- Application Number
- CN202411243061.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-12-31
AI Technical Summary
In the prior art, the accuracy of data preprocessing is not high, resulting in the problem of resource waste.
By obtaining valid data, invalid data and mixed data in the communication data to be processed, the trained data classification model is used to identify data, and the target valid data is generated, the data errors in the manual classification process are reduced, and the accuracy of effective data is improved.
Improve the accuracy of effective data, avoid resource waste, and improve data processing efficiency.
Smart Images
Figure CN119226880A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method, device, equipment and storage medium. Background Art
[0002] During model training, a large amount of manual work is required to preprocess the training samples. The main purpose of these preprocessing processes is to eliminate invalid samples and improve training efficiency. However, the number of valid samples required to train a model is large, and the workload of data preprocessing is large. In addition, due to the manual preprocessing, some valid data may be regarded as invalid data, or vice versa, which will affect the subsequent model training efficiency and cause a waste of data resources.
[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention
[0004] The main purpose of the present invention is to provide a data processing method, device, equipment and storage medium, aiming to solve the technical problem of low accuracy of data preprocessing and waste of resources in the prior art.
[0005] To achieve the above object, the present invention provides a data processing method, which comprises the following steps:
[0006] Acquire first valid data, invalid data, and mixed data in the communication data to be processed;
[0007] Inputting the mixed data into a trained data classification model for data recognition to obtain second valid data;
[0008] Identify third valid data corresponding to the communication data to be processed by the trained data classification model;
[0009] Target valid data is generated according to the first valid data, the second valid data and the third valid data.
[0010] In one embodiment, the obtaining of first valid data, invalid data, and mixed data in the communication data to be processed includes:
[0011] Preprocessing the communication data to be processed to obtain first valid data and invalid data;
[0012] Performing feature extraction on the first valid data to obtain valid feature data;
[0013] Mixed data is generated according to the valid feature data and the invalid data.
[0014] In one embodiment, the preprocessing of the communication data to be processed includes:
[0015] The communication data to be processed is classified according to at least one of data cleaning, data conversion and data normalization.
[0016] In one embodiment, generating mixed data according to the valid feature data and the invalid data includes:
[0017] independently storing the invalid data;
[0018] Determining a target storage interval for the invalid data;
[0019] The test data is imported into the target storage interval based on a preset mixing strategy, the test data is the valid feature data, and the preset mixing strategy is used to mix the test data and the invalid data.
[0020] In one embodiment, before inputting the mixed data into a trained data classification model for data recognition to obtain the second valid data, the method further includes:
[0021] Perform data recognition training through the initial classification model based on effective feature data;
[0022] The model parameters of the initial classification model are optimized through a cross-validation strategy to obtain a trained data classification model.
[0023] In one embodiment, generating target valid data according to the first valid data, the second valid data and the third valid data includes:
[0024] Screening the second valid data to obtain fourth valid data;
[0025] The first valid data, the third valid data and the fourth valid data are used to generate target valid data, where the target valid data is a union of the first valid data, the third valid data and the fourth valid data.
[0026] In one embodiment, screening the second valid data includes:
[0027] Determine the actual valid data in the test data;
[0028] The second valid data is screened according to the actual valid data to obtain fourth valid data, where the fourth valid data is the second valid data remaining after the actual valid data is removed.
[0029] In addition, to achieve the above object, the present invention further provides a data processing device, the data processing device comprising:
[0030] An acquisition module, used for acquiring first valid data, invalid data and mixed data in the communication data to be processed;
[0031] A recognition module, used for inputting the mixed data into a trained data classification model for data recognition to obtain second valid data;
[0032] The identification module is further used to identify third valid data corresponding to the communication data to be processed through the trained data classification model;
[0033] A generating module is used to generate target valid data according to the first valid data, the second valid data and the third valid data.
[0034] In addition, to achieve the above-mentioned purpose, the present invention also proposes a data processing device, which includes: a memory, a processor, and a data processing program stored in the memory and executable on the processor, and the data processing program is configured to implement the steps of the data processing method described above.
[0035] In addition, to achieve the above-mentioned purpose, the present invention further proposes a storage medium, on which a data processing program is stored, and when the data processing program is executed by a processor, the steps of the data processing method described above are implemented.
[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the data processing method described above are implemented.
[0037] One or more technical solutions proposed in the present application have at least the following technical effects: The present application provides a data processing method, which includes: obtaining first valid data, invalid data and mixed data in the communication data to be processed; inputting the mixed data into a trained data classification model for data identification to obtain second valid data; identifying third valid data corresponding to the communication data to be processed through the trained data classification model; generating target valid data according to the first valid data, the second valid data and the third valid data. Compared with the prior art, the present application performs preliminary division of the communication data to be processed to obtain first valid data, invalid data and mixed data, and inputs the mixed data into a trained data classification model for data identification, thereby further screening the communication data to be processed and increasing the proportion of valid data. At the same time, the original communication data to be processed is identified through the trained data classification model to reduce data errors in the manual classification process. Finally, the final target valid data is comprehensively determined by the identified first valid data, second valid data and third valid data, thereby maximizing the accuracy of the valid data and avoiding the technical problem of low accuracy of data preprocessing and waste of resources in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0040] Figure 1 It is a flowchart of a first embodiment of a data processing method of the present invention;
[0041] Figure 2 It is a flowchart of a second embodiment of a data processing method of the present invention;
[0042] Figure 3 It is a flowchart of a third embodiment of a data processing method of the present invention;
[0043] Figure 4 It is a structural block diagram of the first embodiment of the data processing device of the present invention.
[0044] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0045] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.
[0046] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.
[0047] Based on this, the embodiment of the present invention provides a data processing method, referring to Figure 1 , Figure 1 The figure is a flow chart of a first embodiment of a data processing method of the present invention.
[0048] In this embodiment, the data processing method includes the steps of:
[0049] Step S10: Acquire first valid data, invalid data and mixed data in the communication data to be processed.
[0050] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, data processing, etc. The following takes data processing as an example to illustrate this embodiment and the following embodiments.
[0051] It should be understood that the communication data to be processed refers to information carrying communication data between multiple communication terminals such as users and users, devices and devices, and users and devices; the first valid data refers to valid data obtained after manual preprocessing and classification, wherein the first valid data may contain some invalid data generated by manual preprocessing; invalid data refers to invalid data obtained after manual preprocessing and classification, wherein the invalid data may contain some valid data generated by manual preprocessing; mixed data refers to mixed data of invalid data and the first valid data after feature extraction.
[0052] Step S20: input the mixed data into a trained data classification model for data recognition to obtain second valid data.
[0053] The trained data classification model can identify valid data in the input data. During the classification process of invalid data, certain errors may occur due to human errors, and certain valid data may be mixed inside. These valid data cannot be identified through simple classification. Therefore, in this embodiment, the invalid data a and the mixed data are mixed and then screened through the trained data classification model, so that the valid data e in the invalid data a can be further selected.
[0054] Among them, the trained data classification model can also adjust the model parameters according to the prediction results of the mixed data so that the model can be fitted and output data classification results with higher accuracy.
[0055] Furthermore, before inputting the mixed data into a trained data classification model for data recognition to obtain the second valid data, the method further includes:
[0056] Perform data recognition training through the initial classification model based on effective feature data;
[0057] The model parameters of the initial classification model are optimized through a cross-validation strategy to obtain a trained data classification model.
[0058] In the specific implementation, when invalid data a is eliminated, since it is a manual operation and has certain errors, certain valid data will be mixed inside, called valid data e. The invalid data a and the test data are mixed and then screened through the model. Before this, since the first valid data contains valid data b and invalid data b, after feature extraction, the difference between the two can be clearly seen, so a trained data classification model can be trained.
[0059] Specifically, the model training process is to use SVM classifier for training and optimize parameters through cross-validation method. When the training samples are linearly separable, a linear classifier is learned by maximizing the hard interval, that is, a linear separable support vector machine; when the training data is approximately linearly separable, slack variables are introduced, and a linear classifier is learned by maximizing the soft interval, that is, a linear support vector machine; when the training data is linearly inseparable, a nonlinear support vector machine is learned by using kernel techniques and maximizing the soft interval.
[0060] While avoiding overfitting and underfitting, the model's parameters, algorithms, and data are optimized. Parameter adjustment is to optimize the model performance by adjusting the model's hyperparameters, such as learning rate, number of iterations, etc. Algorithm optimization is to improve and optimize the model's algorithm to improve the model's predictive performance. Data enhancement is to improve the model's generalization ability by increasing the amount of data, changing the data distribution, etc.
[0061] Step S30: Identify third valid data corresponding to the communication data to be processed by using the trained data classification model.
[0062] It is understandable that the trained data classification model can also perform data validity classification for subsequently acquired communication data, and thus when the amount of data is large and there is new communication data that needs to be classified, data validity classification can be automatically performed to improve data processing efficiency.
[0063] Step S40: generating target valid data according to the first valid data, the second valid data and the third valid data.
[0064] In this embodiment, since the amount of communication data obtained is large and the communication data is data generated during real-time communication, when data validity classification is performed, the target valid data finally obtained in this embodiment refers to the first valid data manually screened out, the third valid data identified by the trained data classification model, and the valid data that is omitted or incorrectly identified during manual screening.
[0065] This embodiment performs a preliminary division of the communication data to be processed to obtain first valid data, invalid data and mixed data, and inputs the mixed data into a trained data classification model for data identification, thereby further screening the communication data to be processed and increasing the proportion of valid data. At the same time, the original communication data to be processed is identified through the trained data classification model to reduce data errors in the manual classification process. Finally, the final target valid data is comprehensively determined by identifying the first valid data, the second valid data and the third valid data, thereby maximizing the accuracy of the valid data and avoiding the technical problem of low accuracy of data preprocessing and waste of resources in the prior art.
[0066] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , step S10, comprising:
[0067] Step S101: pre-process the communication data to be processed to obtain first valid data and invalid data.
[0068] Step S102: extracting features from the first valid data to obtain valid feature data.
[0069] Step S103: generating mixed data according to the valid feature data and the invalid data.
[0070] It should be noted that preprocessing the communication data to be processed refers to classifying the communication data to be processed into first valid data and invalid data through manual screening. Since manual screening may have errors and the amount of communication data to be processed is large, there may be some invalid data in the first valid data, and there may also be some valid data that is misscreened in the invalid data. Therefore, further screening of the first valid data and invalid data is required to extract valid data with higher accuracy.
[0071] Further, the preprocessing of the communication data to be processed includes:
[0072] The communication data to be processed is classified according to at least one of data cleaning, data conversion and data normalization.
[0073] In a specific implementation, the communication data set to be processed is read and necessary preprocessing is performed to separate valid data a and invalid data a, and the valid data a and invalid data a are classified and marked to facilitate further data classification and identification.
[0074] It is understandable that the data preprocessing steps include data collection, cleaning, conversion and normalization. When collecting data, attention should be paid to the integrity, accuracy and timeliness of the data. Cleaning data includes removing duplicate values, processing missing values and outliers, and conversion and normalization are to unify the data to the same scale for subsequent model training and application.
[0075] It should be noted that the step of extracting features from the first valid data to obtain valid feature data can be to select and optimize the extracted features based on the intrinsic features and patterns of the data learned by the feature extraction model, so as to obtain valid feature data, and can also screen out invalid data that has been classified into the first valid data due to manual screening.
[0076] Among them, the feature extraction model can be a model based on the Convolutional Neural Network (CNN) algorithm, which extracts features from the input data by performing a dot product operation between the input data and the convolution kernel. Feature extraction is to extract different features from the input data, such as edges, textures, etc., by designing different convolution kernels.
[0077] Further, generating mixed data according to the valid feature data and the invalid data includes:
[0078] independently storing the invalid data;
[0079] Determining a target storage interval for the invalid data;
[0080] The test data is imported into the target storage interval based on a preset mixing strategy, the test data is the valid feature data, and the preset mixing strategy is used to mix the test data and the invalid data.
[0081] In a specific implementation, by independently storing invalid data, it can be ensured that subsequent data mixing will not interfere with the original data, that is, data processing and data classification can be performed without destroying the original data components.
[0082] When performing data mixing, the prepared valid feature data can be imported into the invalid data storage space, and then the test data and the marked invalid data a are mixed and shuffled. The valid feature data includes valid data b and invalid data b, wherein the valid data b refers to the valid data manually screened, and the invalid data b refers to the data that is mistakenly screened as the first valid data during manual screening.
[0083] The preset mixing strategy includes, but is not limited to, intermittent insertion, pseudo-random mixing, etc., and this embodiment does not impose any specific restrictions on this.
[0084] This embodiment obtains first valid data and invalid data by preprocessing the communication data to be processed, and obtains mixed data by extracting features from the first valid data and mixing it with the invalid data, so as to facilitate further data validity classification through the trained model and improve the effect of data processing.
[0085] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , step S40, comprising:
[0086] Step S401: Screen the second valid data to obtain fourth valid data.
[0087] Step S402: Generate target valid data from the first valid data, the third valid data and the fourth valid data, where the target valid data is the union of the first valid data, the third valid data and the fourth valid data.
[0088] It should be noted that the valid data d output by the qualified model contains the valid data b in the test data, and the data that needs to be eliminated is the valid data e that was mistakenly deleted in the invalid data a.
[0089] Furthermore, the screening of the second valid data includes:
[0090] Determine the actual valid data in the test data;
[0091] The second valid data is screened according to the actual valid data to obtain fourth valid data, where the fourth valid data is the second valid data remaining after the actual valid data is removed.
[0092] Specifically, the accuracy of the final data and the versatility of the model can be further improved through the valid data e. The actual valid data b in the test data is the real result. The model can be evaluated by comparing the predicted result generated by testing the model with the real result. Invalid data a is mixed in the test data. The invalid data a contains certain valid data e. Therefore, these valid data e will be screened out together with the actual valid data b after being screened by the model to obtain valid data d, that is, the valid data d is a mixture of the actual valid data b and the valid data e.
[0093] Valid data a is the valid data screened manually, valid data c is the valid data screened by the model after training, and valid data e is the valid data missed during manual screening. The above data are combined to form the final total valid data. A variety of methods such as linear regression and neural network are used to further analyze and interpret the prediction results. Make corresponding decisions or suggestions based on the analysis results.
[0094] This embodiment eliminates the influence of actual valid data in mixed data by screening the second valid data output by the model, reduces the false screening rate of invalid data screened out manually, and improves the effect and efficiency of data processing and data classification processing.
[0095] This application also provides a data processing device, please refer to Figure 4 , the data processing device comprises:
[0096] The acquisition module 10 is used to acquire first valid data, invalid data and mixed data in the communication data to be processed.
[0097] The identification module 20 is used to input the mixed data into a trained data classification model for data identification to obtain second valid data.
[0098] The identification module 30 is further configured to identify third valid data corresponding to the communication data to be processed by using the trained data classification model.
[0099] The generating module 40 is configured to generate target valid data according to the first valid data, the second valid data and the third valid data.
[0100] This embodiment performs a preliminary division of the communication data to be processed to obtain first valid data, invalid data and mixed data, and inputs the mixed data into a trained data classification model for data identification, thereby further screening the communication data to be processed and increasing the proportion of valid data. At the same time, the original communication data to be processed is identified through the trained data classification model to reduce data errors in the manual classification process. Finally, the final target valid data is comprehensively determined by identifying the first valid data, the second valid data and the third valid data, thereby maximizing the accuracy of the valid data and avoiding the technical problem of low accuracy of data preprocessing and waste of resources in the prior art.
[0101] In one embodiment, the acquisition module 10 is further used to preprocess the communication data to be processed to obtain first valid data and invalid data; perform feature extraction on the first valid data to obtain valid feature data; and generate mixed data based on the valid feature data and the invalid data.
[0102] In one embodiment, the acquisition module 10 is further used to classify the communication data to be processed according to at least one of data cleaning, data conversion and data normalization.
[0103] In one embodiment, the acquisition module 10 is also used to independently store the invalid data; determine a target storage interval for the invalid data; import test data into the target storage interval based on a preset mixing strategy, wherein the test data is the valid feature data, and the preset mixing strategy is used to mix the test data and the invalid data.
[0104] In one embodiment, the recognition module 20 is further used to perform data recognition training through an initial classification model based on effective feature data; and optimize model parameters of the initial classification model through a cross-validation strategy to obtain a trained data classification model.
[0105] In one embodiment, the generating module 40 is further used to filter the second valid data to obtain fourth valid data; generate target valid data from the first valid data, the third valid data and the fourth valid data, and the target valid data is the union of the first valid data, the third valid data and the fourth valid data.
[0106] In one embodiment, the generation module 40 is further used to determine actual valid data in the test data; the second valid data is screened according to the actual valid data to obtain fourth valid data, and the fourth valid data is the second valid data remaining after removing the actual valid data.
[0107] The present application provides a data processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data processing method in the above-mentioned embodiment 1.
[0108] The data processing device provided by the present application adopts the data processing method in the above embodiment to solve the technical problem of data processing. Compared with the prior art, the beneficial effects of the data processing device provided by the present application are the same as the beneficial effects of the data processing method provided by the above embodiment, and other technical features in the data processing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0109] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0110] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0111] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the data processing method in the above-mentioned embodiment.
[0112] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.
[0113] The computer-readable storage medium may be included in a data processing device, or may exist independently without being incorporated into a data processing device.
[0114] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a data processing device, the data processing device performs data processing.
[0115] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0116] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0117] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0118] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned data processing method, and can solve the technical problems of data processing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the data processing method provided in the above-mentioned embodiment, and will not be repeated here.
[0119] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned data processing method when executed by a processor.
[0120] The computer program product provided by this application can solve the technical problem of data processing. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the data processing method provided by the above embodiment, which will not be repeated here.
[0121] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.
Claims
1. A data processing method, characterized in that: The data processing method comprises: Acquire first valid data, invalid data, and mixed data in the communication data to be processed, wherein the mixed data is data obtained by mixing the first valid data and the invalid data; Inputting the mixed data into a trained data classification model for data recognition to obtain second valid data; Identify third valid data corresponding to the communication data to be processed by the trained data classification model; Target valid data is generated according to the first valid data, the second valid data and the third valid data.
2. The method according to claim 1, characterized in that The obtaining of first valid data, invalid data and mixed data in the communication data to be processed includes: Preprocessing the communication data to be processed to obtain first valid data and invalid data; Performing feature extraction on the first valid data to obtain valid feature data; Mixed data is generated according to the valid feature data and the invalid data.
3. The method according to claim 2, characterized in that The preprocessing of the communication data to be processed comprises: The communication data to be processed is classified according to at least one of data cleaning, data conversion and data normalization.
4. The method according to claim 2, characterized in that The generating mixed data according to the valid feature data and the invalid data comprises: independently storing the invalid data; Determining a target storage interval for the invalid data; The test data is imported into the target storage interval based on a preset mixing strategy, the test data is the valid feature data, and the preset mixing strategy is used to mix the test data and the invalid data.
5. The method according to claim 1, characterized in that Before inputting the mixed data into the trained data classification model for data recognition to obtain the second valid data, the method further includes: Perform data recognition training through the initial classification model based on effective feature data; The model parameters of the initial classification model are optimized through a cross-validation strategy to obtain a trained data classification model.
6. The method according to claim 1, characterized in that The generating target valid data according to the first valid data, the second valid data and the third valid data comprises: Screening the second valid data to obtain fourth valid data; The first valid data, the third valid data and the fourth valid data are used to generate target valid data, where the target valid data is a union of the first valid data, the third valid data and the fourth valid data.
7. The method according to claim 6, characterized in that The screening of the second valid data includes: Determine the actual valid data in the test data; The second valid data is screened according to the actual valid data to obtain fourth valid data, where the fourth valid data is the valid data remaining after the actual valid data is removed.
8. A data processing device, characterized in that: The data processing device comprises: An acquisition module, used for acquiring first valid data, invalid data and mixed data in the communication data to be processed; A recognition module, used for inputting the mixed data into a trained data classification model for data recognition to obtain second valid data; The identification module is further used to identify third valid data corresponding to the communication data to be processed through the trained data classification model; A generating module is used to generate target valid data according to the first valid data, the second valid data and the third valid data.
9. A data processing device, characterized in that: The data processing device comprises: a memory, a processor, and a data processing program stored in the memory and executable on the processor, wherein the data processing program is configured to implement the steps of the data processing method according to any one of claims 1 to 7.
10. A storage medium, characterized in that: The storage medium stores a data processing program, and when the data processing program is executed by the processor, the steps of the data processing method according to any one of claims 1 to 7 are implemented.