Data processing method, device, equipment and storage medium

Abstract

The embodiments of the present application provide a data processing method, device, equipment and storage medium to improve the efficiency of data collection. The method includes: collecting data streams; analyzing the data streams by at least one classifier in a classifier group to determine corresponding classification results; creating corresponding classification data sets based on the classification results to determine training data based on the classification data sets. It can provide various classified data sets with rich types and great data value, which can not only improve processing efficiency but also improve the value of data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and device, an electronic device and a storage medium. Background Art

[0002] In the field of artificial intelligence, a large amount of rich and effective data is needed as the basis for processing.

[0003] Currently, data collection mainly relies on manual methods and is based on subjective judgment. That is, people judge whether the required target appears in the data stream and whether the target should be cut out for collection, labeling, and other processing.

[0004] However, manual collection methods are inefficient, time-consuming and laborious. For example, to find mice in kitchen surveillance videos, one may need to manually watch thousands of hours of video footage to find a few scenes where mice appear, which is very inefficient. Summary of the invention

[0005] The embodiment of the present application provides a data processing method to improve the efficiency of data collection.

[0006] Correspondingly, an embodiment of the present application also provides a data processing device, an electronic device and a storage medium to ensure the implementation and application of the above method.

[0007] In order to solve the above problems, an embodiment of the present application discloses a data processing method, which includes: collecting data streams; analyzing the data streams through at least one classifier in a classifier group to determine corresponding classification results; creating a corresponding classification data set based on the classification results to determine training data based on the classification data set.

[0008] An embodiment of the present application also provides a data processing device, including: an acquisition module, used to acquire data streams; a classification module, used to analyze the data streams through at least one classifier in a classifier group to determine corresponding classification results; a data set determination module, used to create a corresponding classification data set based on the classification results, so as to determine training data based on the classification data set.

[0009] The embodiment of the present application also provides an electronic device, including: a processor; and

[0010] A memory having executable code stored thereon, which, when executed, enables the processor to execute one or more methods described in the embodiments of the present application.

[0011] The embodiments of the present application also provide one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes one or more methods described in the embodiments of the present application.

[0012] Compared with the prior art, the embodiments of the present application have the following advantages:

[0013] In an embodiment of the present application, after collecting the data stream, the data stream can be analyzed by at least one classifier in the classifier group. Through a series of classifier analyses, the classification result corresponding to the data stream can be determined with high efficiency; then a corresponding classification data set can be created based on the classification result, thereby providing various classified data sets with rich types and greater data value, which can not only improve processing efficiency but also improve the value of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a schematic diagram of a data processing example of an embodiment of the present application;

[0015] Figure 2 It is a schematic diagram of an example of data detection and collection in an embodiment of the present application;

[0016] Figure 3 is a flowchart of steps of an embodiment of a data processing method of the present application;

[0017] Figure 4 is a structural block diagram of an embodiment of a data processing device of the present application;

[0018] Figure 5 is a structural block diagram of another data processing device embodiment of the present application;

[0019] Figure 6 It is a schematic diagram of the structure of a device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0021] The embodiments of the present application can be applied to the field of data processing, in various fields based on data processing, such as machine learning, deep learning, intelligent vision, etc. There are many artificial intelligence (AI) requirements and algorithms related to detection and recognition in these fields, such as vehicle detection, face detection, vehicle tracking and other scenes in image detection. The algorithm models required for these scenes need to be trained with a large amount of data, and the effectiveness of the training data affects the quality of the algorithm model, so it is necessary to be able to collect the effective data required for the scene and mark it. In addition, the embodiments of the present application can be applied to various AI data-related scenes, such as face detection and recognition based on computer vision, applied to pedestrian and vehicle recognition in the traffic field, such as pedestrian re-identification (Person Re-identification, ReID) processing, and can also be applied to video surveillance, video AI tags, content recognition and other video-based processing, as well as voice collection and understanding, natural language processing, etc. in the field of audio processing. The data of the embodiments of the present application can be various multimedia data such as audio, video, and images, which can be determined according to the applied scene.

[0022] The embodiment of the present application can analyze the collected data stream through the classifier in the classifier group, determine the classification result and mark it, and then determine the classification data set corresponding to the classification label according to the required scene as the training data of the scene, so as to collect the required valid data and mark it, thereby improving the data processing efficiency and data validity. The type of data stream can be determined according to the demand, application scenario, etc., for example, the data stream is a video stream, etc.

[0023] Reference Figure 1 , showing a data processing schematic diagram of an embodiment of the present application.

[0024] Step 102, collecting data stream.

[0025] Various required data can be collected, such as through designated equipment or through the network. Taking the field of image analysis as an example, video streams can be obtained as data streams through various cameras and other image acquisition devices, such as monitoring video streams, live streams, on-demand streams, audio streams, and crawling images, video streams, voice data streams, etc. on the network, so that data streams can be obtained in various ways. In this way, a large number of data streams can be obtained so that preprocessing can be performed and the required samples can be obtained as training data, so as to provide various required sample data for training and improve the accuracy of the model. For example, the data streams are monitoring video streams of various communities and roads, and voice streams of various music and voice processing applications.

[0026] Step 104: Analyze the data stream by using at least one classifier in the classifier group to determine a corresponding classification result.

[0027] The embodiment of the present application pre-processes the collected data stream so as to screen various sample data required as training data for training the corresponding model. For the pre-processing of the data stream, it can be analyzed by a classifier group. Among them, the classifier group includes at least one classifier, which is used to classify the data stream. The classifier can be obtained by training various required mathematical model algorithms, and can also be called a classification model, an algorithm model, a data mapping set, etc. A mathematical model is a scientific or engineering model constructed using mathematical logic methods and mathematical language. A mathematical model is a mathematical structure that is generally or approximately expressed using mathematical language with reference to the characteristics or quantitative dependencies of a certain system of things. This mathematical structure is a pure relational structure of a certain system portrayed by means of mathematical symbols. A mathematical model can be one or a group of algebraic equations, differential equations, difference equations, integral equations or statistical equations and combinations thereof, which quantitatively or qualitatively describe the mutual relationship or causal relationship between the variables of the system through these equations. In addition to mathematical models described by equations, there are also models described by other mathematical tools, such as algebra, geometry, topology, mathematical logic, etc. A mathematical model describes the behavior and characteristics of a system rather than the actual structure of the system.

[0028] The data stream is input into the classifier group and analyzed by each classifier to obtain the corresponding output result, determine the classification result of the data stream, and then mark the classification label of the data stream according to the classification result. The classification label is used to indicate one or more categories to which the data stream belongs. The classification standard can be determined according to the needs, such as the needs of model training.

[0029] The classifiers in the classifier group of the embodiment of the present application are associated with each other, wherein the next classifier can be determined based on the output result of the previous classifier. After the data stream is input into the classifier in the classifier group for analysis, the next classifier can be determined based on the output result, thereby analyzing the data stream through a series of classifiers to obtain the corresponding classification result. In an optional embodiment, the data stream is analyzed by at least one classifier in the classifier group, and the corresponding classification result is determined and marked, including: inputting the data stream into the classifier group, analyzing it through at least one classifier, and determining the classification result corresponding to the data stream, wherein the next classifier in the classifier group is determined based on the output result of the previous classifier.

[0030] The data stream can be input into the classifier group, and at least one classifier in the classifier group can analyze the data and output the corresponding output result. According to the output result, it can be determined to continue to input the next classifier, or the classification result can be obtained. After the classification result is determined, the data stream can be labeled according to the classification result, and the corresponding classification label can be set for the data stream. A data stream can belong to one or more classifications. Among them, the classifiers in the classifier group can be set according to certain requirements and rules. For example, the classifiers are divided according to a certain level. The classifiers of the first layer are used to divide the sample categories of the data stream, such as determining whether the data stream is valid, the scene and target of the data stream, etc. The classifiers of the second layer are used to analyze the sample status of the data stream, such as the quality of the data stream, the status of the target, etc. Among them, each layer can include at least one classifier, and the output result of a classifier in each layer of the classifiers can be used to determine the next classifier until the required output result is obtained or the analysis of all classifiers is completed. Then the output result of the last classifier of the first layer or the overall output result of the first layer classifier can determine the first classifier of the second layer, and then the classifier of the second layer is analyzed to obtain the corresponding output result and determine the classification result of the data stream.

[0031] Therefore, in an optional embodiment, the data stream is analyzed by at least one classifier in the classifier group to determine the corresponding classification result, including: inputting the data stream into the first layer classifier of the classifier group for analysis, determining the second layer classifier according to the output result of the first layer classifier, and using the second layer classifier for analysis to obtain the corresponding classification result.

[0032] In some other optional embodiments, the classifiers in the classifier group can also be divided according to the purpose, etc., to divide the classifiers into different categories or levels. For example, the first classifier, the second classifier and the third classifier are divided according to the validity, scene and state of the sample; the first classifier is used to analyze the validity of the data stream, and the validity can be determined according to various needs, such as whether it is a valid target, etc. The valid target can be a generalized target, that is, to check whether there may be a target in the image and other data, such as a pure color image, which may not contain a target; the second classifier is used to analyze the scene target of the data stream, and the scene to which the data stream belongs can be analyzed by the second classifier, and then the target in the scene can be determined, such as vehicle detection and vehicle tracking scenes, with vehicles as the target, face detection scenes with faces as the target, and keywords as the target for voice recognition scenes, as well as targets in various AI processing scenes; the third classifier is used to analyze the sample status corresponding to the data stream, such as the number of targets, blur or not, whether there is interference and other states.

[0033] The data stream is input into a classifier group, analyzed by at least one classifier, and a classification result corresponding to the data stream is determined, including: inputting the data stream into a first classifier of the classifier group, the first classifier being used to analyze the validity of the data stream; when the output result of the first classifier is valid, inputting the data stream into a second classifier, the second classifier being used to analyze the scene target of the data stream; based on the scene target information output by the second classifier, inputting the data stream into at least one corresponding third classifier, the third classifier being used to analyze the sample state corresponding to the data stream; and determining the classification result corresponding to the data stream based on the output result of the at least one third classifier.

[0034] In the embodiment of the present application, the first classifier can be a classifier obtained by training the model with a small-scale sample. The classifier can be a rough version of the classifier, and the model used is not limited to any framework or any language. The data stream is input into the first classifier trained by the model for analysis. The first classifier can roughly divide the data stream into two categories, that is, let the classifier automatically determine whether the data stream contains a valid target. If it contains, it is valid data and can output a valid result. If it does not contain, it is invalid data and can output an invalid result. Taking the video stream as an example of the data stream, the first classifier can automatically determine which video streams belong to valid samples that need to be collected (samples that may need targets) and which belong to invalid samples (samples without targets). When training and using the first classifier, the present application can reduce the discrimination threshold to an extremely low level, thereby obtaining a very high recall rate, and the precision rate does not need to be concerned, so that the first classifier can roughly divide valid and invalid samples. For example, in the recognition of images and videos, it is determined whether there may be a target, such as pure black, pure white and other pure color images or noise images, etc., where there is no target.

[0035] After the data stream is input into the first classifier of the classifier group for analysis, a corresponding output result can be obtained. If the classification result of the first classifier is invalid, the data stream is discarded. After obtaining a rough classification, for invalid samples, these video streams and other data streams are directly discarded and filtered out. This is invalid data and no further processing is required. Of course, in some examples, it can also be marked as a classification label such as invalid data and then discarded. It is determined according to the specific needs, and the embodiment of the present application does not limit this.

[0036] When the output result of the first classifier is valid, the data stream is input into the second classifier, and the scene to which the data stream belongs can be analyzed by the second classifier, and the target to be identified can be determined based on the scene, thereby obtaining the corresponding scene target information as the output result. For example, if a highway, road or other road scene is detected, the scene can be determined to be a road scene, and the target can be a vehicle, a pedestrian or the like, and the corresponding scene target information can be obtained. For another example, if a crowd scene such as an elevator or a sidewalk is identified, a face can be determined to be the target, and the corresponding scene information can be obtained. Thus, the corresponding third classifier can be determined based on the scene target information.

[0037] Wherein, the step of inputting the data stream into at least one corresponding third classifier according to the scene target information output by the second classifier includes at least one of the following: selecting at least one third classifier corresponding to the scene target according to the scene target information output by the second classifier; inputting the data stream into at least one corresponding third classifier to detect the target and obtain a corresponding output result. In an embodiment of the present application, there may be multiple third classifiers, so that the data stream can be analyzed by at least one third classifier to obtain a corresponding output result. For example, a group of third classifiers corresponding to each scene can be divided according to the scene, and the third classifiers between different groups may be the same or different, which is specifically determined according to the scene requirements.

[0038] The data stream is analyzed by at least one third classifier, and the sample state of the data stream can be determined based on the output result of at least one third classifier, wherein the sample state refers to the state of the data stream corresponding to the sample, such as clarity, number of targets, interference, etc., so the sample state includes at least one of the following: simple sample state and complex sample state. Simple sample state refers to the situation where the state of the data stream and the targets therein is relatively simple, such as the target is clear, unobstructed, and uninterfered; complex sample state refers to the situation where the state of the data stream and the targets therein is relatively complex, such as a large number of targets, overlap, interference, etc. Among them, since detection can be performed through multiple third classifiers, the sample state, such as the number of targets, overlap, etc., can be determined based on the output result of the previous third classifier, so that the next third classifier can be selected based on the sample state.

[0039] like Figure 2 In the example shown, the data stream is analyzed by the classifier group to determine the corresponding output result, and the output result can be combined with a certain algorithm to obtain the corresponding classification result, as follows:

[0040] For simple sample states, at least one of the following is included: clear state, non-occluded interference state. The target can be detected by the classifier group, and the data stream is determined to be a high-confidence and correct sample. Such samples generally contain the following features: 1) Clear state: The target appears very clearly in the data (such as pictures, audio and video); 2) Non-occluded interference state: There is no overlap or occlusion between targets, such as by calculating the bounding box and combining the target detection algorithm (Intersection Over Union, IOU) to find that there is no overlap or occlusion in the data stream. In some applications, since the model is sufficiently trained on such data, there is no need to accumulate too many data samples in the simple sample state, and an appropriate amount is sufficient.

[0041] The complex sample state includes at least one of the following: crowded state, occlusion interference state, blur state, environmental impact state, false detection and missed detection state. The target can be detected by the classifier group, but the state of the target and data is relatively complex, which may include the following situations:

[0042] 1) Crowded state:

[0043] A large number of targets to be detected can be detected in one or more frames of data in the data stream. For example, in a vehicle detection task, a large number of vehicles can be detected in peak traffic conditions in the morning and evening. If the third classifier detects multiple targets in one frame of data in multiple frames of data in the data stream, it can be determined as a congested state.

[0044] Data streams containing congestion or multiple frames of data in a congested state can be marked as congestion tags. For such data, the number of detected targets can be used to locate the data, and the data in the data stream such as the screens where more targets appear can be captured based on the number of detection results. For example, the number of targets in the output result is determined, and if the number of targets exceeds a certain threshold, it is determined to be a congested state.

[0045] 2) Occlusion interference state:

[0046] One or more frames of data in the data stream can detect an occlusion interference state. In this state, the detected targets are blocked or overlapped. For example, in a vehicle detection task, two vehicles parked side by side are detected to overlap in the surveillance image taken from the camera angle, which can be determined as an occlusion interference state.

[0047] For the data stream in which the occlusion interference state is detected or one or more frames of data corresponding to the occlusion interference state, it can be marked as an occlusion interference label. For this type of data, it can be located according to the detected target, wherein, according to the frame corresponding to the target, it can be determined according to the position of the frame corresponding to the target. For example, if more frames appear and the distance is close, and the IOU between the targets is determined to be large in combination with the target detection algorithm, it can be determined as an occlusion interference state. For example, through the output results of the targets and the frames where the targets are located, if the number of targets exceeds a certain threshold and there is overlap between the frames, it is determined to be an occlusion interference state. Further, the overlap threshold between the frames, the IOU threshold between the targets, etc. can be set to determine the occlusion interference state.

[0048] 3) Fuzzy state:

[0049] One or more frames of data in the data stream can detect a blur state, in which the detected target appears blurry. The output result includes the target, the confidence of the output result is low, and the blur estimation of the data in the data stream can be performed, and a blur state can be determined when a certain threshold is reached.

[0050] 4) Environmental impact status:

[0051] Data streams such as surveillance video streams and meteorological data streams are often affected by the environment, such as the four seasons, weather, and different time periods every day such as morning, noon, and evening. For example, the number of vehicles on the road is related to the morning and evening rush hours and other time periods. Different weather conditions such as rain and snow will also affect data streams such as surveillance videos. Therefore, data collection in different environments can be achieved by combining the local time, season, and weather in various places. Environmental scenes can be detected for data streams in different environments to obtain data streams or data with corresponding environmental impact status.

[0052] 5) False detection and missed detection status

[0053] In addition to the complex sample states mentioned above, there may still be false detections, missed detections, etc. in the data stream. This part of the data also needs to be collected to improve the accuracy of the corresponding model training, reduce the false detection and missed detection rate of the model, and improve accuracy.

[0054] In some optional embodiments, it is determined whether the output results corresponding to the consecutive frames in the data stream match; if the output results corresponding to the consecutive frames do not match, the data stream is rechecked. The output results in the analyzer group corresponding to each frame of data in the data stream can be combined to determine whether the output results corresponding to the consecutive frames match. For example, a target is detected in a frame, but the target does not exist in the previous and next frames; for another example, the target is detected in the previous and next frames but does not exist in the current frame. This detection jump can be determined as an output result mismatch. In the case where the output results corresponding to the consecutive frames do not match, it indicates that a false detection or missed detection may have occurred, which is determined to be a false detection or missed detection state, and the data stream can be rechecked. Collecting data streams with false detection or missed detection states as training data for model training helps to improve the robustness of the model.

[0055] In the embodiment of the present application, the classification label of the data stream can also be marked according to the classification result. In the data stream, a complex sample state can be detected in one or more frames of data. For the data stream with complex sample states, the entire data stream can be marked, so that the entire data stream can be collected. It is also possible to mark the frame data and its associated frame data based on each frame data where the complex sample state appears, such as grabbing the frame data and the previous and next N frames of data, or grabbing the first N frames of the first frame of the continuous frame data and the last N frames of the last frame, so as to collect the marked data stream. Therefore, in some optional embodiments, the sample state corresponding to the data stream can be determined according to the classification result; determine the data frame corresponding to the sample state, grab the data frame and the associated data frame, and determine the corresponding segmented data stream; mark the segmented data stream according to the sample state to obtain the corresponding classification label, wherein the associated data frame includes the first N frames of the data frame and / or the last N frames of the data frame, and N is a positive integer. Different segmented data streams in a data stream may correspond to different classification labels.

[0056] In some optional embodiments of the present application, after the data stream is collected by detecting and labeling the data stream in the above manner, further review can be performed manually. The corresponding classification results can be determined by detecting the data stream through the classifier, so the manual review usually only needs to be fine-tuned. The embodiment of the present application detects the data stream through the above classifier group, which can greatly improve the work efficiency of data labeling personnel and reduce their huge workload.

[0057] Step 106: Create a corresponding classification data set according to the classification result, so as to determine training data according to the classification data set.

[0058] Based on the classification results corresponding to the data stream or a segmented data stream therein, data collection can be performed, such as collecting data streams or segmented data streams with the classification label marked by the classification result, and creating corresponding classification data sets by collecting data streams or segmented data streams. Each classification data set may include multiple data streams or multiple segmented data streams.

[0059] In some other embodiments, after completing the collection and accumulation of data, the above-mentioned classification data sets can also further optimize the classifiers in the classifier group, and then the optimized classifier group continues to monitor and collect data, forming a virtuous circle, constantly improving the quality of data collection and accumulation and the accuracy of the classifier group, and better meeting application requirements.

[0060] After obtaining the classified data set through the above preprocessing, the data stream and segmented data stream in the corresponding classified data set can be provided as training data model training according to the needs of each platform and server training model. In this way, different classifications correspond to different sample states, which can provide richer and more targeted training data for model training and improve the quality of model training.

[0061] When the data is collected to meet the required requirements, the data of various tasks in different scenarios and detection models such as better classifier groups can be accumulated. These collected and accumulated data have extremely high application value, which can help the corresponding service functions to more scientifically evaluate the algorithm indicators of other institutions, and can also serve as a reserve for data mining and accumulate a rich data warehouse.

[0062] In addition, the classifiers in the above classifier group provide accuracy by continuously detecting data and optimizing. After reaching certain standards, they can also be used as models for corresponding service applications and applied to specific scenarios.

[0063] After collecting the data stream, the data stream can be analyzed by at least one classifier in the classifier group. Through a series of classifier analyses, the classification results corresponding to the data stream can be determined and labeled, which is very efficient. Then, the corresponding classification data set can be created based on the labeled classification labels, so that the data stream can be collected according to the classification obtained by analysis. Various classified data sets with rich types and greater data value can be obtained. Therefore, the above method can not only improve processing efficiency but also improve the value of data.

[0064] On the basis of the above embodiments, the embodiments of the present application also provide a data processing method, which can quickly, massively and efficiently collect the data required in the scene, and use intelligent automatic collection to replace the existing work that requires human intervention. Moreover, since the data collected is always in the real scene, there is no problem of false data in the artificial data solution.

[0065] Reference Figure 3 , showing a step flow chart of an embodiment of a data processing method of the present application.

[0066] Step 302, collecting data stream.

[0067] Then, the data stream can be analyzed by at least one classifier in the classifier group to determine the corresponding classification result and mark it. The data stream is input into the classifier group, analyzed by at least one classifier, and the classification result corresponding to the data stream is determined, wherein the next classifier in the classifier group is determined based on the output result of the previous classifier, which can be specifically implemented by the following steps:

[0068] Step 304: input the data stream into the first classifier of the classifier group.

[0069] The first classifier is used to analyze the validity of the data stream. If the output result is valid, execute step 308; if the output result is invalid, execute step 306.

[0070] Step 306: discard the data flow.

[0071] Step 308: input the data stream into a second classifier to obtain a corresponding output result.

[0072] The second classifier is used to analyze the scene target of the data stream, and the output result includes scene target information, which includes: scene information of the data stream, target information in the scene, etc.

[0073] Step 310: input the data stream into at least one corresponding third classifier according to the scene target information output by the second classifier.

[0074] The third classifier is used to analyze the sample state corresponding to the data stream; the inputting the data stream into the corresponding at least one third classifier according to the scene target information output by the second classifier includes: selecting at least one third classifier corresponding to the scene target according to the scene target information output by the second classifier; inputting the data stream into the selected at least one third classifier to detect the target and obtain the corresponding output result. According to the sample state in the output result of the third classifier, the next third classifier is selected.

[0075] Step 312: Determine a classification result corresponding to the data stream according to the output result of the at least one third classifier.

[0076] The sample state includes at least one of the following: a simple sample state and a complex sample state. The simple sample state includes at least one of the following: a clear state and a non-blocking interference state; the complex sample state includes at least one of the following: a crowded state, a blocked interference state, a fuzzy state, an environmental impact state, and a false detection or missed detection state.

[0077] Step 314: determine whether the output results corresponding to the consecutive frames in the data stream match.

[0078] If they match, execute step 318; if they do not match, execute step 316.

[0079] Step 316: recheck the data flow.

[0080] Step 318: label the data stream with a classification label according to the classification result.

[0081] Among them, marking the classification label of the data stream based on the classification result includes: determining the sample state corresponding to the data stream based on the classification result; capturing the data frame corresponding to the sample state and its associated data frame to determine the corresponding segmented data stream, wherein the associated data frame includes the first N frames and / or the last N frames of the data frame; marking the classification label of the segmented data stream based on the sample state.

[0082] Step 320: Create a corresponding classification data set according to the classification label to determine training data according to the classification data set.

[0083] The corresponding steps in the embodiments of the present application are similar to those in the above embodiments, and please refer to the description of the above embodiments for details.

[0084] The embodiments of the present application can intelligently and automatically collect the required data and realize data reflux, thereby further refining the process of classifiers and models and realizing a closed loop on the data and model side, greatly saving labor costs, improving algorithm development efficiency, enhancing algorithm effects, and mining and accumulating data wealth.

[0085] Moreover, compared with the problem that the trained data is difficult to accurately detect real scenes by synthesizing data artificially through transformation, this application always collects data from real scenes, and there is no problem of false data in artificial data solutions, so the trained model can be applied to the detection of real scenes. Provide various data required for AI data processing scenarios, such as providing a data set of complex sample data whose data quality is affected by weather such as rain and snow in traffic recognition scenarios, and providing a data set of complex sample data with low speech clarity due to background sound, environmental noise, etc. in the field of natural language recognition.

[0086] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0087] On the basis of the above embodiments, this embodiment further provides a data processing device, which is applied to electronic devices such as terminal devices and servers.

[0088] Reference Figure 4 , shows a structural block diagram of a data processing device embodiment of the present application, which may specifically include the following modules:

[0089] The acquisition module 402 is used to acquire data streams.

[0090] The classification module 404 is used to analyze the data flow through at least one classifier in the classifier group to determine a corresponding classification result.

[0091] The data set determination module 406 is used to create a corresponding classification data set according to the classification result, so as to determine training data according to the classification data set.

[0092] In summary, after collecting the data stream, the data stream can be analyzed by at least one classifier in the classifier group. Through a series of classifier analyses, the classification results corresponding to the data stream can be determined and labeled, which is very efficient. Then, the corresponding classification data set can be created based on the labeled classification labels, so that the data stream can be collected according to the classification obtained by the analysis. Various classified data sets with rich types and greater data value can be obtained. Therefore, the above method can not only improve the processing efficiency, but also improve the value of the data.

[0093] Reference Figure 5 , shows a structural block diagram of a data processing device embodiment of the present application, which may specifically include the following modules:

[0094] The acquisition module 402 is used to acquire data streams.

[0095] The classification module 404 is used to analyze the data flow through at least one classifier in the classifier group to determine a corresponding classification result.

[0096] The labeling module 408 is used to label the classification label of the data flow according to the classification result.

[0097] The data set determination module 406 is used to create a corresponding classification data set according to the classification result, so as to determine training data according to the classification data set.

[0098] The classification module 404 is used to input the data stream into the classifier group, analyze it through at least one classifier, and determine the classification result corresponding to the data stream, wherein the next classifier in the classifier group is determined based on the output result of the previous classifier.

[0099] The classification module 404 is used to input the data stream into the first classifier of the classifier group, and the first classifier is used to analyze the validity of the data stream; when the output result of the first classifier is valid, the data stream is input into the second classifier, and the second classifier is used to analyze the scene target of the data stream; based on the scene target information output by the second classifier, the data stream is input into at least one corresponding third classifier, and the third classifier is used to analyze the sample state corresponding to the data stream; based on the output result of the at least one third classifier, the classification result corresponding to the data stream is determined.

[0100] The classification module 404 is further configured to discard the data flow if the classification result of the first classifier is invalid.

[0101] The classification module 404 is used to select at least one third classifier corresponding to the scene target according to the scene target information output by the second classifier; input the data stream into the selected at least one third classifier to detect the target and obtain a corresponding output result. The next third classifier is selected according to the sample state in the output result of the third classifier.

[0102] The sample state includes at least one of the following: a simple sample state and a complex sample state.

[0103] The classification module 404 is further configured to determine whether the output results corresponding to the consecutive frames in the data stream match; if the output results corresponding to the consecutive frames do not match, recheck the data stream.

[0104] The labeling module 408 is used to determine the sample state corresponding to the data stream based on the classification result; capture the data frame corresponding to the sample state and its associated data frame to determine the corresponding segmented data stream, wherein the associated data frame includes the first N frames and / or the last N frames of the data frame; and label the classification label of the segmented data stream based on the sample state.

[0105] After obtaining the classified data set through the above preprocessing, the data stream and segmented data stream in the corresponding classified data set can be provided as training data model training according to the needs of each platform and server training model. In this way, different classifications correspond to different sample states, which can provide richer and more targeted training data for model training and improve the quality of model training.

[0106] When the data is collected to meet the required requirements, the data of various tasks in different scenarios and detection models such as better classifier groups can be accumulated. These collected and accumulated data have extremely high application value, which can help the corresponding service functions to more scientifically evaluate the algorithm indicators of other institutions, and can also serve as a reserve for data mining and accumulate a rich data warehouse.

[0107] In addition, the classifiers in the above classifier group provide accuracy by continuously detecting data and optimizing. After reaching certain standards, they can also be used as models for corresponding service applications and applied to specific scenarios.

[0108] After collecting the data stream, the data stream can be analyzed by at least one classifier in the classifier group. Through a series of classifier analyses, the classification results corresponding to the data stream can be determined and labeled, which is very efficient. Then, the corresponding classification data set can be created based on the labeled classification labels, so that the data stream can be collected according to the classification obtained by analysis. Various classified data sets with rich types and greater data value can be obtained. Therefore, the above method can not only improve processing efficiency but also improve the value of data.

[0109] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0110] The present application embodiment provides one or more machine-readable media on which instructions are stored, and when executed by one or more processors, an electronic device executes one or more of the methods described in the above embodiments. In the present application embodiment, the electronic device includes various types of devices such as terminal devices and servers (clusters).

[0111] The embodiments of the present disclosure may be implemented as a device configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the device may include electronic devices such as terminal devices and servers (clusters). Figure 6 An exemplary apparatus 600 that may be used to implement various embodiments described in this application is schematically illustrated.

[0112] For one embodiment, Figure 6An exemplary apparatus 600 is shown having one or more processors 602, a control module (chip set) 604 coupled to at least one of the processor(s) 602, a memory 606 coupled to the control module 604, a non-volatile memory (NVM) / storage device 608 coupled to the control module 604, one or more input / output devices 610 coupled to the control module 604, and a network interface 612 coupled to the control module 604.

[0113] The processor 602 may include one or more single-core or multi-core processors, and the processor 602 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 600 can be used as a terminal device, server (cluster), etc. described in the embodiments of the present application.

[0114] In some embodiments, the apparatus 600 may include one or more computer-readable media (e.g., memory 606 or NVM / storage device 608) having instructions 614 and one or more processors 602 combined with the one or more computer-readable media and configured to execute the instructions 614 to implement a module to perform the actions described in the present disclosure.

[0115] For one embodiment, control module 604 may include any suitable interface controller to provide any suitable interface to at least one of processor(s) 602 and / or any suitable device or component in communication with control module 604 .

[0116] The control module 604 may include a memory controller module to provide an interface to the memory 606. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0117] The memory 606 may be used, for example, to load and store data and / or instructions 614 for the device 600. For one embodiment, the memory 606 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 606 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0118] For one embodiment, control module 604 may include one or more input / output controllers to provide an interface to NVM / storage device 608 and input / output device(s) 610 .

[0119] For example, NVM / storage 608 may be used to store data and / or instructions 614. NVM / storage 608 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0120] NVM / storage device 608 may include storage resources that are physically part of the device on which apparatus 600 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 608 may be accessed via input / output device(s) 610 over a network.

[0121] (One or more) input / output devices 610 may provide an interface for the apparatus 600 to communicate with any other appropriate device, and the input / output device 610 may include a communication component, an audio component, a sensor component, etc. The network interface 612 may provide an interface for the apparatus 600 to communicate through one or more networks, and the apparatus 600 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0122] For one embodiment, at least one of the processor(s) 602 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 604. For one embodiment, at least one of the processor(s) 602 may be packaged together with the logic of one or more controllers of the control module 604 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 602 may be integrated on the same die with the logic of one or more controllers of the control module 604. For one embodiment, at least one of the processor(s) 602 may be integrated on the same die with the logic of one or more controllers of the control module 604 to form a system-on-chip (SoC).

[0123] In various embodiments, the device 600 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the device 600 may have more or fewer components and / or different architectures. For example, in some embodiments, the device 600 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0124] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in a memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0125] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0126] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0127] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0128] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0130] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0131] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0132] The data processing method and device, an electronic device and a storage medium provided by the present application are introduced in detail above. The principles and implementation methods of the present application are explained in this article using specific examples. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Collect data streams; Inputting the data stream into a first classifier of a classifier group, the first classifier being used to analyze the validity of the data stream, wherein a next classifier in the classifier group is determined based on an output result of a previous classifier; When the output result of the first classifier is valid, the data stream is input into a second classifier, and the second classifier is used to analyze the scene target of the data stream; According to the scene target information output by the second classifier, the data stream is input into at least one corresponding third classifier, and the third classifier is used to analyze the sample state corresponding to the data stream, and the sample state includes at least one of the following: a simple sample state and a complex sample state, wherein the simple sample state refers to the situation where the state of the data stream and the target therein is simple, including the target being clear, unobstructed, and uninterfered, and the complex sample state refers to the situation where the state of the data stream and the target therein is complex, including a large number of targets, overlap, and interference; Determining a classification result corresponding to the data stream according to an output result of the at least one third classifier; A corresponding classification data set is created according to the classification result, so as to determine training data according to the classification data set.

2. The method according to claim 1, characterized in that Also includes: According to the classification result, a classification label of the data flow is marked.

3. The method according to claim 1, characterized in that Also includes: If the classification result of the first classifier is invalid, the data flow is discarded.

4. The method according to claim 1, characterized in that: The step of inputting the data stream into at least one corresponding third classifier according to the scene target information output by the second classifier comprises: Selecting at least one third classifier corresponding to the scene target according to the scene target information output by the second classifier; The data stream is input into at least one selected third classifier to detect the target and obtain a corresponding output result.

5. The method according to claim 4, characterized in that The next third classifier is selected according to the sample status in the output result of the third classifier.

6. The method according to claim 4 or 5, characterized in that: The sample state includes at least one of the following: a simple sample state and a complex sample state.

7. The method according to any one of claims 1 to 5, characterized in that: Also includes: Determining whether output results corresponding to consecutive frames in the data stream match; In the case where the output results corresponding to the consecutive frames do not match, the data stream is rechecked.

8. The method according to claim 2, characterized in that: The step of marking the classification label of the data flow according to the classification result includes: Determine the sample state corresponding to the data stream according to the classification result; Capture the data frame corresponding to the sample state and its associated data frame to determine the corresponding segmented data stream, wherein the associated data frame includes the first N data frames and / or the last N data frames of the data frame; According to the sample status, the segmented data stream is labeled with a corresponding classification label.

9. A data processing device, characterized in that: include: A collection module, used for collecting data streams; A classification module, used for inputting the data stream into a first classifier of a classifier group, the first classifier being used to analyze the validity of the data stream, wherein the next classifier in the classifier group is determined based on the output result of the previous classifier; when the output result of the first classifier is valid, the data stream is input into a second classifier, the second classifier being used to analyze the scene target of the data stream; according to the scene target information output by the second classifier, the data stream is input into at least one corresponding third classifier, the third classifier being used to analyze the sample state corresponding to the data stream; according to the output result of the at least one third classifier, the classification result corresponding to the data stream is determined, the sample state including at least one of the following: a simple sample state and a complex sample state, the simple sample state refers to a situation where the state of the data stream and the target therein is simple, including a clear target, no obstruction, and no interference, the complex sample state refers to a situation where the state of the data stream and the target therein is complex, including a large number of targets, overlap, and interference; The data set determination module is used to create a corresponding classification data set according to the classification result, so as to determine training data according to the classification data set.

10. An electronic device, characterized in that: include: processor; and A memory having executable codes stored thereon, which, when executed, cause the processor to perform the method according to one or more of claims 1-8.

11. One or more machine-readable media having executable codes stored thereon, which, when executed, cause a processor to perform the method according to one or more of claims 1-8.

Images