Data quality detection method, intelligent equipment and storage medium

Abstract

The invention relates to the technical field of data processing, particularly provides a data quality detection method, intelligent equipment and a storage medium, and aims to solve the problems that an existing data quality detection method is low in efficiency and high in cost. In order to achieve the purpose, the method provided by the invention comprises the steps of obtaining to-be-detected data based on an original image marked with a target; cutting an original image in the to-be-detected data to obtain a plurality of sub-images and target labeling information and real information corresponding to each sub-image; inputting each sub-image and the corresponding target annotation information into a trained quality detection model to obtain a quality detection result of each sub-image, including a target orientation detection result and a three-dimensional annotation box detection result; and obtaining a quality detection result of the to-be-detected data based on the quality detection results of the plurality of sub-images. Through the above implementation mode, automatic screening of large-scale data driven by a small model is realized, the efficiency and accuracy of data quality detection are improved, and the cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data quality detection method, an intelligent device and a storage medium. Background Art

[0002] With the rapid development of autonomous driving technology, the demand for high-quality training data in autonomous driving systems has become increasingly important. Autonomous driving data is usually collected and generated by a large number of sensors, and the data sets used to train models are obtained through manual or automatic annotation. However, as the scale of data increases, errors and noise (i.e. "dirty data") in automatic or manual annotation are difficult to avoid and are easily overlooked, which not only affects the data quality, but also the performance and safety of the model.

[0003] Most existing data quality detection methods rely on manual review or the use of large, computationally complex models for data screening, which results in low efficiency and high costs, and is unable to meet the real-time processing needs of massive data.

[0004] Therefore, there is an urgent need for a method that can automatically detect and identify dirty data in the autonomous driving data labeling process to improve data accuracy and labeling efficiency and reduce the cost of autonomous driving data processing. Summary of the invention

[0005] In order to overcome the above-mentioned defects, the present application is proposed to solve or at least partially solve the technical problems of low efficiency and high cost of existing data quality detection methods.

[0006] In a first aspect, a data quality detection method is provided, the method comprising:

[0007] Acquire the data to be detected based on the original image with the target marked;

[0008] The original image in the data to be detected is cropped to obtain a plurality of sub-images and target annotation information and real information corresponding to each sub-image; wherein the plurality of sub-images respectively correspond to different targets in the original image;

[0009] Input each of the sub-images, the corresponding target annotation information and the real information into a trained quality detection model to obtain a quality detection result of each of the sub-images; the quality detection result includes a detection result of the orientation of the target and a detection result of a three-dimensional annotation box;

[0010] A quality detection result of the data to be detected is obtained based on the quality detection results of the multiple sub-images.

[0011] In a technical solution of the above data quality detection method, the cropping of the original image in the data to be detected to obtain multiple sub-images and target annotation information and real information corresponding to each sub-image includes:

[0012] Obtaining a two-dimensional annotation box of each of the targets in the original image;

[0013] Enlarging the two-dimensional annotation frame based on a preset ratio to obtain a clipping frame of the target;

[0014] Cropping each of the targets in the original image based on the cropping frame to obtain a sub-image corresponding to each target, the target annotation information and real information;

[0015] The target annotation information is information obtained by annotating the target in the original image, and the real information at least includes the yaw angle of the sub-image.

[0016] In a technical solution of the above data quality detection method, the trained quality detection model includes a feature extraction module, an orientation detection module and a three-dimensional annotation box detection module; inputting each of the sub-images, the corresponding target annotation information and the real information into the trained quality detection model to obtain the quality detection result of each of the sub-images includes:

[0017] Adjusting the sub-image corresponding to each of the targets to a preset size;

[0018] Inputting the sub-image of the preset size, the target annotation information and the real information into the feature extraction module to obtain a feature vector of the target;

[0019] Inputting the feature vector of the target into the orientation detection module and the three-dimensional annotation box detection module respectively, to obtain the orientation detection result and the three-dimensional annotation box detection result of the target;

[0020] The orientation detection result includes a correct orientation, a deflected orientation, and a reversed orientation, and the three-dimensional annotation frame detection result includes a correct three-dimensional annotation frame and an incorrect three-dimensional annotation frame.

[0021] In a technical solution of the above data quality detection method, after inputting the sub-image of the preset size, the target annotation information and the real information into the feature extraction module to obtain the feature vector of the target, the method further includes:

[0022] Inputting the yaw angle of the sub-image into the encoding module to obtain a numerical code corresponding to the yaw angle;

[0023] The feature vector and the numerical code of the target are input into the direction detection module to obtain the direction detection result of the target.

[0024] In a technical solution of the above data quality detection method, the quality detection model is trained based on at least the following steps:

[0025] Get training set and test set;

[0026] Based on the training set, the feature extraction module, the orientation detection module and the three-dimensional annotation box detection module in the quality detection model are trained;

[0027] When the quality detection model reaches a first preset accuracy, the quality detection model is iteratively trained based on the test set until the quality detection model reaches a second preset accuracy, thereby completing the quality detection model training.

[0028] In a technical solution of the above data quality detection method, obtaining a training set and a test set includes:

[0029] Obtain machine pre-labeled data sets and manually labeled data sets;

[0030] trimming each machine-labeled image in the machine pre-labeled data set to obtain a plurality of sub-images labeled with objects as a training set;

[0031] Each manually annotated image in the manually precisely labeled data set is cropped to obtain a plurality of sub-images annotated with targets as the test set.

[0032] In a technical solution of the above data quality detection method, the method further includes:

[0033] Performing data enhancement processing on the training set to obtain a negative sample set;

[0034] The feature extraction module, the orientation detection module and the three-dimensional annotation box detection module in the quality detection model are trained based on the training set and the negative sample set.

[0035] In a technical solution of the above data quality detection method, performing data enhancement processing on the training set to obtain a negative sample set includes at least one of the following:

[0036] Reversing the three-dimensional annotation boxes of at least part of the sub-images in the training set to obtain a negative sample set with reversed orientation and correct three-dimensional annotation boxes;

[0037] Changing the angle of the three-dimensional annotation boxes of at least part of the sub-images in the training set to obtain a negative sample set with deflected orientation and wrong three-dimensional annotation boxes;

[0038] resizing the three-dimensional annotation boxes of at least some of the sub-images in the training set to obtain a negative sample set with correct orientations and incorrect three-dimensional annotation boxes;

[0039] The inversion and the size change are performed on the three-dimensional annotation boxes of at least part of the sub-images in the training set to obtain a negative sample set with inverted orientation and wrong three-dimensional annotation boxes.

[0040] In a second aspect, a smart device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any one of the technical solutions of the above-mentioned data quality detection method is implemented.

[0041] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the method described in any one of the technical solutions of the above-mentioned data quality detection method.

[0042] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:

[0043] In the technical solution of the data quality detection method provided in the present application, the original image in the data to be detected is cropped into multiple sub-images, and quality detection is performed on each sub-image respectively. Since the size of the sub-image targeted by the quality detection is smaller than the original image, there is no need to use a large model when detecting the sub-image. Only a small model with a simple structure can be used to complete the quality detection. According to the quality detection results, dirty data in the labeled data can be automatically identified. It is suitable for real-time processing of large-scale data, and realizes the automatic screening of large-scale data driven by small models, avoiding excessive resource consumption of high-computation models on large data sets, improving the efficiency and accuracy of data quality detection, and reducing the cost of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The disclosure of the present application will become easier to understand with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. Among them:

[0045] Figure 1 It is a flowchart of the main steps of a data quality detection method according to an embodiment of the present application;

[0046] Figure 2 This is a schematic flow chart of the main steps of trimming the original image in the data to be detected according to an embodiment of the present application;

[0047] Figure 3 It is a flowchart of the main steps of training a quality detection model according to an embodiment of the present application;

[0048] Figure 4 It is a flowchart of the main steps of inputting each sub-image and the corresponding target annotation information into a trained quality detection model to obtain the quality detection result of each sub-image according to an embodiment of the present application;

[0049] Figure 5 It is a main flow chart of a data quality detection method according to an embodiment of the present application;

[0050] Figure 6 is a schematic diagram of a smart device according to an embodiment of the present application.

[0051] Reference numerals:

[0052] 61: memory; 62: processor; 63: sensor. DETAILED DESCRIPTION

[0053] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0054] In the description of the present application, the term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one of A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B, or A and B. The singular terms "a", "the" and "the" may also include plural forms.

[0055] As described in the background technology, with the rapid development of autonomous driving technology, the demand for high-quality training data in autonomous driving systems has become increasingly important. Autonomous driving data is usually collected and generated by a large number of sensors, and the data set used to train the model is obtained through manual or automatic annotation. However, as the scale of data increases, errors and noise (i.e., "dirty data") in automatic or manual annotation are difficult to avoid and are easily overlooked, which not only affects the data quality, but also affects the performance and safety of the model.

[0056] Data quality detection methods include the following:

[0057] 1. Statistical and rule-based quality inspection: This is the earliest quality inspection method, which relies on manually set rules and statistical thresholds, such as identifying outliers, duplicate annotations, and missing annotations. This type of method has low implementation costs, but has limited effectiveness in identifying complex errors or mislabeled data that is difficult to quantify.

[0058] 2. Quality inspection models based on supervised learning: In recent years, quality inspection models based on machine learning have been developed, which automatically identify potential dirty data by learning the error patterns in the labeled data. For example, supervised models are used to detect abnormal patterns or samples that significantly deviate from the data distribution. Such methods can automatically detect some labeling errors on large-scale data sets, but their accuracy is more dependent on the quality of the training data.

[0059] 3. Comparison between deep learning and automatic labeling: The powerful recognition ability of deep learning is used for quality inspection. For example, a deep learning model with a small number of parameters is used to detect label data in low-resolution samples and compare it with the results of a large model to identify labeling errors. This method has high detection accuracy and is suitable for image or text data, but it has high computational cost and data dependence.

[0060] 4. Human-machine collaborative quality inspection: In order to optimize the quality inspection process, some fields have also adopted a human-machine collaborative quality inspection model, which gives the results of automated inspections to manual review to improve the accuracy of annotations. This model saves manual review time, but there are still problems such as high costs and difficulty in real-time processing.

[0061] In summary, although the above-mentioned data quality detection methods provide a certain degree of automation capabilities for labeling data quality detection, most of them rely on manual review or use large models with complex calculations for data screening, which are inefficient and costly, and cannot meet the real-time processing needs of massive data.

[0062] In order to solve the above problems, the present application provides a data quality detection method, an intelligent device and a storage medium.

[0063] See attached Figure 1 , Figure 1 FIG. 1 is a flow chart of the main steps of a data quality detection method according to an embodiment of the present application. Figure 1 As shown, the data quality detection method in the embodiment of the present application mainly includes the following steps S101 to S104.

[0064] Step S101: acquiring data to be detected based on the original image with the target marked;

[0065] Step S102: cropping the original image in the to-be-detected data to obtain a plurality of sub-images and target annotation information and real information corresponding to each sub-image;

[0066] Among them, multiple sub-images correspond to different targets in the original image respectively.

[0067] Step S103: input each sub-image, the corresponding target annotation information and the real information into the trained quality detection model to obtain the quality detection result of each sub-image;

[0068] Among them, the quality detection results include the target orientation detection results and the three-dimensional annotation box detection results.

[0069] Step S104: obtaining a quality detection result of the data to be detected based on the quality detection results of the multiple sub-images.

[0070] Based on the method described in steps S101 to S104 above, the original image in the data to be tested is cropped into multiple sub-images, and quality testing is performed on each sub-image respectively. Since the size of the sub-image targeted by the quality test is smaller than the original image, there is no need to use a large model when testing the sub-image. Only a small model with a simple structure can be used to complete the quality test. Dirty data in the labeled data is automatically identified based on the quality test results. This method is suitable for real-time processing of large-scale data, realizes automatic screening of large-scale data driven by small models, avoids excessive resource consumption of high-computation models on large data sets, improves the efficiency and accuracy of data quality testing, and reduces the cost of data processing.

[0071] The above steps S101 to S104 are further explained below.

[0072] In some implementations of the above step S101, the image data annotated manually or automatically, that is, the original image annotated with target information, can be used as the data to be detected. The original image in the data to be detected has been annotated with information such as the three-dimensional annotation box, the two-dimensional annotation box, the orientation, the category and the attributes of each target. In addition, the data to be detected also includes the real information of the original image.

[0073] Among them, in the field of autonomous driving, the target refers to various objects in the surrounding environment that need to be identified and located during the driving process of the autonomous driving vehicle, such as vehicles, pedestrians, obstacles, traffic signs and signal lights.

[0074] The three-dimensional annotation box of the target is to mark the position of the target in the three-dimensional space corresponding to the original image through a solid geometric figure similar to a cuboid. The two-dimensional annotation box is to mark the boundary range of the target on the original image through a simple rectangular box. The orientation of the target refers to the direction of the target object. Taking a vehicle as an example, the orientation of the vehicle can be the direction in which the front of the vehicle is pointing. The category of the target refers to the category to which the target belongs. For example, in a traffic scene image, the category of the target can include vehicles, pedestrians, traffic signs, street lights, etc. The attributes of the target are the characteristic descriptions of the target object itself. Taking a vehicle as an example, its attributes can include vehicle model, body size, speed, etc.

[0075] The real information of the original image is the real label data of the original image, including at least the yaw angle. The yaw angle of an image refers to the rotation angle of the image content in the horizontal plane relative to a reference direction. This reference direction can be the initial orientation of the image acquisition device (such as a camera). For targets with a clear forward direction, the yaw angle is closely related to the orientation of the target. For a vehicle, for example, if the yaw angle of the vehicle is 0 degrees, and the front is defined as the 0-degree reference direction, then it can be determined that the vehicle is moving forward and the front of the vehicle is facing forward; if the yaw angle is 90 degrees, then the vehicle is moving to the right and the front of the vehicle is facing the right.

[0076] In addition, the real information of the original image may also include numerical information of the three-dimensional annotation box (such as the length, width, height size information and center position coordinates of the three-dimensional annotation box), numerical information of the two-dimensional annotation box (such as the length and width of the two-dimensional annotation box, and the center position coordinates), and internal and external parameters of the image acquisition device (internal parameters mainly include focal length, principal point coordinates and distortion coefficients, and external parameters mainly include translation vectors, rotation matrices or equivalent Euler angles, such as yaw angles, pitch angles, roll angles, etc.), which are not limited here.

[0077] It should be pointed out that the above examples of the annotation information and real information of the original image are only illustrative. In practical applications, those skilled in the art can make settings according to specific scenarios, which are not limited here.

[0078] The above is a further description of step S101 , and the following is a further description of step S102 .

[0079] In some implementations of the above step S102, the original image in the to-be-detected data may be cropped to obtain a plurality of sub-images and target annotation information and true information corresponding to each sub-image.

[0080] For details, see the attached Figure 2 , Figure 2 FIG. 1 is a flow chart of the main steps of trimming the original image in the detection data according to an embodiment of the present application. Figure 2 As shown, step S102 mainly includes the following steps S1021 to S1023.

[0081] Step S1021: Obtain a two-dimensional annotation box of each target in the original image;

[0082] Specifically, the corresponding two-dimensional annotation box can be obtained according to the annotation information of each target in the original image.

[0083] Step S1022: enlarging the two-dimensional annotation frame based on a preset ratio to obtain a clipping frame of the target;

[0084] When cropping an object in an original image, cropping only by relying on a two-dimensional annotation frame may result in incomplete object information or missing key parts in the cropped sub-image. Therefore, the two-dimensional annotation frame can be enlarged according to a preset ratio to obtain a cropping frame, ensuring that the subsequent cropped sub-image is more representative and complete.

[0085] Specifically, a magnification ratio value (such as 1.2 times, 1.5 times, etc.) can be preset, and the acquired two-dimensional annotation frame of each target is proportionally enlarged in both length and width according to the preset ratio. For example, if the length of a two-dimensional annotation frame of a target in the original image is 100 pixels and the width is 80 pixels, and the preset magnification ratio is 1.2 times, then the length of the enlarged cropping frame becomes 120 pixels and the width becomes 96 pixels, so as to obtain a larger cropping frame for subsequent cropping operations.

[0086] Step S1023: cropping each target in the original image based on the cropping frame to obtain a sub-image, target annotation information and real information corresponding to each target;

[0087] Among them, the target annotation information is the information obtained by annotating the target in the original image, and the target annotation information may include the target's three-dimensional annotation box, two-dimensional annotation box, orientation, category, attribute, size of the original image, and position information of the sub-image on the original image, etc.; the real information is the real label data of the original image, and the real information may include the yaw angle, pitch angle, roll angle, etc. of the sub-image.

[0088] Specifically, the area defined by each cropping frame may be used as the cropping range, and a corresponding portion may be cut out from the original image to obtain a sub-image corresponding to each target.

[0089] When the sub-image is cropped, the target annotation information and real information corresponding to the target in the sub-image should be extracted. In addition to the 3D annotation box, 2D annotation box, orientation, category and attributes of the target mentioned above, the target annotation information also includes the size of the original image and the position information of the sub-image on the original image, such as the position of the center coordinates of the sub-image in the original image, which is used to clarify the specific position of the original image from which the sub-image is cropped.

[0090] Through the above-mentioned cropping operation, the original image can be decomposed into multiple sub-images containing only a single target and its surrounding related information. Since the size of the sub-image targeted by quality inspection is smaller than the original image, there is no need to use a large model when inspecting the sub-image. Only a small model with a simple structure can be used to analyze and detect the target in each sub-image.

[0091] The above is a further description of step S102 , and the following is a further description of step S103 .

[0092] In some implementations of the above step S103, a quality detection model with a simple structure and a small number of parameters can be constructed first. Specifically, in order to detect whether the target orientation and the three-dimensional annotation box in the annotation data are accurate, the constructed quality detection model can include a feature extraction module, an orientation detection module and a three-dimensional annotation box detection module.

[0093] Among them, the feature extraction module can be a convolutional neural network (CNN), such as Resnet50, etc. The feature extraction module is used to extract various feature information that can characterize the target from the input data, such as mining the texture, shape, color and other aspects of the target, and combine multiple related features together to form an ordered feature vector.

[0094] In some embodiments, the feature extraction module may include an encoding module. In the target annotation information, the yaw angle related to the target orientation is usually expressed by an angle. However, if the angle data is directly input into the model, the model may be difficult to recognize, so an encoding module may be set in the feature extraction module, and a specific encoding algorithm or rule may be used to encode the angle data and convert the angle data into a numerical form that is more suitable for model recognition and processing.

[0095] The orientation detection module and the 3D annotation box detection module can be fully connected layers (FC). The orientation detection module uses various features including the encoded angle data provided by the feature extraction module to accurately detect the orientation of the target. At the same time, the 3D annotation box detection module also determines the accuracy of the target's 3D annotation box based on the feature information extracted by the feature extraction module.

[0096] Among them, the target orientation detection result output by the orientation detection module may include correct orientation, deflected orientation and reversed orientation; the 3D annotation box detection result output by the 3D annotation box detection module may include correct 3D annotation box and incorrect 3D annotation box.

[0097] Furthermore, after constructing the above-mentioned simple quality detection model, the quality detection model can be trained.

[0098] Since the quality detection model has a simple structure and relatively few neuron connections and network layers, the number of parameters that the model needs to learn is also small. During the training process, the model continuously learns and optimizes the parameters of each module (such as weight parameters, bias parameters, etc.), and finally obtains a quality detection model with a simple structure and a small number of parameters.

[0099] For details, see the attached Figure 3 , Figure 3 FIG. 1 is a flow chart of the main steps of training a quality detection model according to an embodiment of the present application. Figure 3 As shown, it mainly includes the following steps S301 to S303.

[0100] Step S301: Obtain training set and test set;

[0101] In some implementations, step S301 may include the following steps S3011 to S3013:

[0102] Step S3011: Acquire a machine pre-labeled data set and a manually precisely labeled data set;

[0103] Specifically, two datasets with different degrees of annotation can be collected. The machine-pre-annotated dataset is a dataset obtained by automatically annotating a large amount of data collected by sensors through the target detection business model. This annotation method is relatively fast but has certain inaccuracies. The manually-annotated dataset is a dataset obtained by manually annotating a small amount of data. The amount of annotated data in this dataset is small and the accuracy is relatively high.

[0104] Step S3012: trimming each machine-annotated image in the machine pre-annotated data set to obtain a plurality of sub-images annotated with objects as a training set;

[0105] By performing a cropping operation on each machine-labeled image in the machine pre-labeled dataset, it can be processed into multiple small-sized sub-images that only have a single target labeled, and these sub-images constitute the training set.

[0106] Step S3013: crop each manually annotated image in the manually precisely labeled data set to obtain multiple sub-images annotated with targets as a test set.

[0107] Similarly, by cropping each manually annotated image in the manual precision labeling dataset, we can obtain multiple small-sized sub-images with only a single target annotated on them as a test set.

[0108] The method for trimming the machine-annotated image and the manually-annotated image is described in detail in the embodiment of the above step S102 and will not be described again here.

[0109] Furthermore, in some implementations, in order to improve the generalization ability of the quality detection model, data enhancement processing may be performed on the training set to obtain a negative sample set.

[0110] Among them, the negative sample set corresponds to the positive samples of the training set (i.e., samples with correct annotations). By introducing negative samples, the model can better learn various situations in the data, including erroneous situations, thereby improving the model's ability to recognize different situations.

[0111] Specifically, the three-dimensional annotation boxes of at least some sub-images in the training set may be reversed to obtain a negative sample set with reversed orientation and correct three-dimensional annotation boxes.

[0112] When the orientation of the 3D annotation box is reversed, for example, the 3D annotation box of a car originally faces the front of the car, but after being reversed, it faces the rear of the car. From the perspective of the target, the orientation of the 3D annotation box changes by 180 degrees, but the length, width, height and center coordinates of the 3D annotation box do not change, and the position and size of the 3D annotation box are still correct relative to the target. The sample obtained in this way is a negative sample with reversed orientation and correct 3D annotation box.

[0113] In some implementations, the angle of the 3D annotation boxes of at least some sub-images in the training set may be changed to obtain a negative sample set with deflected orientation and incorrect 3D annotation boxes.

[0114] The angle change range of the 3D annotation frame can be between 20° and 340°. When the orientation of the 3D annotation frame is deflected, for example, the 3D annotation frame of the vehicle is rotated 30 degrees clockwise, the length, width, and height of the 3D annotation frame in the 3D space cannot accurately reflect the true orientation and position of the target, resulting in the orientation of the annotation being deflected and the 3D annotation frame being wrong. The sample obtained in this way is a negative sample with a deflected orientation and an incorrect 3D annotation frame.

[0115] In some implementations, the size of the 3D annotation boxes of at least some sub-images in the training set may be changed to obtain a negative sample set with correct orientations and incorrect 3D annotation boxes.

[0116] Specifically, by only changing the size of the 3D annotation box in the sub-image, negative samples with the correct orientation and the wrong 3D annotation box can be obtained.

[0117] In some implementations, the 3D annotation boxes of at least some sub-images in the training set may be reversed and resized to obtain a negative sample set with reversed orientation and incorrect 3D annotation boxes.

[0118] In practical applications, technical personnel in this field can use one or more of the above-mentioned data enhancement methods to obtain negative sample sets according to specific needs, such as reversing the 3D annotation boxes of all sub-images in the training set to obtain a negative sample set with reversed orientation and correct 3D annotation boxes; or changing the angle of the 3D annotation boxes of some sub-images in the training set, reversing and changing the size of the 3D annotation boxes of another part of the sub-images in the training set to obtain a negative sample set with deflected orientation and incorrect 3D annotation boxes, as well as a negative sample set with reversed orientation and incorrect 3D annotation boxes, etc., so that the model can more comprehensively recognize different types of annotation errors and further improve the model's ability to judge the accuracy of orientation detection and the accuracy of 3D annotation boxes when detecting data quality.

[0119] The above is a further description of step S301.

[0120] Step S302: training a feature extraction module, an orientation detection module, and a 3D annotation box detection module in the quality detection model based on the training set;

[0121] Specifically, the feature extraction module, orientation detection module and 3D annotation box detection module in the quality detection model can be trained based on the training set and the negative sample set. During the training process, the model will continuously adjust the internal parameters of each module so that each module can better complete its corresponding detection task.

[0122] Step S303: When the quality detection model reaches a first preset accuracy, the quality detection model is iteratively trained based on the test set until the quality detection model reaches a second preset accuracy, and the quality detection model training is completed.

[0123] Specifically, when the quality detection model reaches the first preset accuracy in the training process based on the training set, the test set can be used to drive the model for iterative training. Among them, the first preset accuracy is a pre-set indicator to measure the performance of the model on the training set, such as the accuracy reaching a certain value. By introducing the test set for iterative training after reaching the first preset accuracy, the model can be optimized until the model reaches the second preset accuracy. Among them, the second preset accuracy is an indicator to measure the performance of the model on the test set, such as converging to a preset error value, etc. At this time, it is considered that the model has been trained and can perform quality detection on the data more stably.

[0124] The above is an explanation of the quality inspection model.

[0125] Furthermore, in some implementations of step S103, each sub-image and the corresponding target annotation information may be input into a trained quality detection model to obtain a quality detection result for each sub-image.

[0126] See attached Figure 4 , Figure 4 This is a flow chart of the main steps of inputting each sub-image and the corresponding target annotation information into a trained quality detection model to obtain the quality detection result of each sub-image according to an embodiment of the present application. Figure 4 As shown, step S103 mainly includes the following steps S1031 to S1033.

[0127] Step S1031: adjusting the sub-image corresponding to each target to a preset size;

[0128] Specifically, in order to make the data input into the quality inspection model have a uniform specification and facilitate the model to process and analyze, the sub-images corresponding to different targets can be adjusted to a preset size to ensure that the model will not produce deviations due to different sizes when processing data.

[0129] The preset size may be a fixed width and height, such as 224×224 pixels, which is not limited here.

[0130] Step S1032: inputting the sub-image of a preset size, target annotation information and real information into a feature extraction module to obtain a feature vector of the target;

[0131] After inputting the sub-image of preset size, target annotation information and real information into the quality detection model, the feature extraction module can use image processing technology and algorithms to analyze and process the input data, extract various feature information that can characterize the target, and combine these feature information into a feature vector.

[0132] Furthermore, the feature extraction module may include a coding module, and the yaw angle of the sub-image may be input into the coding module to obtain a numerical code corresponding to the yaw angle.

[0133] Among them, the encoding module is used to encode the yaw angle of the target, and convert the yaw angle into a corresponding numerical value through a set encoding algorithm or rule, so that the angle data becomes a numerical form that is more suitable for model processing.

[0134] Step S1033: inputting the feature vector of the target into the orientation detection module and the three-dimensional annotation box detection module respectively, to obtain the orientation detection result and the three-dimensional annotation box detection result of the target.

[0135] Specifically, the target's feature vector and numerical code can be input into the orientation detection module to obtain the target's orientation detection result. The orientation detection module will determine the target's orientation based on the received target's feature vector and the numerical code corresponding to the yaw angle, and obtain three results: correct orientation, deflected orientation, and reversed orientation.

[0136] The direction detection result has multiple output forms. For example, the deviation between the target's marked direction and the actual direction is output in the form of an angle. When the deviation between the target's marked direction and the actual direction is 0°±10°, it indicates that the direction is accurate; when the deviation between the target's marked direction and the actual direction is 180°±10°, it indicates that the direction is reversed; when the deviation between the target's marked direction and the actual direction is other angles, it indicates that the direction is deflected.

[0137] In addition, the direction detection result can also be output in the form of a numerical value, such as 0 for the correct direction, 1 for the reverse direction, and 2 for the deflected direction.

[0138] The 3D annotation box detection module detects the length, width, height, and center point of the target's 3D annotation box based on the input feature vector. The 3D annotation box detection results include two situations: the 3D annotation box is correct and the 3D annotation box is incorrect.

[0139] The 3D annotation box detection result can also be output in a variety of forms. For example, in the form of a numerical value, 0 indicates that the 3D annotation box is correct, and 1 indicates that the 3D annotation box is wrong. Or it can be expressed in the form of a score. For example, a score between 0 and 100 is output, and a score threshold is set. When the output score is greater than or equal to the threshold, the 3D annotation box is determined to be correct, and when it is lower than the threshold, the 3D annotation box is determined to be wrong.

[0140] Specifically, the score threshold can be set according to actual needs. If a higher accuracy is required, the score threshold can be set higher, such as 80; if a higher recall rate is required, the score threshold can be set lower, such as 60. There is no limitation here.

[0141] It should be pointed out that the above examples of the target orientation detection results and the three-dimensional annotation box detection results are only schematic illustrations. In practical applications, those skilled in the art can make settings according to specific scenarios, and no limitation is made here.

[0142] Through the above-mentioned quality detection model, the quality detection results of each sub-image can be obtained, which is suitable for real-time processing of large-scale data. It realizes the automatic screening of large-scale data driven by small models and avoids excessive resource consumption of high-computation models on large data sets.

[0143] The above is the description of step S103.

[0144] Furthermore, in some implementations of the above step S104, the quality detection results obtained from the various sub-images may be summarized to obtain the quality detection result of the data to be detected, so as to identify dirty data in the data to be detected according to the quality detection result.

[0145] The above is the description of step S101 to step S104.

[0146] For further information, see Appendix Figure 5 , Figure 5 It is a main flow chart of a data quality detection method according to an embodiment of the present application.

[0147] like Figure 5 As shown, firstly, the data to be detected 52 is obtained based on the original image 51 with the target marked, and the original image in the data to be detected 52 is cropped to obtain multiple sub-images 53 and the target marking information and real information corresponding to each sub-image;

[0148] Further, the sub-image 53 corresponding to each target is adjusted to a preset size, and then the sub-image of the preset size, the target annotation information and the real information are input into the feature extraction module 54 to obtain the feature vector of the target, and the numerical code corresponding to the yaw angle is obtained through the encoding module 541 in the feature extraction module;

[0149] Furthermore, the target's feature vector and numerical code are input into the orientation detection module 55 to obtain the target's orientation detection result 56, and the target's feature vector is input into the three-dimensional annotation box detection module 57 to obtain the target's orientation detection result 58, thereby obtaining the quality detection result of each sub-image, and finally, based on the quality detection results of multiple sub-images, the quality detection result 59 of the data to be detected is obtained.

[0150] Through the above implementation, it is possible to drive the automated screening of large-scale data based on a small model with a simple structure, avoid excessive resource consumption of high-computation models on large data sets, realize automated and low-cost identification of dirty data, timely discover erroneous annotations, improve data quality, and provide higher-quality data support for business models.

[0151] The data quality detection method provided in this application is applicable to a wide range of business scenarios. It can be applied not only to new data of automated labeling production lines, but also to existing stock data, making the above technical solution practical and economical in a variety of scenarios.

[0152] For new data of the automated labeling production line, a small model with a simple structure can be integrated into the data automation production line, and dirty data can be identified by analyzing the labeled data, ensuring that the quality of the newly added data can be screened in real time after the labeling results are generated.

[0153] It also has the ability to clean dirty data in existing data, and can quickly locate and remove incorrect annotations in the labeled data in the existing autonomous driving data set, thereby improving the overall data quality of the model training set and providing a guarantee for improving model performance.

[0154] The above is an explanation of the data quality detection method provided in this application.

[0155] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in this application, and therefore will also fall within the scope of protection of this application.

[0156] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of each of the above-mentioned method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. Non-transitory computer-readable storage media may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0157] Furthermore, the present application also provides a smart device.

[0158] In an embodiment of an intelligent device according to the present application, the intelligent device may include at least one processor; and a memory connected to the at least one processor in communication; wherein a computer program is stored in the memory, and when the computer program is executed by at least one processor, the data quality detection method described in any of the above embodiments is implemented. "Processor" may include hardware, software, or a combination of the two. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware, or a combination of the two. The intelligent devices described in the present application may include driving devices, smart cars, robots, and other devices.

[0159] In some embodiments of the present application, the smart device may further include at least one sensor, which is used to sense information and is communicatively connected to any type of processor mentioned in the present application.

[0160] See attached Figure 6 , Figure 6 is a schematic diagram of a smart device according to an embodiment of the present application, such as Figure 6As shown, the smart device may include a memory 61, a processor 62 and a sensor 63, wherein the memory 61, the processor 62 and the sensor 63 are communicatively connected via a bus.

[0161] In some embodiments of the present application, the smart device may also include an autonomous driving system, which is used to guide the smart device to drive itself or assist in driving. The processor communicates with the sensor and / or the autonomous driving system to complete the method described in any of the above embodiments.

[0162] Furthermore, the present application also provides a computer-readable storage medium.

[0163] In an embodiment of a computer-readable storage medium according to the present application, the computer-readable storage medium may be configured to store a program for executing the data quality detection method of the above method embodiment, and the program may be loaded and run by a processor to implement the above data quality detection method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method section of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

[0164] It should be noted that the relevant user personal information that may be involved in the various embodiments of the present application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0165] The user personal information processed by this application will vary depending on the specific product / service scenario, and shall be based on the specific scenario in which the user uses the product / service, and may involve the user's account information, device information, driving information, vehicle information or other related information. This application will treat the user's personal information and its processing with a high degree of diligence.

[0166] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.

[0167] So far, the technical solution of the present application has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

Claims

1. A data quality detection method, characterized in that: The method comprises: Acquire the data to be detected based on the original image with the target marked; The original image in the data to be detected is cropped to obtain a plurality of sub-images and target annotation information and real information corresponding to each sub-image; wherein the plurality of sub-images respectively correspond to different targets in the original image; Input each of the sub-images, the corresponding target annotation information and the real information into a trained quality detection model to obtain a quality detection result of each of the sub-images; the quality detection result includes a detection result of the orientation of the target and a detection result of a three-dimensional annotation box; A quality detection result of the data to be detected is obtained based on the quality detection results of the multiple sub-images.

2. The data quality detection method according to claim 1, characterized in that: The cropping of the original image in the to-be-detected data to obtain a plurality of sub-images and target annotation information and real information corresponding to each sub-image includes: Obtaining a two-dimensional annotation box of each of the targets in the original image; Enlarging the two-dimensional annotation frame based on a preset ratio to obtain a clipping frame of the target; Cropping each of the targets in the original image based on the cropping frame to obtain a sub-image corresponding to each of the targets, the target annotation information and real information; The target annotation information is information obtained by annotating the target in the original image, and the real information at least includes the yaw angle of the sub-image.

3. The data quality detection method according to claim 2, characterized in that: The trained quality detection model includes a feature extraction module, an orientation detection module and a three-dimensional annotation box detection module; the inputting each of the sub-images, the corresponding target annotation information and the real information into the trained quality detection model to obtain the quality detection result of each of the sub-images includes: Adjusting the sub-image corresponding to each of the targets to a preset size; Inputting the sub-image of the preset size, the target annotation information and the real information into the feature extraction module to obtain a feature vector of the target; Inputting the feature vector of the target into the orientation detection module and the three-dimensional annotation box detection module respectively, to obtain the orientation detection result and the three-dimensional annotation box detection result of the target; The orientation detection result includes a correct orientation, a deflected orientation, and a reversed orientation, and the three-dimensional annotation frame detection result includes a correct three-dimensional annotation frame and an incorrect three-dimensional annotation frame.

4. The data quality detection method according to claim 3, characterized in that: After inputting the sub-image of the preset size, the target annotation information and the real information into the feature extraction module to obtain the feature vector of the target, the method further includes: Inputting the yaw angle of the sub-image into the encoding module to obtain a numerical code corresponding to the yaw angle; The feature vector and the numerical code of the target are input into the direction detection module to obtain the direction detection result of the target.

5. The data quality detection method according to claim 3, characterized in that: The quality detection model is trained based on at least the following steps: Get training set and test set; Based on the training set, the feature extraction module, the orientation detection module and the three-dimensional annotation box detection module in the quality detection model are trained; When the quality detection model reaches a first preset accuracy, the quality detection model is iteratively trained based on the test set until the quality detection model reaches a second preset accuracy, thereby completing the quality detection model training.

6. The data quality detection method according to claim 5, characterized in that: The obtaining of the training set and the test set comprises: Obtain machine pre-labeled data sets and manually labeled data sets; trimming each machine-labeled image in the machine pre-labeled data set to obtain a plurality of sub-images labeled with objects as a training set; Each manually annotated image in the manually precisely labeled data set is cropped to obtain a plurality of sub-images annotated with targets as the test set.

7. The data quality detection method according to claim 5, characterized in that: The method further comprises: Performing data enhancement processing on the training set to obtain a negative sample set; The feature extraction module, the orientation detection module and the three-dimensional annotation box detection module in the quality detection model are trained based on the training set and the negative sample set.

8. The data quality detection method according to claim 7, characterized in that: The performing data enhancement processing on the training set to obtain a negative sample set includes at least one of the following: Reversing the three-dimensional annotation boxes of at least part of the sub-images in the training set to obtain a negative sample set with reversed orientation and correct three-dimensional annotation boxes; Changing the angle of the three-dimensional annotation boxes of at least part of the sub-images in the training set to obtain a negative sample set with deflected orientation and wrong three-dimensional annotation boxes; resizing the three-dimensional annotation boxes of at least some of the sub-images in the training set to obtain a negative sample set with correct orientations and incorrect three-dimensional annotation boxes; The inversion and the size change are performed on the three-dimensional annotation boxes of at least part of the sub-images in the training set to obtain a negative sample set with inverted orientation and wrong three-dimensional annotation boxes.

9. A smart device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; Wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the data quality detection method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the data quality detection method according to any one of claims 1 to 8.

Images