Quality processing evaluation method and system based on perception service data

Abstract

The invention provides a quality processing evaluation method and system based on perception service data. The method comprises the following steps: inputting the collected perception service data into a system through a data entry module; performing de-duplication filtering on the sensing service data through a data cleaning module; carrying out quantity amplification on the sensing service data through a data enhancement module; performing automatic sample labeling on the business field data through a data labeling module; and setting an evaluation weight ratio of each dimension through a data quality evaluation module, calculating a comprehensive data quality score of the perception service data based on the data coverage, the data uniformity, the data signal-to-noise ratio and the data definition, and visually outputting a result to complete quality processing evaluation based on the perception service data. By applying the technical scheme of the invention, the technical problems of single data processing evaluation mode, low efficiency and backward method in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent perception technology, and in particular to a quality processing evaluation method and system based on perception business data. Background Art

[0002] Artificial intelligence technology based on big data is developing rapidly around the world. It has achieved remarkable results in visual processing, natural language understanding, speech, recommendation, reinforcement learning and other fields. It has now become a new focus of international competition. Various technological powers are trying to seize favorable opportunities to position themselves at the commanding heights in this field. However, the rapid development of artificial intelligence is inseparable from the support of high-quality and highly reliable sample data. It can be said that there is no AI without data. Especially in the field of perception technology, various algorithms need to rely on a large amount of data support obtained through actual measurement, simulation, small sample amplification, etc., and require tedious classification, cleaning, labeling and quality inspection. However, the existing data processing and evaluation methods are single, inefficient, and backward, resulting in complex sample data processing methods and low reliability of quality evaluation. Summary of the invention

[0003] The present invention provides a quality processing evaluation method and system based on perceived business data, which can solve the technical problems in the prior art of single data processing evaluation method, low efficiency and backward method.

[0004] According to one aspect of the present invention, a quality processing and evaluation method based on perceived business data is provided, and the quality processing and evaluation method based on perceived business data includes: inputting the collected perceived business data into the system through a data entry module; deduplicating and filtering the perceived business data through a data cleaning module; amplifying the quantity of the perceived business data through a data enhancement module; automatically annotating samples of business domain data through a data annotation module; setting the evaluation weight ratio of each dimension through a data quality assessment module, calculating the comprehensive data quality score of the perceived business data based on data coverage, data uniformity, data signal-to-noise ratio and data clarity, and visually outputting the results, thereby completing the quality processing and evaluation based on the perceived business data.

[0005] Furthermore, through the data cleaning module, deduplication and filtering of perception business data specifically include: data diagnosis and format conversion of data sources in different formats to complete data source organization; unifying different standards and formats to merge data sources; discovering and clearing quality problems of various data in data sources by rotating or mirroring image restoration, removing similar images, removing blurred images and cropping useless parts of images; in the cleaning process, similar images above the similarity threshold are removed, the higher the threshold, the higher the image similarity, and blurred images above the blur threshold are removed during the cleaning process; for the deblurring operator, the blur threshold needs to be configured, the configuration range is 0-100%, the higher the threshold, the higher the image blur; for the rotation operator, the rotation angle needs to be configured; for the cropping operator, the cropping range needs to be configured.

[0006] Furthermore, de-similarization cleaning is used to achieve image deduplication by calculating image hash values, feature comparisons such as histograms or gray-level co-occurrence matrices, identifying duplicate images using convolutional neural network models, and extracting image features and mapping them into a space to identify duplicate images. A feature vector fast search and matching algorithm is used to obtain similar images, and each sample is taken as a query object in turn. The k-means clustering method is used to determine the top N subspaces greater than the search threshold, and the top K samples greater than the search threshold are matched in the screened subspaces. The system divides images according to similarity levels, and supports screening and cleaning images by setting similarity levels.

[0007] Furthermore, in addition to the system's built-in cleaning operators, the data cleaning module also provides the ability to integrate custom cleaning operators. The system specifies a custom operator access interface specification in the form of RESTful, which can support the integration of various user-defined intelligent models, expand the system's cleaning capabilities, and improve the system's applicability. Before integration, it is first necessary to obtain a cleaning model based on deep learning framework training, create an inference program, install the inference program's related dependencies, and encapsulate the inference program as a service according to the interface specification, complete the service image production, add a custom cleaning service to the system configuration page, fill in the service description information, service address and port number, and set the cleaning threshold. After registration is complete, the service can be tested in the system interface.

[0008] Furthermore, the image enhancement module can perform image data processing operations to enhance data quality. By building a wealth of data processing operators, it comprehensively covers the image data processing demands of users in deep learning scenarios, orchestrates scheduling services, and cooperates with visual interactive components. Users can flexibly combine various data processing operators. The image enhancement module implements the adversarial sample generation function based on the SMA attack algorithm, DeepFool attack algorithm, BIM attack algorithm, C&W attack algorithm, and PGD attack algorithm, and provides a standard service interface. It can also support the access and integration of new adversarial sample generation algorithms.

[0009] Furthermore, any attack algorithm is composed of an access layer, a scheduling layer, an operator layer, and a computing layer. The access layer provides management interfaces such as initiating, querying, and stopping image enhancement tasks. The scheduling layer manages and schedules image enhancement tasks based on K8S. The operator layer defines the enhancement operator framework based on a unified interface and provides a full enhancement operator pool. The computing layer is used to schedule and execute the calculation of specific enhancement operators and output the enhanced results.

[0010] Furthermore, through the data labeling module, automatic sample labeling of business domain data specifically includes: screening the data to be labeled from the enhanced database; using the existing labeling basic model library to perform intelligent labeling of data; when performing intelligent labeling, using threshold discrimination to screen difficult examples, and performing manual confirmation and re-labeling, adding the manually processed difficult examples to the model training again, using incremental learning to continuously improve the accuracy of the labeling model; after iterative rounds, finally generating a labeling model with better quality, and incorporating it into the labeling model warehouse for unified management.

[0011] Furthermore, the quality evaluation module provides image quality evaluation methods in both subjective and objective modes. The subjective evaluation is based on coverage analysis. Industry experts observe a series of charts based on their experience, conduct quality assessments on them, and give corresponding scores. The evaluation results are calculated and measured by taking the average score or percentage to measure the quality of the evaluated data set. The coverage analysis is mainly for labeled data sets. According to the annotation information, the target type statistics, environment type statistics, annotation box aspect ratio distribution, annotation box area distribution, stacking degree distribution, etc. are calculated in real time, and displayed uniformly in appropriate charts. It can be used for data information content quality inspection and review, to discover potential problems and improve the credibility of data annotation results. The objective evaluation uses a weighted approach to obtain a comprehensive image quality evaluation in view of the signal-to-noise ratio, uniformity and degree of blur.

[0012] Furthermore, the image signal-to-noise ratio SNR can be calculated based on Calculate and obtain, the uniformity coefficient U can be obtained according to Calculate and obtain the clarity p(e i ) can be based on Calculation acquisition, image quality assessment IQA can be based on IQA = p(e i )+λU+βSNR is calculated, where K is the noise level of the image, E is the comprehensive matrix, I0 is the overall average pixel value of the image, and w(e i ) is the transition width of the edge point, w(c) is the local estimated value of the image blur threshold, β is the sample parameter, λ is the first weighted value, and β is the second weighted value.

[0013] According to another aspect of the present invention, a quality processing and evaluation system based on perceived business data is provided. The quality processing and evaluation system based on perceived business data uses the quality processing and evaluation method based on perceived business data as described above to perform quality processing and evaluation. The quality processing and evaluation system based on perceived business data includes: a data entry module, the data entry module is used to enter the collected perceived business data; a data cleaning module, the data cleaning module is used to deduplicate and filter the perceived business data; a data enhancement module, the data enhancement module is used to increase the quantity of the perceived business data; a data labeling module, the data labeling module is used to automatically label samples of business domain data; a data quality assessment module, the data quality assessment module is used to set the evaluation weight ratio of each dimension, calculate the comprehensive data quality score of the perceived business data based on data coverage, data uniformity, data signal-to-noise ratio and data clarity, and output the results in a visualized manner, thereby completing the quality processing and evaluation based on the perceived business data.

[0014] The technical solution of the present invention is applied to provide a quality processing and evaluation method based on perception business data. The present invention can solve the problem of multi-dimensional label classification management of data by designing the perception metadata information specification, and realize the on-demand screening of data from the label dimension. The system also integrates components such as cleaning, enhancement, and annotation, realizes the process and toolization of data processing, solves the phenomenon that the traditional data processing method is single, inefficient, and backward. At the same time, a quality processing and evaluation method based on data coverage, data uniformity, data signal-to-noise ratio, and data clarity is established, and the quantitative rating of the quality of perception business data is realized, which provides a new way for experts in the perception field to better select high-quality image data. The present invention is of great significance to the data processing and analysis evaluation in the perception field, and significantly reduces the processing cost of perception business data. The present invention can make up for the shortcomings of insufficient frequency and high cost of cooperative target data collection, and limited means of non-cooperative target sample data collection. By integrating the quality processing and evaluation system designed by the present invention, the quality improvement and objective evaluation of all aspects of data in the perception business field can be realized, and the quality of sample data generation is reliable and the processing method is convenient, thereby accelerating the application of artificial intelligence technology in typical scenarios. The present invention covers multiple types of data processing methods, meets the use requirements of multiple types of perception sensors and various intelligent application mode scenarios, provides high-confidence, high-richness, and massive sample data, and supports the improvement of intelligent perception capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The included drawings are used to provide a further understanding of the embodiments of the present invention, which constitute a part of the specification, are used to illustrate the embodiments of the present invention, and together with the text description, explain the principles of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 The overall architecture diagram of the quality processing and evaluation system based on perceived service data provided according to a specific embodiment of the present invention is shown;

[0017] Figure 2 An example diagram of data annotation of perception service metadata provided according to a specific embodiment of the present invention is shown;

[0018] Figure 3 A schematic diagram of a cleaning module according to a specific embodiment of the present invention is shown;

[0019] Figure 4 A schematic diagram of an enhancement module provided according to a specific embodiment of the present invention is shown;

[0020] Figure 5A schematic diagram of a labeling module provided according to a specific embodiment of the present invention is shown;

[0021] Figure 6 A schematic diagram of a quality assessment module provided according to a specific embodiment of the present invention is shown;

[0022] Figure 7 A flow chart of a data quality processing and evaluation system provided according to a specific embodiment of the present invention is shown. DETAILED DESCRIPTION

[0023] like Figures 1 to 7 As shown, according to a specific embodiment of the present invention, a quality processing and evaluation method based on perceived business data is provided, and the quality processing and evaluation method based on perceived business data includes: inputting the collected perceived business data into the system through a data entry module; deduplicating and filtering the perceived business data through a data cleaning module; amplifying the quantity of the perceived business data through a data enhancement module; automatically annotating samples of business domain data through a data annotation module; setting the evaluation weight ratio of each dimension through a data quality assessment module, calculating the comprehensive data quality score of the perceived business data based on data coverage, data uniformity, data signal-to-noise ratio and data clarity, and visually outputting the results to complete the quality processing and evaluation based on the perceived business data.

[0024] By applying this configuration method, a quality processing and evaluation method based on perception business data is provided. The present invention can solve the problem of multi-dimensional label classification management of data by designing the perception metadata information specification, and realize the on-demand screening of data from the label dimension. The system also integrates components such as cleaning, enhancement, and annotation, realizes the process and toolization of data processing, solves the phenomenon that the traditional data processing method is single, inefficient, and backward. At the same time, a quality processing and evaluation method based on data coverage, data uniformity, data signal-to-noise ratio, and data clarity is established, and the quantitative rating of the quality of perception business data is realized, which provides a new way for experts in the perception field to better select high-quality image data. The present invention is of great significance to the data processing and analysis evaluation in the perception field, and significantly reduces the processing cost of perception business data. The present invention can make up for the shortcomings of insufficient frequency and high cost of cooperative target data collection, and limited means of non-cooperative target sample data collection. By integrating the quality processing and evaluation system designed by the present invention, the quality improvement and objective evaluation of all aspects of data in the perception business field can be realized, and the quality of sample data generation is reliable and the processing method is convenient, thereby accelerating the application of artificial intelligence technology in typical scenarios. The present invention covers multiple types of data processing methods, meets the use requirements of multiple types of perception sensors and various intelligent application mode scenarios, provides high-confidence, high-richness, and massive sample data, and supports the improvement of intelligent perception capabilities.

[0025] Specifically, the present invention provides a quality processing and evaluation method and system based on perceived business data, which can realize efficient processing and quality quantitative rating of perceived business data, changing the phenomenon that traditional data processing and evaluation methods are single, inefficient, and backward, and ensuring that the sample data processing method is convenient and the quality evaluation is reliable.

[0026] The present invention discloses a quality processing and evaluation method and system based on perceived business data, comprising the following steps: inputting the collected perceived business data into the system through a data entry module; filtering the perceived business data through a data cleaning module; amplifying the perceived business data through a data enhancement module; automatically labeling samples of business domain data through a data labeling module; setting the evaluation weight ratio of each dimension through a data quality assessment module, calculating the comprehensive data quality score of the perceived business data, and visually outputting the result. The present invention realizes the quantitative rating of the quality of perceived business data, and overcomes the problem that the quality of business data is difficult to distinguish and the judgment of the quality of data is too subjective in the field of intelligent perception.

[0027] This system is based on technologies such as K8S resource scheduling and Docker containers to help implement data services on the cloud, including data set creation, hierarchical classification management, import / export, modification, query, retrieval, etc. It also provides sample cleaning, sample enhancement, sample labeling, and quality assessment for perception business samples, and saves the processed data as a sample data set for use on other platforms. It supports on-demand customization and provides secondary development support such as function expansion and interface supplementation. It supports permission management, data meta-tag structure design, and data maintenance, and provides support for the overall management of data resources. It mainly includes five functional modules: data management, data cleaning, data enhancement, data labeling, and system management. The overall architecture design is as follows: Figure 1 shown.

[0028] (1) Cleaning module design

[0029] The data cleaning module includes version management before and after cleaning and cleaning function selection. The former is used to ensure version control before and after cleaning, and the latter is used to implement the cleaning function.

[0030] The data cleaning process is divided into several steps, including data source arrangement, data source merging, data cleaning, and forming a new version. Data source arrangement performs data diagnosis and format conversion on data sources of different formats, unifies different standards and formats to merge data sources, and discovers and removes quality problems in various data in data sources by rotating or mirroring images, removing similar images, removing blurred images, and cropping useless parts of images. During the cleaning process, similar images above the threshold are removed. The higher the threshold, the higher the image similarity. During the cleaning process, blurred images above the threshold are removed. For the deblurring operator, the fuzziness threshold needs to be configured. The configuration range is 0-100%. The higher the threshold, the higher the image fuzziness. For the rotation operator, the rotation angle needs to be configured. For the cropping operator, the cropping range needs to be configured.

[0031] Among them, similarity cleaning is used to achieve image deduplication by calculating the hash value of the image, comparing features such as histogram or gray level co-occurrence matrix, identifying duplicate images with convolutional neural network models, extracting image features and mapping them into a space to identify duplicate images. A feature vector fast search and matching algorithm is used to obtain similar images. Each sample is taken as a query object in turn, and the top N subspaces greater than the search threshold are determined by k means clustering, and the top K samples greater than the search threshold are matched in the screened subspace. The system divides the images by similarity level and supports screening and cleaning of images by setting the similarity level. Based on the deep learning model, feature vectors of massive images are extracted and a vector feature library is constructed, which greatly reduces the computational complexity of feature matching. Large-scale feature vectors are merged into multiple subspaces by clustering, and the top few most similar clusters are matched, and then matched one by one. When using the CPU, this method only takes 60 seconds to clean one million data, and ensures that there is no omission or loss in the cleaning results.

[0032] At the same time, through artificial intelligence reasoning technology, we can rationally utilize the trained and optimized models, dynamically configure the appropriate cleaning threshold, start the task of filtering out targetless images, and temporarily store the results for random sampling by subsequent reviewers.

[0033] In addition to the system's built-in cleaning operators, the system also provides the ability to integrate custom cleaning operators. The system specifies the custom operator access interface specification in the form of RESTful, which can support the integration of various user-defined intelligent models, expand the system's cleaning capabilities, and improve the system's applicability. Before integration, you first need to obtain a cleaning model based on deep learning framework training, make an inference program, install the inference program's related dependencies, and encapsulate the inference program as a service according to the interface specification, complete the service image production, add a custom cleaning service to the system configuration page, fill in the service description information, service address and port number, and set the cleaning threshold. After registration is complete, you can test the service in the system interface.

[0034] The cleaning results are finally presented to the service caller in a visual way with mixed graphics and text, with emphasis on the comparison effect before and after cleaning. Users can further confirm the cleaning results based on the effects and perform retention or deletion operations.

[0035] (3) Enhanced module design

[0036] The image enhancement module can perform image data processing operations to enhance data quality. By building a rich set of data processing operators, it fully covers the image data processing requirements of users in deep learning scenarios, arranges scheduling services, and cooperates with visual interactive components. Users can flexibly combine various data processing operators. Mainly based on image types, it provides enhancement functions such as cutting, moving, rotating, contrast, RGB distribution adjustment, brightness, clarity, random occlusion, and left-right flipping. For image classification, target detection, and other no less than 5 adversarial sample generation algorithms, it is used to generate new adversarial samples from existing data sets. Any adversarial sample generation algorithm is composed of an access layer, a scheduling layer, an operator layer, and a computing layer. The access layer provides management interfaces such as image enhancement task initiation, query, and stop. The scheduling layer manages and schedules image enhancement tasks based on K8S. The operator layer defines the enhancement operator framework based on a unified interface and provides a full-scale enhancement operator pool. Finally, the computing layer schedules and executes the calculation of specific enhancement operators and outputs the enhanced results.

[0037] Based on five classic attack algorithms (JSMA attack algorithm, DeepFool attack algorithm, BIM attack algorithm, C&W attack algorithm and PGD attack algorithm), the adversarial sample generation function is implemented, and a standard service interface is provided. It can also support the access and integration of new adversarial sample generation algorithms. JSMA (Jacobian-based Saliency MapAttack) is an attack based on the L0 norm. The algorithm modifies one pixel of the original sample at a time and monitors the impact on the output result. The saliency map is obtained by calculating the output change gradient (i.e., Jacobian matrix). Once the saliency map is calculated, the algorithm will choose to change the most effective pixel to deceive the network, and then repeat this process until the pixel changed in the adversarial sample reaches the maximum allowed value or the deception is successful.

[0038] DeepFool is an attack based on the L2 norm. This algorithm continuously adds perturbations to the sample so that the sample crosses the decision boundary of the classifier. In each iteration, a small vector is used to perturb the image. The direction of the small vector is the vertical direction of the approximate boundary obtained by linearizing the boundary of the area where the sample resides. When the perturbed image crosses the decision boundary of the classifier and changes its label, the total perturbation is the final perturbation. Fast Gradient Sign Method (FGSM) is a perturbation method that only applies to linearity. It adds slight perturbations when the gradient changes the most, resulting in misclassification.

[0039] The formula is:

[0040]

[0041] but:

[0042]

[0043] Among them, ε is a small constant, θ is the network parameter, and J(θ,x,y) is the cost function used to train the network. is an adversarial sample, x is the original input sample, δ x For disturbance, To calculate the gradient function of the input sample with respect to the gradient of the cost function, sign x is the sign function for calculating the sign of the gradient. This method only needs to calculate the gradient once and is not targeted. It generates adversarial samples quickly and effectively. The Basic Iterativa Method (BIM) is one of the many variants of FGSM. The improvement of BIM is to iterate FGSM a limited number of times on clean samples to add adversarial perturbations, thereby generating more powerful adversarial samples within the constraints.

[0044] BIM attacks are also based on L ∞ The main idea of ​​this algorithm is to iteratively use FGSM on samples and control the size of the disturbance within the preset L ∞ The specific formula is as follows:

[0045]

[0046] in, is the adversarial sample with iteration 0, X is the original input sample, For adversarial samples that iterate N+1 times, Clip x,e is a clipping function that restricts each input feature point to the e neighborhood of the input sample x, where x is the input sample and e is the neighborhood size. is an adversarial sample iterated N times,∈ is the iteration step length, is the gradient function for calculating the gradient of the input sample with respect to the cost function, J is the cost function of the training network, y true is the sample label.

[0047] The advantage of BIM over FGSM is that the iterative process can provide a more accurate description of the loss function space around the sample points, greatly improving the success rate of the attack. However, the speed at which adversarial samples are generated is much lower than that of FGSM.

[0048] C&W is an attack based on the L2 norm, which has a smaller value in adversarial samples but a larger value in normal samples, so adversarial samples can be searched by minimizing this function. The optimized function of CW can be abbreviated as:

[0049] L=D(x,x+δ)+c*f(x+δ)

[0050] Where c is the selected positive constant, D is the Euclidean distance under the selected paradigm (the distance between the sample point and the perturbed sample), L is the optimization function, δ is the random perturbation, x is the input sample, and f(x+δ) is the objective function.

[0051] PGD ​​can be seen as a multi-step iteration of FGSM. When facing a nonlinear model, only one iteration may not be completely correct. However, by iterating multiple times based on the previous algorithm, the strongest adversarial sample within the range can be found. The principle is as follows:

[0052]

[0053] Among them, Clip can limit the range of disturbance to not exceed the agreed ∈, x t ' +1 is the adversarial sample of the t+1th iteration, Π x,∈ is a projection function that projects a value into the e neighborhood of the input sample x, x t ' is the adversarial sample of the tth iteration, ∈ is the neighborhood size, α is the iteration step length, is the gradient function for calculating the gradient of the input sample with respect to the cost function, L is the cost function of the training network, sign() is the sign function for calculating the sign of the gradient, y is the sample label, and θ is the network parameter.

[0054] During the image enhancement process, the serial or parallel enhancement execution strategy can be automatically and adaptively selected according to the image type and the purpose of this enhancement. Compared with using a single algorithm, the generated sample data has stronger adversarial capabilities.

[0055] The data before and after enhancement are integrated, and the differences between the images before and after enhancement are highlighted through text lists and image previews, and the effects of image enhancement are reflected through objective quality evaluation indicators.

[0056] (4) Annotation module design

[0057] In the present invention, the data in the business field is automatically labeled with samples through the data labeling module, specifically including: screening the data to be labeled from the enhanced database; using the existing labeling basic model library to perform intelligent labeling of the data; when performing intelligent labeling, using a threshold discrimination method to screen difficult examples, and performing manual confirmation and re-labeling, adding the manually processed difficult examples to the model training again, using incremental learning to continuously improve the accuracy of the labeling model; after iterative rounds, finally generating a labeling model with better quality, and incorporating it into the labeling model warehouse for unified management.

[0058] The annotation module can be connected to the automatic annotation algorithm as needed, and provide a visual and easy-to-operate annotation interface and standard management. Annotation tasks cover classification annotation, box annotation, region annotation, point annotation, sequence annotation, entity annotation, and relationship annotation. For images in jpg, png, jpeg and other formats, a rich set of image annotation tools and annotation templates are provided to improve image annotation efficiency and annotation quality, and meet the data annotation requirements in AI application scenarios such as image classification, target recognition, and image segmentation. For video annotation, it is convenient for users to only annotate part of the video. Users can play the video, then click pause, and annotate in the pause interface. Based on the video encoding and decoding capabilities provided by the underlying hardware devices, it provides services or interfaces for efficiently decoding H.264 / H.265 bitstream format video files into images, thereby improving the processing efficiency of edge reasoning applications. For txt text content, a feature-rich annotation component is provided to meet the data annotation requirements in AI application scenarios such as text classification, sentiment analysis, and text entity-level entity relationship extraction.

[0059] Intelligent labeling uses the existing labeling basic model library and adopts real-time reasoning to realize the automatic labeling function. It is mainly divided into two modes. One is automatic labeling based on preset models, and the other is to support automatic labeling based on the trained model after training the labeling model based on a small number of labeled data. Intelligent labeling models for three typical perception images, infrared, visible light and SAR, are preset. The target detection model based on deep learning requires a large number of labeled samples for training. In actual scenarios, it is often difficult to obtain high-quality labeled samples of corresponding scale. Small sample target detection is a fusion of traditional target detection technology and small sample learning technology. It learns detection models with good generalization performance through a small number of labeled samples, reduces dependence on large-scale label data, and supports training automatic labeling models with at least 10 samples. Among them, intelligent labeling uses threshold discrimination to screen difficult examples, and performs manual confirmation and re-labeling to ensure that the labeling quality is reliable and trustworthy. The difficult examples that have been manually processed are added to the model training again, and incremental learning is used to continuously improve the accuracy of the labeling model. After iterative rounds, a good quality labeling model is finally generated and included in the labeling model warehouse for unified management.

[0060] Environment management provides management of basic images. Users can directly pull basic images from the platform, and based on the existing results of the model warehouse, quickly build various dependency libraries required for the operation of the inference service into an AI service image that complies with the OCI specification locally, and then use the image warehouse for storage and distribution. It supports iterative management, storage, and push of models locally, reducing the cost of model deployment and maintenance for R&D personnel. Combined with image version management, rapid iterative updates and historical version rollbacks of AI service deployment can be quickly implemented. The above are intelligent annotation services in data annotation, all of which exist in the form of images and need to be uniformly managed, version controlled, and updated and iterated.

[0061] (5) Quality evaluation module design

[0062] The quality evaluation module provides three functional components: image data quality inspection, annotation data quality inspection and table data quality inspection. It collects a large amount of basic indicator data and provides different types of visualization component modules such as bar charts, pie charts, and curve charts to facilitate administrators to quickly evaluate the quality of data sets.

[0063] The quality evaluation module mainly obtains and analyzes the image attributes and content information that are consistent with the evaluation indicators for the original image type data set according to the user-specified analysis indicators, including image size distribution, aspect ratio distribution, resolution distribution, color cast distribution, clarity distribution, etc. Finally, through appropriate visual charts and visualization components, the results are presented to the user in a suitable legend, such as histograms, bar charts, pie charts, line charts, distribution charts, and stacked charts, and displayed to the end user.

[0064] The purpose of comprehensive image quality evaluation is to determine the overall visual quality and information content of the image. In practical applications, it can help algorithm personnel select the best image processing algorithm, optimization method and compression algorithm, improve image rendering and display effects, and improve sample data quality. It provides image quality evaluation methods in both subjective and objective modes.

[0065] Subjective evaluation is based on coverage analysis. Industry experts observe a series of charts based on their experience, conduct quality assessments, and give corresponding scores. The evaluation results are calculated and measured by taking the average score or percentage to measure the quality of the evaluated data set. Coverage analysis is mainly aimed at annotated data sets. According to the annotation information, it calculates the target type statistics, environment type statistics, annotation box aspect ratio distribution, annotation box area distribution, stacking degree distribution, etc. in real time, and displays them uniformly with appropriate charts. It can be used to check and review the quality of data information content, discover potential problems, and improve the credibility of data annotation results.

[0066] In terms of objective evaluation, considering the signal-to-noise ratio, uniformity and blur level, this system uses a weighted approach to obtain a comprehensive image quality evaluation.

[0067] In the signal-to-noise ratio analysis, the image signal-to-noise ratio evaluation method under weighted processing is used. The Hosaka block idea is adopted, and quadtree decomposition is used to cancel the restriction of Hosaka block on image size. The size of the block reflects the main frequency components of the image in the block; then the noise of each sub-block is detected, and the signal-to-noise ratio value is calculated according to the degree of pollution of the image. The image to be evaluated is first judged whether it is a color image. If it is a color image, it can be converted to color space first, its brightness component can be extracted, and then the length and width of the image are adjusted to be less than or equal to 8 after dividing by an integer power of 2, so as to facilitate image segmentation. The idea of ​​Hosaka block is adopted, and the image is divided into sub-blocks with different high and low frequency components and different lengths and widths according to the mean square error value of the image. For an M×N grayscale image f, its mean square error MSE is:

[0068]

[0069] Where M is the number of pixels on the length of the image, N is the number of pixels on the width of the image, i is the horizontal coordinate of the image pixel value (i, j), j is the vertical coordinate of the image pixel value (i, j), and f ij is the gray value of the image at (i, j), is the mean of the image. If the mean square error exceeds a certain threshold threshold1, it means that there are many high-frequency components in the image. Then, quadtree decomposition is used to continue dividing the image into blocks, and the length and width of the image are divided equally. This process continues until the MSE of each image block is less than threshold1 or one of the length or width of the image block is less than or equal to 8. All pixel points in the divided image blocks are analyzed to obtain the distribution of noise points in the image sub-blocks G(i,j).

[0070] G(i,j)=∑(g(i,j)-threshold2)

[0071] Among them, g(i,j) is the minimum value of the gradient of each direction of the pixel in the image sub-block, threshold2 is the noise threshold, and the gradient value g(i,j) greater than the noise threshold is a noise point.

[0072] This signal-to-noise ratio detection function increases the weight λ i =log2(min(m,n)) to meet human visual perception. Using visual masking weight λ i By weighting the noise distribution G(i,j) of the image sub-blocks and taking the average value, the noise level K of the entire image can be obtained:

[0073]

[0074] Wherein, m is the length of the image sub-block, and n is the width of the image sub-block.

[0075] From this we can get the image signal-to-noise ratio SNR, or noise:

[0076]

[0077] By evaluating the uniformity of electronic image pixel values ​​as one of the criteria for image quality evaluation, the overall average pixel value I0 of the image sample to be evaluated is obtained. The calculation method of I0 is as follows.

[0078]

[0079] Among them, A is the number of pixels in each row of the image, B is the number of pixels in each column of the image; p is the horizontal coordinate of each pixel in the image, q is the vertical coordinate of each pixel in the image, and I(p,q) is the pixel value of each pixel in the image.

[0080] Then the image to be evaluated is divided into M'×N' rectangular areas of equal area (i.e., M' rows and N' columns of image sub-blocks of equal area) according to the number of its pixels, and the regional average pixel value I of each rectangular area is calculated. ij and the mean square error of regional pixel values ​​S ij , Iij , S ij The resulting method is shown below.

[0081]

[0082] Where a is the number of pixels in each row of each rectangular area; b is the number of pixels in each column of each rectangular area; (p, q) is the coordinate value of each pixel in the image; I(p, q) is the pixel value of each pixel in the image. ij is the average pixel value of the rectangular area in the i-th row and j-th column.

[0083] Then, according to the average pixel value and the mean square error of the regional pixel value, the regional average pixel value matrix I and the regional pixel value mean square error matrix S of the image are obtained. The pixel value upper limit matrix C and the pixel value lower limit matrix D are calculated by the average pixel value matrix and the mean square error matrix. The pixel value upper limit matrix C and the pixel value lower limit matrix D are shown as follows.

[0084]

[0085] Among them I ij +S ij Indicates the upper limit of the pixel value in each rectangular area. ij -S ij Indicates the lower limit of the pixel value in each rectangular area.

[0086] According to the pixel value upper limit matrix C and the pixel value lower limit matrix D, they are combined into a comprehensive matrix E, E = [C, D]

[0087]

[0088] Finally, the uniformity coefficient U is obtained, which is calculated as follows:

[0089]

[0090] Wherein, max(E) is the maximum element in the matrix E, min(E) is the minimum element in the matrix E, and I0 is the overall average pixel value of the image. max(E)-min(E) represents the maximum average fluctuation value of the pixel values ​​between the rectangular regions in the image. (max(E)-min(E)) / I0 represents the ratio of the maximum average fluctuation value of the pixel values ​​between the rectangular regions in the image to the overall average pixel value of the image.

[0091] The uniformity of the image can be evaluated according to the uniformity coefficient U. If the uniformity coefficient U is closer to a value of 1, it is considered that the uniformity of the image is better. If the uniformity coefficient U is closer to a value of 0, it is considered that the uniformity of the image is worse.

[0092] At the same time, the image clarity detection algorithm of edge width is integrated to evaluate the clarity level of the image. By establishing a clarity detection model to evaluate the clarity of the image, as a key parameter in the image quality evaluation system, this software is used to judge the clarity p(e i ) is shown below.

[0093]

[0094] Among them, w(e i ) is the transition width of the edge point, w(c) is the local estimate of the image blur threshold, β is the sample parameter, and the value of parameter β is obtained by fitting a batch of samples with different contrasts using the least squares method.

[0095] When measuring its clarity, p(e i ) is closer to 0, the clearer the image is; i ) is closer to 1, the blurrier the image.

[0096] This tool implements the quality assessment function by combining the signal-to-noise ratio, uniformity and clarity weighting to obtain the image quality assessment (IQA):

[0097] IQA=p(e i )+λU+βSNR

[0098] Wherein, λ is the first weighted value, and β is the second weighted value.

[0099] In order to obtain the specific method based on the specific λ and β values, the IQA evaluation must be compared with the subjective evaluation. Since they are all RGB three-color images, only their brightness values ​​are considered here. The commonly used logistic method is used to fit the IQA and DMOS (subjective evaluation value DMOS (Differential mean opinion scores)) evaluated. After a large number of experiments, it was found that λ=105.2, β=35.7 is the most reasonable. The picture quality is divided into 5 levels through scoring. The lower the IQA value, the better the picture quality.

[0100]

[0101] According to another aspect of the present invention, a quality processing and evaluation system based on perceived business data is provided. The quality processing and evaluation system based on perceived business data uses the quality processing and evaluation method based on perceived business data as described above to perform quality processing and evaluation. The quality processing and evaluation system based on perceived business data includes a data entry module, a data cleaning module, a data enhancement module, a data annotation module and a data quality assessment module. The data entry module is used to enter the collected perceived business data, the data cleaning module is used to deduplicate and filter the perceived business data, the data enhancement module is used to increase the quantity of the perceived business data, the data annotation module is used to automatically annotate samples of business domain data, and the data quality assessment module is used to set the evaluation weight ratio of each dimension, calculate the comprehensive data quality score of the perceived business data based on data coverage, data uniformity, data signal-to-noise ratio and data clarity, and output the results in a visualized manner to complete the quality processing and evaluation based on the perceived business data.

[0102] By applying this configuration method, a quality processing and evaluation system based on perception business data is provided. The present invention can solve the problem of multi-dimensional label classification management of data by designing the perception metadata information specification, and realize the on-demand screening of data from the label dimension. The system also integrates components such as cleaning, enhancement, and annotation, realizes the process and toolization of data processing, solves the phenomenon that the traditional data processing method is single, inefficient, and backward. At the same time, a quality processing and evaluation method based on data coverage, data uniformity, data signal-to-noise ratio, and data clarity is established, and the quantitative rating of the quality of perception business data is realized, which provides a new way for experts in the perception field to better select high-quality image data. The present invention is of great significance to the data processing and analysis evaluation in the perception field, and significantly reduces the processing cost of perception business data. The present invention can make up for the shortcomings of insufficient frequency and high cost of cooperative target data collection, and limited means of non-cooperative target sample data collection. By integrating the quality processing and evaluation system designed by the present invention, the quality improvement and objective evaluation of all aspects of data in the perception business field can be realized, and the quality of sample data generation is guaranteed to be reliable and the processing method is convenient, thereby accelerating the application of artificial intelligence technology in typical scenarios. The present invention covers multiple types of data processing methods, meets the use requirements of multiple types of perception sensors and various intelligent application mode scenarios, provides high-confidence, high-richness, and massive sample data, and supports the improvement of intelligent perception capabilities.

[0103] In order to further understand the present invention, the following Figures 1 to 7 The quality processing evaluation method based on perceived service data provided by the present invention is described in detail.

[0104] like Figures 1 to 7As shown, according to a specific embodiment of the present invention, a quality processing and evaluation method based on perception business data is provided, through which data quality is graded, screened and processed, and data quality is quantitatively evaluated from multiple dimensions. This method of perception business data quality processing and evaluation provides a new technical means, and can also help experts in the field to better select high-quality perception business data, indirectly improving the economic benefits of the field of intelligent technology. The quality processing and evaluation method based on perception business data provided by the present invention has the following innovations.

[0105] (1) Metadata information specification based on perceived business data

[0106] A metadata annotation specification that can be used for data in specific perception business fields is proposed to solve the problem of multi-dimensional label classification management of data and realize on-demand screening from the label dimension.

[0107] (2) Cleaning operator integration technology based on deep learning similar image retrieval and filtering

[0108] The custom cleaning operator integration function was invented to expand the system cleaning capability. At the same time, the cleaning operator design adopts similar image retrieval technology, extracts feature vectors of massive images based on deep learning models and builds a vector feature library, which greatly reduces the computational complexity of feature matching. Large-scale feature vectors are clustered into multiple subspaces, and the most similar first few clusters are matched, and then matched one by one. When using the CPU, this method can control the time consumption of cleaning one million perception business data to seconds, and ensures that there is no omission or loss in the cleaning results.

[0109] (3) Enhanced operator serial parallel fusion technology based on adversarial sample generation

[0110] By building a rich set of data enhancement operators and orchestration and scheduling services, combined with visual interactive components, users can flexibly combine various data enhancement operators, including conventional geometric transformations, color transformation algorithms, and a variety of adversarial sample generation algorithms, and provide standard service interfaces to complete the access and integration of enhancement algorithms at any time. At the same time, it can automatically and adaptively select serial or parallel enhancement execution strategies based on image type and enhancement purpose, so that the generated sample data has stronger adversarial capabilities compared to using a single algorithm. The data before and after enhancement are integrated, and the difference between the images before and after enhancement is highlighted through text lists and image previews, which intuitively reflects the effect of image enhancement and solves the technical problem that data can only be enhanced using a single conventional enhancement method with low efficiency.

[0111] (4) Agile access and iteration technology based on automatic data annotation model

[0112] The present invention incorporates small sample technology, active learning technology, and incremental learning technology, selects representative data from the unlabeled sample pool to train the annotation model, and then combines AI reasoning technology to automatically annotate the unlabeled samples and screen difficult samples for manual confirmation. This technology can realize the training of the annotation model with a small number of samples, effectively improve the accuracy of difficult example annotation, solve the practical problems of large data annotation volume and low annotation efficiency in the perception business field, and realize the automatic annotation of data in this field. At the same time, this technology uses docker image packaging technology to package the executable program of the annotation model and the required minimum operating environment and dependent library programs into an executable image program. The annotation model image generated in this way is convenient for version maintenance and management on the one hand, and can realize the cross-platform integrated application of the annotation model on the other hand, which greatly expands the scenario and scope of model application, ensures the stable operation of the overall system, and quickly realizes the rapid iterative update and historical version rollback of AI automatic annotation service deployment.

[0113] (5) Comprehensive quality assessment method

[0114] The present invention designs a quality assessment method and provides a data quality inspection function module that can quickly acquire and parse information such as data attributes and content related to the evaluation indicators. It establishes a comprehensive weighted quality evaluation method based on data coverage, data uniformity, data signal-to-noise ratio and data clarity, and provides different types of visualization components such as charts to facilitate users to quickly evaluate the quality of data sets.

[0115] The data quality processing and evaluation system integrates various tool components, streamlines the data processing method, and toolizes the quality evaluation method. Below, we use visible light image data as an example to complete the data quality processing and evaluation. The system components used in this case include: perception business data entry component, data cleaning component, data enhancement component, data labeling component, and data quality evaluation component.

[0116] Step 1: Input of perception business data. Perception business data includes data ontology, data attribute information, and data annotation.

[0117] The data ontology mainly refers to the data collected by sensors such as visible light, infrared, and SAR in the field of perception. This case uses visible light image data. When entering business data, you can upload it by selecting a folder and a compressed package. During the upload process, you can customize the data set-related attribute information according to business requirements, such as the data set type (visible light image in this example), data set source, and other information. After that, select an existing data annotation file, enter the label name and label color, and import the annotation file after the data set is uploaded. The annotation file supports standard formats such as VOC, COCO2017, and YOLO, as well as the metadata specification format specified by this system.

[0118] Step 2: Data cleaning.

[0119] (1) Cleaning operator configuration

[0120] Select the dataset and dataset version to be cleaned, and determine the cleaning operator. Desimilarization, deblurring, mirroring, rotation, and cropping are built-in cleaning operators. At the same time, configure the cleaning threshold, and the configuration range is 0-100%. The filter non-target image cleaning model is a custom service access operator. When registering the cleaning model, you need to configure the service name, service URL, filtered target type, and confidence optional range (usually configured to 0-100%).

[0121] (2) Cleaning task creation

[0122] After all configurations are completed, submit the cleaning execution task and wait for the cleaning task to be successfully executed. After the cleaning is completed, manual confirmation can be performed to determine whether a new version of the data set is generated.

[0123] Step 3: Data Augmentation

[0124] (1) Enhanced operator configuration

[0125] Configure the enhancement operator, select the dataset and dataset version to be enhanced, select the operator to be enhanced, and the execution strategy for enhancement. The strategy selects the serial or parallel mode of the enhancement operator.

[0126] (2) Enhanced task creation

[0127] After the configuration is completed, submit the enhancement task and wait for the execution result. After the enhancement task is completed, each original image data will generate a corresponding enhanced image, and the comparison between the original image and the enhanced image will be visualized, and manual confirmation will be made whether to generate the next version of the data set.

[0128] Step 4: Data Annotation

[0129] (1) Annotation model configuration integration

[0130] When integrating the automatic annotation service, you can distinguish between fully automatic annotation and semi-automatic annotation by configuring whether to support model iteration. In the fully automatic annotation mode, you need to submit a list of annotation categories supported by the model.

[0131] (2) Creating a labeling task

[0132] Add a new dataset and determine the automatic annotation template selected for dataset annotation. In semi-automatic mode, you need to provide at least 10 annotations for each category of images. In fully automatic annotation mode, you can start the annotation task after selecting the image to be annotated. During the annotation process, you can use GPU computing resources to accelerate the annotation model.

[0133] (3) Difficult Case Screening

[0134] In the semi-automatic annotation mode, after the annotation task is completed, the annotated images need to be manually reviewed and the difficult examples need to be annotated again. After the difficult examples are annotated, the second round of training is started again and the results are waited for; this cycle is repeated until all the data are annotated.

[0135] (4) Labeling review

[0136] The annotation results can be reviewed one by one, and the accuracy of the annotation can be manually corrected and adjusted. It can support the selection of elements to be exported, such as data sets, annotation files, and category files, and download the selected resources.

[0137] Step 5: Data quality assessment: Configure quality inspection factors and visualize them.

[0138] (1) Creation of quality assessment tasks.

[0139] Select the data set, data set version, quality inspection data information, etc. Select the fields and quality inspection factors to be analyzed, and submit the quality inspection task.

[0140] (2) Analysis of coverage, uniformity, clarity, and signal-to-noise ratio.

[0141] Based on the selected data above, select the dimensions to be analyzed, including coverage, uniformity, clarity, and signal-to-noise ratio, start the quality analysis program, obtain the original data information of each image, and comprehensively obtain the IQA value.

[0142] (3) Calculation of comprehensive weighted score of data quality

[0143] Using the IQA value obtained in the previous process, the corresponding images are divided into corresponding levels according to the image quality scoring criteria, forming image sets at different levels and outputting a quality inspection report.

[0144] In summary, the present invention provides a quality processing and evaluation method based on perception business data. By designing the perception metadata information specification, the present invention can rely on the system to solve the problem of multi-dimensional label classification management of data and realize on-demand screening of data from the label dimension. The system also integrates components such as cleaning, enhancement, and annotation, realizes the process and toolization of data processing, solves the phenomenon that the traditional data processing method is single, inefficient, and backward. At the same time, a quality processing and evaluation method based on data coverage, data uniformity, data signal-to-noise ratio, and data clarity is established, and the quantitative rating of the quality of perception business data is realized, which provides a new way for experts in the perception field to better select high-quality image data. The present invention is of great significance to the data processing and analysis evaluation in the perception field, and significantly reduces the processing cost of perception business data. The present invention can make up for the shortcomings of insufficient frequency and high cost of cooperative target data collection, and limited means of non-cooperative target sample data collection. By integrating the quality processing and evaluation system designed by the present invention, the quality improvement and objective evaluation of all aspects of data in the perception business field can be realized, and the quality of sample data generation is reliable and the processing method is convenient, thereby accelerating the application of artificial intelligence technology in typical scenarios. The present invention covers multiple types of data processing methods, meets the use requirements of multiple types of perception sensors and various intelligent application mode scenarios, provides high-confidence, high-richness, and massive sample data, and supports the improvement of intelligent perception capabilities.

[0145] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0146] In addition, it should be noted that the use of terms such as "first" and "second" to limit components is only for the convenience of distinguishing the corresponding components. If not otherwise stated, the above terms have no special meaning and therefore cannot be understood as limiting the scope of protection of the present invention.

[0147] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A quality processing evaluation method based on perceived service data, characterized in that: The quality processing evaluation method based on perceived service data includes: Input the collected perception business data into the system through the data entry module; Through the data cleaning module, the perception business data is deduplicated and filtered; Through the data enhancement module, the quantity of perception business data is increased; Through the data annotation module, automatic sample annotation of business domain data is performed; Through the data quality assessment module, the evaluation weight ratio of each dimension is set, and the comprehensive data quality score of the perception business data is calculated based on data coverage, data uniformity, data signal-to-noise ratio and data clarity. The results are then visualized and output to complete the quality processing evaluation based on the perception business data.

2. The quality processing evaluation method based on perceived service data according to claim 1 is characterized in that: Through the data cleaning module, deduplication and filtering of perception business data include: Perform data diagnosis and format conversion on data sources of different formats to complete data source organization; Merge data sources by unifying different standards and formats; By rotating or mirroring images, restoring, removing similar images, removing blurred images, and cropping useless parts of images, quality problems in various data in the data source can be discovered and cleared; in the cleaning process, similar images above the similarity threshold are removed. The higher the threshold, the higher the image similarity. In the cleaning process, blurred images above the blur threshold are removed; for the deblurring operator, the blur threshold needs to be configured. The configuration range is 0-100%. The higher the threshold, the higher the image blur; for the rotation operator, the rotation angle needs to be configured; for the cropping operator, the cropping range needs to be configured.

3. The quality processing evaluation method based on perceived service data according to claim 2 is characterized in that: Similarity cleaning is used to achieve image deduplication by calculating the hash value of the image, comparing features such as histogram or gray-level co-occurrence matrix, identifying duplicate images using a convolutional neural network model, and extracting image features and mapping them into a space to identify duplicate images. A feature vector fast search and matching algorithm is used to obtain similar images. Each sample is taken as a query object in turn, and the top N subspaces greater than the search threshold are determined through k-means clustering. The top K samples greater than the search threshold are matched in the screened subspaces. The system divides images according to similarity levels and supports screening and cleaning images by setting similarity levels.

4. The quality processing evaluation method based on perceived service data according to claim 3 is characterized in that: In addition to the system's built-in cleaning operators, the data cleaning module also provides the ability to integrate custom cleaning operators. The system specifies the custom operator access interface specification in the form of RESTful, which can support the integration of various user-defined intelligent models, expand the system's cleaning capabilities, and improve the system's applicability. Before integration, it is necessary to first obtain a cleaning model based on deep learning framework training, make an inference program, install the inference program's related dependencies, and encapsulate the inference program as a service according to the interface specification, complete the service image production, add a custom cleaning service to the system configuration page, fill in the service description information, service address and port number, and set the cleaning threshold; After registration is complete, you can test the service in the system interface.

5. The quality processing evaluation method based on perceived service data according to claim 4 is characterized in that: The image enhancement module can perform image data processing operations to enhance data quality. By building a rich set of data processing operators, it comprehensively covers the image data processing demands of users in deep learning scenarios, orchestrates scheduling services, and cooperates with visual interactive components. Users can flexibly combine various data processing operators. The image enhancement module implements the adversarial sample generation function based on the SMA attack algorithm, DeepFool attack algorithm, BIM attack algorithm, C&W attack algorithm, and PGD attack algorithm, and provides a standard service interface. It can also support the access and integration of new adversarial sample generation algorithms.

6. The quality processing evaluation method based on perceived service data according to claim 5 is characterized in that: Any of the attack algorithms is composed of an access layer, a scheduling layer, an operator layer and a computing layer. The access layer provides management interfaces such as initiating, querying and stopping image enhancement tasks. The scheduling layer manages and schedules image enhancement tasks based on K8S. The operator layer defines an enhancement operator framework based on a unified interface and provides a full enhancement operator pool. The computing layer is used to schedule and execute the calculation of specific enhancement operators and output the enhanced results.

7. The quality processing evaluation method based on perceived service data according to claim 6 is characterized in that: Through the data annotation module, automatic sample annotation of business domain data includes: Filter the data that needs to be labeled from the enhanced database; Use the existing annotation basic model library to perform intelligent data annotation; When performing intelligent labeling, we use threshold discrimination to screen difficult examples, and then manually confirm and re-label them. We add the manually processed difficult examples back into the model training, and use incremental learning to continuously improve the accuracy of the labeling model. After multiple rounds of iterations, a high-quality annotation model is finally generated and included in the annotation model warehouse for unified management.

8. The quality processing evaluation method based on perceived service data according to claim 7 is characterized in that: The quality evaluation module provides image quality evaluation methods in two modes: subjective and objective. The subjective evaluation is based on coverage analysis. Industry experts observe a series of charts based on their experience, conduct quality evaluation on them, and give corresponding scores. The evaluation results are calculated and measured by taking the average score or percentage to measure the quality of the evaluated data set. The coverage analysis is mainly aimed at the labeled data set. According to the annotation information, the target type statistics, environment type statistics, annotation box aspect ratio distribution, annotation box area distribution, stacking degree distribution, etc. are calculated in real time, and displayed uniformly in appropriate charts, which can be used for data information content quality inspection and review, to discover potential problems and improve the credibility of data annotation results. The objective evaluation uses a weighted approach to obtain a comprehensive image quality evaluation in view of the signal-to-noise ratio, uniformity and blur degree.

9. The quality processing evaluation method based on perceived service data according to claim 8, characterized in that: The image signal-to-noise ratio SNR can be based on Calculate and obtain, the uniformity coefficient U can be obtained according to Calculate and obtain the clarity p(e i ) can be based on Calculation is performed, and the image quality assessment IQA can be obtained according to IQA=p(e i )+λU+βSNR is calculated, where K is the noise level of the image, E is the comprehensive matrix, I0 is the overall average pixel value of the image, and w(e i ) is the transition width of the edge point, w(c) is the local estimated value of the image blur threshold, β is the sample parameter, λ is the first weighted value, and β is the second weighted value.

10. A quality processing evaluation system based on perceived business data, characterized in that: The quality processing evaluation system based on perceived service data uses the quality processing evaluation method based on perceived service data according to any one of claims 1 to 9 to perform quality processing evaluation, and the quality processing evaluation system based on perceived service data includes: A data entry module, wherein the data entry module is used to enter the collected perception service data; A data cleaning module, wherein the data cleaning module is used to perform deduplication and filtering on the sensing service data; A data enhancement module, wherein the data enhancement module is used to increase the quantity of sensing service data; A data annotation module, which is used to automatically annotate samples of business domain data; The data quality assessment module is used to set the evaluation weight ratio of each dimension, calculate the comprehensive data quality score of the perception business data based on data coverage, data uniformity, data signal-to-noise ratio and data clarity, and output the results in a visual manner to complete the quality processing evaluation based on the perception business data.

Images