Visual detection optimization method and system based on data set classification

Abstract

The invention discloses a visual detection optimization method and system based on data set classification. The method comprises the steps of introducing a scene classification label and carrying out standardization processing on scene features in a data set labeling and preprocessing stage, and carrying out scene classification multi-task learning in a model selection and training stage, so that a model dynamically adjusts feature learning in different scenes, and the multi-scene adaptive capacity is improved. And in the model evaluation and verification stage, scene-based hierarchical evaluation is introduced and corresponding optimization is carried out. In the model deployment and reasoning stage, a dynamic scene is introduced to adapt to reasoning, and the reasoning path of the model is dynamically adjusted to adapt to the characteristics of the scene, so that the reasoning accuracy is improved. In the later optimization, scene feature clustering and transfer learning are used, and the multi-scene performance of the model is optimized. According to the method, under the condition that the model is not jointly trained through multiple data sets, the multi-scene adaptive capacity of the visual detection model is improved through data set classification.

Description

Technical Field

[0001] The present invention relates to the field of visual inspection technology, and in particular to a visual inspection optimization method and system based on data set classification. Background Art

[0002] Existing visual inspection technologies are usually implemented using deep learning models, which are first trained with a large number of annotated data sets to learn the features of different objects or regions in the image. After preprocessing and data enhancement, the data set is input into a pre-trained or custom model for training, such as YOLO and FasterR-CNN. After training, the model can be deployed in a real environment to detect new images through inference.

[0003] Existing visual inspection technology has the following disadvantages:

[0004] 1. Visual detection tasks rely on diverse datasets, and existing datasets are generally classified according to data types. Indoor scenes such as SUN RGB-D and NYU DepthV2 provide rich RGB and depth data for 3D reconstruction and scene parsing, while outdoor datasets such as Cityscapes and KITTI focus on target detection tasks in autonomous driving and urban traffic environments. DeepGlobe and Spacenet in the remote sensing field support surface target detection in satellite images, and medical imaging datasets such as BraTS and ISIC are used for tumor and lesion detection. In addition, autonomous driving datasets such as nuScenes and Waymo OpenDataset provide high-precision perception data to promote the development of autonomous driving technology. Agricultural and drone datasets such as CropDeep and DeepWeeds help realize crop and weed detection, and 3D reconstruction tasks rely on 3D datasets such as ShapeNet and ModelNet40. However, these diverse datasets only support the training and evaluation of visual detection models, and their utilization rate is not high.

[0005] 2. Existing visual detection models, in some complex scenarios, such as when it is necessary to cover different scenarios or increase the amount of data, use multiple data sets to jointly train the model. In order for the model to handle the labels and distributions of different data sets, it is usually necessary to perform consistent preprocessing and label standardization on different data sets, and the training complexity and resource consumption are high. Summary of the invention

[0006] Based on this, an embodiment of the present application provides a visual detection optimization method and system based on data set classification, which utilizes data set classification to improve the multi-scenario adaptability of the visual detection model without jointly training the model through multiple data sets.

[0007] In a first aspect, a visual inspection optimization method based on data set classification is provided, the method comprising:

[0008] Assigning a corresponding scene label to each collected image sample, and performing a preprocessing operation on the data according to the assigned scene label; wherein the preprocessing operation includes a standardization process, and the standardization process at least includes adjusting the illumination, color, and geometric features of the image;

[0009] A hierarchical training strategy is used to introduce data from different scenarios for model training, and parameters are dynamically adjusted according to the data scenario currently being processed;

[0010] Evaluate models hierarchically based on scene classification labels;

[0011] Deploy the model after hierarchical evaluation and dynamically adjust the model inference strategy based on the scene label of the input image;

[0012] The model is optimized through scene feature clustering and transfer learning, and the optimized model is used for visual detection optimization.

[0013] Optionally, a corresponding scene label is assigned to each collected image sample, and a preprocessing operation is performed on the data according to the assigned scene label, including:

[0014] Assign an additional scene classification label to each image. For each image data x i , corresponding to the target detection label y i External and scene labels i , scene labels are used to distinguish different scene features, through the formula:

[0015]

[0016] Determine the overall loss function of the model, where N is the number of training samples and L is detection is the loss function of target detection, L secene is the scene classification loss function, and λ is the adjustment coefficient used to balance the losses of target detection and scene classification.

[0017] Optionally, the standardization process further includes a normalization operation, wherein:

[0018] For each scene label s i , applying different normalization methods g (si) :

[0019] Among them, g si is the preprocessing operation of scene characteristics, x i is the image data, x' i is the normalized image data.

[0020] Optionally, a hierarchical training strategy is used to introduce different scenario data for model training, and parameters are dynamically adjusted according to the currently processed data scenario, including:

[0021] For each training sample (x i ,y i ,s i ) The optimized joint loss function L multi-task :

[0022] L multi-task =L detection (x i ,y i )+αL scene (x i ,s i )

[0023] Among them, L detection is the loss function of target detection; L secene is the scene classification loss function; α is the coefficient that controls the balance between the two tasks.

[0024] Optionally, the model is evaluated hierarchically based on scene classification labels, including:

[0025] Classify tags by scene i Evaluate the performance of the model in each scenario separately, where the evaluation metric Metric si Can be calculated by scene category:

[0026]

[0027] in, is the predicted output of the model, y i is the target detection label, D si It's scenes i The test set;

[0028] For multi-scenario adaptability evaluation, a new evaluation index is introduced.

[0029]

[0030] Get the multi-scenario evaluation results, where w j It's scenes j The weight of k is the number of scene categories.

[0031] Optionally, the hierarchically evaluated model is deployed and the model inference strategy is dynamically adjusted based on the scene label of the input image, including:

[0032] For an input image x i , first by predicting the scene label s i, and then adjust the model inference path based on the label:

[0033] z i =f(x i )+g(s i )

[0034] Among them, f(x i ) is the image feature extracted by the model, g(s i ) is the scene code, z i It is a feature that combines scene information.

[0035] Optionally, the model is optimized by scene feature clustering and transfer learning, and the optimized model is used to optimize visual detection, including:

[0036] Perform cluster analysis on scene features and optimize the model. The feature clustering formula is:

[0037]

[0038] Among them, μ j is the cluster center, f s (x i ) is the scene feature. After clustering, transfer learning is performed on the data of each subclass to optimize the adaptability of the model in different scenarios.

[0039] In a second aspect, a visual inspection optimization system based on data set classification is provided, the system comprising:

[0040] The data set annotation and preprocessing module is used to assign a corresponding scene label to each collected image sample and perform a preprocessing operation on the data according to the assigned scene label; wherein the preprocessing operation includes a standardization process, and the standardization process at least includes adjusting the illumination, color and geometric features of the image;

[0041] The model training module is used to introduce different scenario data for model training using a layered training strategy and dynamically adjust parameters according to the currently processed data scenario;

[0042] Hierarchical evaluation module, used to perform hierarchical evaluation of the model based on scene classification labels;

[0043] Deploy the inference module, which is used to deploy the model after hierarchical evaluation and dynamically adjust the model inference strategy according to the scene label of the input image;

[0044] The post-processing module is used to optimize the model through scene feature clustering and transfer learning, and use the optimized model to optimize visual detection.

[0045] In a third aspect, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the visual inspection optimization method described in any one of the first aspects is implemented.

[0046] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the visual inspection optimization method described in any one of the first aspects is implemented.

[0047] The beneficial effects brought by the technical solution provided in the embodiments of the present application include:

[0048] 1. No need for joint training of multiple datasets: By introducing scene classification labels, combining hierarchical training and scene adaptation modules, the complexity and inconsistency of joint training of multiple datasets are avoided. The model can gradually learn and adapt to different scenarios, thus achieving good adaptability to multiple scenarios without combining multiple datasets.

[0049] 2. Stronger scene adaptation capability: Through the introduction of scene classification labels and adaptive modules, the model can automatically adjust its feature extraction and reasoning path according to the scene information of the input image, and achieve adaptation to different scenes. Compared with the existing technology, this application improves the flexibility and scene processing capability of the model.

[0050] 3. Simplify the model training process: Through the layered training strategy, this application gradually introduces data from different scene categories for training, which simplifies the training process and allows the model to adapt to each scene more effectively. There is no need for complex data preprocessing and format conversion, and it also avoids labeling conflicts and data inconsistency during model training.

[0051] 4. Enhance the generalization ability of the model: By combining the target detection task with scene classification through multi-task learning, the model not only learns the target features during training, but also learns the scene features, which enhances the generalization ability of the model in different scenes. Even in new scenes that have never been seen, the model can rely on the scene classification module to make adaptive adjustments and maintain high performance.

[0052] 5. Efficient model evaluation and reasoning: Through the hierarchical evaluation method, this application can conduct refined evaluation for different scene categories, detect whether the model's performance in each scene meets expectations, and then perform targeted optimization. In the reasoning stage, the model dynamically adjusts the reasoning path through scene labels to ensure high reasoning efficiency and accuracy in different scenes.

[0053] 6. Support scene feature clustering and transfer learning: Through scene feature clustering and transfer learning strategies, the model can discover the feature similarities between different scenes and share the learned features between these similar scenes, effectively improving the training efficiency and reasoning ability of the model and reducing the risk of repeated learning and overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0055] Figure 1 A flowchart of the steps of the visual inspection optimization method provided in the embodiment of the present application;

[0056] Figure 2 A block diagram of a visual inspection optimization system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] In the description of the present invention, the terms "comprises", "has" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may also include other steps or units that are not explicitly listed but are inherent to these processes, methods, products or apparatuses, or steps or units added based on further optimization schemes conceived by the present invention.

[0059] The present application is applicable to a visual detection optimization method based on data set classification in multiple scenes. Its purpose is to improve the multi-scene adaptability of the visual detection model by using data set classification without jointly training the model through multiple data sets. It mainly includes introducing scene classification labels in the data set annotation and preprocessing stage, standardizing the scene features, and providing scene information for subsequent training. In the model selection and training stage, the hierarchical training and scene adaptation modules are introduced, and the multi-task learning of scene classification is used to enable the model to dynamically adjust feature learning in different scenes and improve the multi-scene adaptability. In the model evaluation and verification stage, the model performance is evaluated by scene hierarchical evaluation, and multi-scene adaptation evaluation is introduced to facilitate the discovery of whether the model has performance deviation in certain specific scenes and to perform corresponding optimization. In the model deployment and reasoning stage, dynamic scene adaptation reasoning is introduced to dynamically adjust the reasoning path of the model to adapt to the characteristics of the scene, thereby improving the accuracy of reasoning. In the later optimization, scene feature clustering and transfer learning are used to optimize the multi-scene performance of the model. The advantage is that this optimization method, by introducing methods such as data set classification labels, hierarchical training and multi-task learning, can make full use of data set classification and improve the multi-scene adaptability of the visual detection model without relying on the joint training of multiple data sets. This method not only simplifies the training process and reduces the problems caused by inconsistent annotations and data processing complexity, but also significantly enhances the generalization and robustness of the model, while reducing computing resources and time costs, and has a wide range of practical application value. This optimization method provides an innovative method for solving multi-scene visual detection tasks, improves the adaptability of the model in different scenes, reduces the complexity of joint training, enhances the generalization ability of the model, and provides an efficient solution for practical applications. In particular, it has broad application prospects in many fields such as robot navigation (indoor and outdoor scene switching, etc.), autonomous driving (urban streets, rural streets, etc.), drone vision (farmland detection, urban mapping, etc.), medical imaging (different imaging equipment such as X-ray, CT, etc.), and intelligent monitoring (day and night switching, etc.).

[0060] Please refer to Figure 1 , which shows a flowchart of a visual inspection optimization method based on data set classification in one embodiment, the method may include the following steps:

[0061] S1, data set annotation and preprocessing: assigning a corresponding scene label to each collected image sample, and performing a preprocessing operation on the data according to the assigned scene label; wherein the preprocessing operation includes a standardization process, and the standardization process at least includes adjusting the illumination, color and geometric features of the image;

[0062] Specifically, by introducing scene classification labels for each sample, the data can be classified according to different scene categories, such as indoor, outdoor, etc. This helps the model distinguish the feature differences in different scenes during training. During the preprocessing process, the data can be standardized according to different scene categories, such as adjusting the lighting, color or geometric features to ensure that the model can adapt to various scene characteristics.

[0063] S2, model selection and training: During the model training process, we first adopt a hierarchical training method to gradually introduce different scene data, and adjust the model parameters through the scene adaptation module so that it can adaptively extract features in different scenes. In addition, the model combines scene classification tasks and target detection tasks through multi-task learning, learning both target features and scene features, improving the generalization ability of the model in multi-scene tasks.

[0064] S3, Model Evaluation and Verification: In the model evaluation phase, the system conducts hierarchical evaluation of the model according to the scene classification labels, that is, the performance of the model in different scenes is evaluated separately to ensure that it can work stably in different scenes. Through this multi-scene adaptive evaluation, the overall performance of the model in a variety of environments can be verified and possible scene-specific problems can be discovered.

[0065] S4, Model deployment and reasoning: In the deployment phase, the model dynamically adjusts the reasoning strategy based on the scene label of the input image. For example, when the model detects that the input image is from outdoors, it adjusts specific weights or feature extraction paths to better adapt to the characteristics of outdoor scenes, thereby ensuring the accuracy and efficiency of reasoning.

[0066] S5, post-processing: In the post-processing stage, through scene feature clustering, we further explore the commonalities and characteristics between different scenes to help the model optimize its multi-scene processing capabilities. At the same time, using the transfer learning method, the model can quickly adapt to new scenes or environments, thereby improving the adaptability and robustness of the model in unknown scenes.

[0067] The specific technical solutions are as follows:

[0068] 1. Dataset Annotation

[0069] Introducing dataset classification labels: When annotating the dataset, add a "scene classification" label (such as "indoor", "outdoor", "city", etc.) to each image or sample, so that each data sample contains not only the target detection label, but also the scene category label, providing additional scene information for the model training process.

[0070] Label format extension: Modify the format of data annotation files, such as extending the label structure of COCO datasets and adding scene category fields. This allows the model to dynamically adjust feature learning based on scene labels during subsequent training.

[0071] Assign an additional scene classification label to each image. For each image data x i , corresponding to the target detection label y i External and scene labels i , the scene label is used to distinguish different scene features, and the formula is described as follows:

[0072]

[0073] Where N is the number of training samples; L detection is the loss function of target detection (e.g. cross entropy loss); L secene is the scene classification loss function (e.g., classification loss); λ is an adjustment coefficient used to balance the losses of detection and scene classification.

[0074] 2. Data Preprocessing

[0075] Standardization of scene features: In the data preprocessing process, in addition to conventional data enhancement (such as flipping, cropping, etc.), data of different scene categories can be standardized. For example, according to different scene labels, the data enhancement strategy is adjusted to make the data enhancement operation in each scene more suitable for the characteristics of the scene (such as lighting conditions, color style, etc.).

[0076] For example, for each scene label s i , applying different normalization methods g (si) :

[0077] Among them, g si is the preprocessing operation of scene characteristics, x i is the image data, x i It is the normalized image data, preprocessing operations such as normalization, enhancement, etc.

[0078] 3. Model selection and training

[0079] Hierarchical training and scene adaptation module: During the model training phase, a hierarchical training strategy can be used to first train the model on a single scene category, and then gradually introduce data from other scene categories. For data from different scene categories, the model can adaptively adjust weights through a scene adaptation module or conditional network (such as introducing scene label conditions in the network structure) to improve performance in different scenarios.

[0080] Assume that the model is first trained on a single scene dataset, and then other scene data are gradually introduced. For each scene s i , the optimized loss function can be:

[0081]

[0082] Among them, D si Indicates scenes i The corresponding data set; L detection is the loss function for target detection.

[0083] In order to adapt the model to different scenarios, a scene adaptation module can be introduced. For example, using scene labels s i Through the conditional network g (si) Dynamic Adjustment Features:

[0084] h' l (x i )=h l (x i )⊙g(s i )

[0085] Among them, h l (x i ) is the feature map of the network layer l, g(s i ) is the scene label s i The encoding function.

[0086] Multi-task learning for scene classification: During the training process, the model not only learns the object detection task, but also the scene classification task. By jointly optimizing the object detection loss function and the scene classification loss function, the model can focus on both the object features and the scene features, thereby improving the performance in multiple scenes.

[0087] In multi-task learning, the model not only needs to learn object detection, but also scene classification tasks. i ,y i ,s i )The optimized joint loss function:

[0088] L multi-task =L detection (x i ,y i )+αL scene (x i ,s i )

[0089] Among them, L detection is the loss function of target detection; L secene is the scene classification loss function; α is the coefficient that controls the balance between the two tasks.

[0090] 4. Model evaluation and validation

[0091] Hierarchical evaluation by scene: During the model evaluation phase, the model can be evaluated hierarchically according to the scene classification label, that is, the performance of the model in different scene categories (such as indoor and outdoor scenes) can be evaluated separately. In this way, it can be found whether the model has performance deviations in certain specific scenes and corresponding optimization can be carried out.

[0092] Classify tags by scene i Evaluate the performance of the model in each scenario separately, where the evaluation metric Metric si Can be calculated by scene category:

[0093]

[0094] in, is the predicted output of the model, y i is the target detection label, D si It's scenes i The test set;

[0095] For multi-scenario adaptability evaluation, new evaluation indicators are introduced through:

[0096]

[0097] Get the multi-scenario evaluation results, where w j It's scenes j The weight of k is the number of scene categories.

[0098] 5. Model deployment and inference

[0099] Dynamic scene adaptation reasoning: During the model deployment phase, the model’s reasoning path can be dynamically adjusted according to the scene label of the input image during the reasoning process. For example, the scene classification module is used to predict the scene category of the current input image and guide the model to select specific weights or network branches to adapt to the characteristics of the scene, thereby improving the accuracy of reasoning.

[0100] For an input image x i , first by predicting the scene label s i , and then adjust the model inference path based on the label:

[0101] z i =f(x i )+g(s i )

[0102] Among them, f(x i ) is the image feature extracted by the model, g(s i ) is the scene code, z i It is a feature that combines scene information.

[0103] In an optional embodiment of the present application, it also includes: using scene coding of large-scale visual models during deployment, large-scale visual models (such as ViT, SAM, etc.) can be combined with scene coding vectors for reasoning, thereby achieving better feature extraction and detection in different scenarios.

[0104] 6. Post-optimization and enhancement

[0105] Scene feature clustering and transfer learning: After training is completed, transfer learning optimization is performed for specific scenarios to further improve the performance of the model in certain specific scenarios. In addition, scene feature clustering can be used to discover the similarity of scene features and share scene features, thereby reducing training time and computing resource consumption.

[0106] For scene features f s Perform cluster analysis and optimize the model. The feature clustering formula is:

[0107]

[0108] Among them, μ j is the cluster center, f s (x i ) is the scene feature. After clustering, transfer learning is performed on the data of each subclass to optimize the adaptability of the model in different scenarios.

[0109] From the above, it can be seen that the key points of this application are:

[0110] ①Introduction of dataset classification labels: By adding scene classification labels (such as "indoor", "outdoor", etc.) to each image sample, the model can obtain scene-related feature information during training, thereby enhancing the performance of the model in multiple scenarios.

[0111] ② Hierarchical training strategy: In model training, first train a single scenario, and then gradually introduce other scenarios. Through hierarchical training, the adaptability of the model in multiple scenarios is gradually enhanced, avoiding data inconsistency problems caused by training multiple scenarios at one time.

[0112] ③ Scene Adaptation Module: Introduce a scene adaptation module or conditional network into the model structure to dynamically adjust the model’s feature extraction and reasoning path according to the scene label of the input image. This module can improve the model’s detection capability by differentially processing different scenes.

[0113] ④Multi-task learning: Combining the target detection task and the scene classification task, a multi-task learning strategy is adopted to enable the model to learn target features and scene features at the same time, thereby improving its comprehensive performance in multi-scene environments.

[0114] ⑤Hierarchical evaluation and scenario-adaptive reasoning: In the model evaluation and reasoning stages, hierarchical evaluation is performed or the reasoning path is dynamically adjusted based on scenario labels to ensure the stability and accuracy of the model's performance in different scenarios.

[0115] In addition, the following technical points are specifically involved:

[0116] ① Visual detection method based on scene classification labels: By adding scene classification labels to the data set, the technical means and processes enable the model to obtain scene information during training and reasoning.

[0117] ② Multi-scenario adaptation method of hierarchical training: An innovative method to enhance model adaptability by gradually introducing multi-scenario datasets through a hierarchical training strategy without joint training of multiple datasets.

[0118] ③ Implementation and application of scene adaptation module: In the visual detection model, the scene classification label is used as the input condition to dynamically adjust the module design and optimization strategy of network feature extraction and reasoning path.

[0119] ④ Combination of scene classification and target detection under the multi-task learning framework: In the multi-task learning framework, the scene classification and target detection tasks are combined to jointly optimize the implementation method of the loss function.

[0120] ⑤ Hierarchical evaluation and reasoning based on scene features: During the model evaluation and reasoning process, the method of stratifying or dynamically adjusting the reasoning path according to scene labels ensures the consistency of the model's performance in multi-scene tasks.

[0121] ⑥ Optimization method of scene feature clustering and transfer learning: Improve the performance of the model in multi-scene data sets through scene feature clustering and transfer learning strategies.

[0122] Compared with the prior art, the advantages of this application are mainly reflected in the following aspects:

[0123] 1. No need for joint training of multiple datasets

[0124] Existing technology: In multi-scenario tasks, it is usually necessary to combine multiple data sets for model training to enhance the multi-scenario adaptability of the model. However, this method is prone to increase the complexity of model training, labeling conflicts, and reduced model generalization ability when the data set annotation methods are inconsistent and the data distribution is different.

[0125] Advantages of this application: This application avoids the complexity and inconsistency of joint training of multiple data sets by introducing scene classification labels, combining hierarchical training and scene adaptation modules. The model can gradually learn and adapt to different scenes, thereby achieving good adaptability to multiple scenes without combining multiple data sets.

[0126] 2. Stronger scene adaptability

[0127] Existing technologies: Most existing visual detection models perform well in a single scenario, or require multiple independent models to be trained to handle different scenarios, resulting in low reasoning efficiency.

[0128] Advantages of this application: By introducing scene classification labels and adaptive modules, the model can automatically adjust its feature extraction and reasoning path according to the scene information of the input image, and achieve adaptation to different scenes. Compared with the existing technology, this application improves the flexibility and scene processing capabilities of the model.

[0129] 3. Simplify the model training process

[0130] Existing technology: Model training in multiple scenarios often requires complex preprocessing of multiple data sets, unifying data formats and annotation standards, etc., which results in high training costs. In addition, the model may not converge efficiently due to differences in data set size or scene complexity during training.

[0131] Advantages of this application: Through the layered training strategy, this application gradually introduces data from different scene categories for training, which simplifies the training process, and the model can adapt to each scene more effectively. There is no need for complex data preprocessing and format conversion, and it also avoids labeling conflicts and data inconsistency during model training.

[0132] 4. Enhance model generalization ability

[0133] Existing technologies: Models usually only perform well in scenarios similar to the training dataset, and their performance often drops significantly when faced with new scenarios or environmental changes.

[0134] Advantages of this application: By combining the target detection task with scene classification through multi-task learning, the model not only learns the target features but also the scene features during training, which enhances the generalization ability of the model in different scenes. Even in new scenes that have never been seen, the model can rely on the scene classification module to make adaptive adjustments and maintain high performance.

[0135] 5. Efficient model evaluation and reasoning

[0136] Existing technologies: Model evaluation in existing technologies is usually performed uniformly on the entire test set, which makes it difficult to carefully evaluate the performance of the model in different scenarios. In addition, the ability to adjust the model for different scenarios during the reasoning process is limited, resulting in poor reasoning results in some complex scenarios.

[0137] Advantages of this application: Through the hierarchical evaluation method, this application can conduct refined evaluation for different scene categories, detect whether the model's performance in each scene meets expectations, and thus perform targeted optimization. In the inference stage, the model dynamically adjusts the inference path through scene labels to ensure that high inference efficiency and accuracy can be maintained in different scenes.

[0138] 6. Support scene feature clustering and transfer learning

[0139] Existing technologies: For multi-scenario problems, existing models usually lack flexible scene feature clustering and transfer learning mechanisms, resulting in the inability of models to effectively share features when processing similar scenarios.

[0140] Advantages of this application: Through scene feature clustering and transfer learning strategies, this application enables the model to discover feature similarities between different scenes and share learned features between these similar scenes, effectively improving the training efficiency and reasoning ability of the model and reducing the risk of repeated learning and overfitting.

[0141] In one embodiment of the present application, Figure 2 , and also provides a block diagram of a visual inspection optimization system based on data set classification, the system comprising:

[0142] The data set annotation and preprocessing module is used to assign a corresponding scene label to each collected image sample and perform a preprocessing operation on the data according to the assigned scene label; wherein the preprocessing operation includes a standardization process, and the standardization process at least includes adjusting the illumination, color and geometric features of the image;

[0143] The model training module is used to introduce different scenario data for model training using a layered training strategy and dynamically adjust parameters according to the currently processed data scenario;

[0144] Hierarchical evaluation module, used to perform hierarchical evaluation of the model based on scene classification labels;

[0145] Deploy the inference module, which is used to deploy the model after hierarchical evaluation and dynamically adjust the model inference strategy according to the scene label of the input image;

[0146] The post-processing module is used to optimize the model through scene feature clustering and transfer learning, and use the optimized model to optimize visual detection.

[0147] For the specific definition of the visual inspection optimization system based on data set classification, please refer to the definition of the visual inspection optimization method based on data set classification above, which will not be repeated here. Each module in the above-mentioned visual inspection optimization system based on data set classification can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0148] In one embodiment, an electronic device is provided, which may be a computer. The electronic device includes a processor, a memory and a network interface connected via a system bus. The processor of the device is used to provide computing and control capabilities. The memory of the device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for visual inspection optimization data based on data set classification. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a visual inspection optimization method based on data set classification is implemented.

[0149] In one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned visual inspection optimization method based on data set classification are implemented.

[0150] In one embodiment of the present application, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned visual inspection optimization method based on data set classification.

[0151] The computer-readable storage medium and computer program product provided in this embodiment have similar implementation principles and technical effects to those of the above method embodiments, and are not described in detail here.

[0152] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in M ​​forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SyMchliMk) DRAM (SLDRAM), memory bus (RaMbus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0153] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A visual inspection optimization method based on data set classification, characterized in that: The method comprises: Assigning a corresponding scene label to each collected image sample, and performing a preprocessing operation on the data according to the assigned scene label; wherein the preprocessing operation includes a standardization process, and the standardization process at least includes adjusting the illumination, color, and geometric features of the image; A hierarchical training strategy is used to introduce data from different scenarios for model training, and parameters are dynamically adjusted according to the data scenario currently being processed; Evaluate models hierarchically based on scene classification labels; Deploy the model after hierarchical evaluation and dynamically adjust the model inference strategy based on the scene label of the input image; The model is optimized through scene feature clustering and transfer learning, and the optimized model is used for visual detection optimization.

2. The visual inspection optimization method according to claim 1, characterized in that: Assign a corresponding scene label to each collected image sample, and perform preprocessing operations on the data according to the assigned scene label, including: Assign an additional scene classification label to each image. For each image data x i , corresponding to the target detection label y i External and scene labels i , scene labels are used to distinguish different scene features, through the formula: Determine the overall loss function of the model, where N is the number of training samples and L is detection is the loss function of target detection, L secene is the scene classification loss function, and λ is the adjustment coefficient used to balance the losses of target detection and scene classification.

3. The visual inspection optimization method according to claim 1, characterized in that: Standardization also includes normalization operations, where: For each scene label s i , applying different normalization methods g (si) : Among them, g si is the preprocessing operation of scene characteristics, x i is the image data, x' i is the normalized image data.

4. The visual inspection optimization method according to claim 2, characterized in that: A hierarchical training strategy is used to introduce data from different scenarios for model training, and parameters are dynamically adjusted according to the data scenario currently being processed, including: For each training sample (x i ,y i ,s i ) The optimized joint loss function L multi-task : L multi-task =L detection (x i ,y i )+αL scene (x i ,s i ) Among them, L detection is the loss function of target detection; L secene is the scene classification loss function; α is the coefficient that controls the balance between the two tasks.

5. The visual inspection optimization method according to claim 1, characterized in that: Models are evaluated hierarchically based on scene classification labels, including: Classify tags by scene i Evaluate the performance of the model in each scenario separately, where the evaluation metric Metric si Can be calculated by scene category: in, is the predicted output of the model, y i is the target detection label, D si It's scenes i The test set; For multi-scenario adaptability evaluation, a new evaluation index is introduced. Get the multi-scenario evaluation results, where w j It's scenes j The weight of k is the number of scene categories.

6. The visual inspection optimization method according to claim 1, characterized in that: Deploy the model after layered evaluation and dynamically adjust the model inference strategy based on the scene label of the input image, including: For an input image x i , first by predicting the scene label s i , and then adjust the model inference path based on the label: z i =f(x i )+g(s i ) Among them, f(x i ) is the image feature extracted by the model, g(s i ) is the scene code, z i It is a feature that combines scene information.

7. The visual inspection optimization method according to claim 1, characterized in that: The model is optimized through scene feature clustering and transfer learning, and the optimized model is used for visual inspection optimization, including: Perform cluster analysis on scene features and optimize the model. The feature clustering formula is: Among them, μ j is the cluster center, f s (x i ) is the scene feature. After clustering, transfer learning is performed on the data of each subclass to optimize the adaptability of the model in different scenarios.

8. A visual inspection optimization system based on data set classification, characterized in that: The system comprises: The data set annotation and preprocessing module is used to assign a corresponding scene label to each collected image sample and perform a preprocessing operation on the data according to the assigned scene label; wherein the preprocessing operation includes a standardization process, and the standardization process at least includes adjusting the illumination, color and geometric features of the image; The model training module is used to introduce different scenario data for model training using a layered training strategy and dynamically adjust parameters according to the currently processed data scenario; Hierarchical evaluation module, used to perform hierarchical evaluation of the model based on scene classification labels; Deploy the inference module, which is used to deploy the model after hierarchical evaluation and dynamically adjust the model inference strategy according to the scene label of the input image; The post-processing module is used to optimize the model through scene feature clustering and transfer learning, and use the optimized model to optimize visual detection.

9. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the visual inspection optimization method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the visual inspection optimization method as described in any one of claims 1 to 7 is implemented.

Images