A Visual Detection Optimization Method and System Based on Dataset Classification

Abstract

This application discloses a visual detection optimization method and system based on dataset classification. The method includes introducing scene classification labels and standardizing scene features during the dataset annotation and preprocessing stage; performing multi-task learning for scene classification during the model selection and training stage, enabling the model to dynamically adjust feature learning under different scenes and improve multi-scene adaptability; introducing scene-based hierarchical evaluation and corresponding optimization during the model evaluation and validation stage; introducing dynamic scene-adaptive inference during the model deployment and inference stage, dynamically adjusting the model's inference path to adapt to the characteristics of the scene, thereby improving inference accuracy; and using scene feature clustering and transfer learning in later optimization stages to optimize the model's multi-scene performance. This application improves the multi-scene adaptability of visual detection models by utilizing dataset classification without jointly training the model on multiple datasets.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, and in particular to a visual inspection optimization method and system based on dataset classification. Background Technology

[0002] Existing visual detection techniques typically employ deep learning models. First, the model is trained on a large labeled dataset to learn the features of different targets or regions in an image. After preprocessing and data augmentation, the dataset is fed into a pre-trained or custom model for training, such as YOLO or Faster R-CNN. Once trained, the model can be deployed in a real-world environment to detect new images through inference.

[0003] Existing visual inspection technologies have the following drawbacks:

[0004] 1. Visual detection tasks rely on diverse datasets, which are generally categorized by data type. Indoor scene datasets such as SUN RGB-D and NYU DepthV2 provide rich RGB and depth data for 3D reconstruction and scene parsing, while outdoor datasets like Cityscapes and KITTI focus on target detection tasks in autonomous driving and urban traffic environments. Remote sensing datasets like DeepGlobe and Spacenet support surface target detection in satellite images, while medical image datasets such as BraTS and ISIC are used for tumor and lesion detection. Furthermore, autonomous driving datasets such as nuScenes and Waymo OpenDataset provide high-precision perception data, driving the development of autonomous driving technology. Agricultural and drone datasets such as CropDeep and DeepWeeds help with crop and weed detection, while 3D reconstruction tasks rely on 3D datasets like ShapeNet and ModelNet40. However, these diverse datasets only support the training and evaluation of visual detection models, resulting in low utilization.

[0005] 2. Existing visual detection models, in certain complex scenarios, such as when it is necessary to cover different scenes or increase the amount of data, are trained jointly using multiple datasets. In order for the model to handle the labels and distributions of different datasets, consistent preprocessing and label standardization are usually required for different datasets, which results in high training complexity and resource consumption. Summary of the Invention

[0006] Based on this, embodiments of this application provide a visual detection optimization method and system based on dataset classification, which improves the multi-scene adaptability of the visual detection model by using dataset classification without jointly training the model through multiple datasets.

[0007] Firstly, a visual detection optimization method based on dataset classification is provided, which includes:

[0008] Each collected image sample is assigned a corresponding scene label, and the data is preprocessed according to the assigned scene label; wherein, the preprocessing operation includes standardization, which includes at least adjusting the lighting, color and geometric features of the image;

[0009] A hierarchical training strategy is adopted to introduce data from different scenarios for model training, and the parameters are dynamically adjusted according to the current data scenario being processed.

[0010] The model is evaluated hierarchically based on scene classification labels;

[0011] The stratified evaluation model is deployed, and the model inference strategy is dynamically adjusted based on the scene label of the input image.

[0012] The model is optimized through scene feature clustering and transfer learning, and the optimized model is then used for visual detection optimization.

[0013] Optionally, each collected image sample is assigned a corresponding scene label, and the data is preprocessed according to the assigned scene label, including:

[0014] Assign an additional scene classification label to each image for each image data x. i , corresponding to the target detection label y i External and scene tags i Scene labels are used to distinguish different scene features, using the formula:

[0015]

[0016] Determine the overall loss function of the model, where N is the number of training samples, and L... detection It is the loss function for object detection, L secene λ is the scene classification loss function, and λ is the adjustment coefficient used to balance the losses of object detection and scene classification.

[0017] Optionally, the standardization process also includes a normalization operation, wherein:

[0018] For each scene label s i Applying different normalization methods g (si) :

[0019] Among them, g si It is a preprocessing operation based on scene characteristics, x i It is image data, x' i It is normalized image data.

[0020] Optionally, a hierarchical training strategy can be adopted to introduce data from different scenarios for model training, and the parameters can be dynamically adjusted according to the current data scenario being processed, including:

[0021] For each training sample (x) i y i s i )Optimized joint loss function L multi-task :

[0022] L multi-task =L detection (x i ,y i )+αL scene (x i ,s i )

[0023] Among them, L detection It is the loss function for object detection; L secene α is the scene classification loss function; α is a coefficient that controls the balance between the two tasks.

[0024] Optionally, the model can be stratified and evaluated based on scene classification labels, including:

[0025] Category tags by scenario i The model's performance was evaluated in each scenario, with the evaluation metric being Metric. si Calculations can be made by scene category:

[0026]

[0027] in, It is the model's predicted output, y i It is the object detection label, D si It is scene s i The test set;

[0028] For multi-scenario adaptability assessment, new assessment indicators are introduced, through...

[0029]

[0030] Multi-scenario evaluation results were obtained, among which, w j It is scene s j The weights are k, where k is the number of scene categories.

[0031] Optionally, the hierarchically evaluated model is deployed, and the model inference strategy is dynamically adjusted based on the scene label of the input image, including:

[0032] For the input image x i First, by predicting scene labels s iThen, adjust the model's inference path based on that label:

[0033] z i =f(x) i )+g(s i )

[0034] Where, f(x) i ) represents the image features extracted by the model, g(s) i ) is scene encoding, z i It is a feature that combines scene information.

[0035] Optionally, the model can be optimized through scene feature clustering and transfer learning, and the optimized model can be used for visual detection optimization, including:

[0036] Cluster analysis is performed on scene features to optimize the model. The feature clustering formula is as follows:

[0037]

[0038] Where, μ j It is the cluster center, f s (x i The data in each subclass is a scene feature. After clustering, transfer learning is performed on the data in each subclass to optimize the model's adaptability in different scenes.

[0039] Secondly, a visual detection optimization system based on dataset classification is provided, which includes:

[0040] The dataset labeling and preprocessing module is used to assign a corresponding scene label to each collected image sample and perform preprocessing operations on the data according to the assigned scene label; wherein, the preprocessing operation includes standardization processing, which includes at least adjusting the lighting, color and geometric features of the image;

[0041] The model training module is used to train the model by introducing data from different scenarios using a hierarchical training strategy, and dynamically adjusts the parameters according to the current data scenario being processed.

[0042] The stratified evaluation module is used to evaluate the model in stratified manner based on scene classification labels;

[0043] The deployment inference module is used to deploy the hierarchically evaluated model and dynamically adjust the model inference strategy based on the scene labels of the input image.

[0044] The post-processing module is used to optimize the model through scene feature clustering and transfer learning, and then use the optimized model for visual detection optimization.

[0045] Thirdly, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the visual detection optimization method described in any of the first aspects above.

[0046] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the visual detection optimization method described in any of the first aspects above.

[0047] The beneficial effects of the technical solutions provided in this application include:

[0048] 1. No need for joint training on multiple datasets: By introducing scene classification labels and combining hierarchical training and scene adaptation modules, the complexity and inconsistency issues of joint training on multiple datasets are avoided. The model can learn and adapt to different scenes step by step, thus achieving good adaptability to multiple scenes without jointly training on multiple datasets.

[0049] 2. Enhanced Scene Adaptability: By introducing scene classification labels and an adaptive module, the model can automatically adjust its feature extraction and inference paths based on the scene information of the input image, achieving adaptation to different scenes. Compared to existing technologies, this application improves the model's flexibility and scene processing capabilities.

[0050] 3. Simplified Model Training Process: By employing a hierarchical training strategy, this application progressively introduces data from different scenario categories for training, simplifying the training process and enabling the model to adapt more effectively to various scenarios. It eliminates the need for complex data preprocessing and format conversion, and avoids annotation conflicts and data inconsistencies during model training.

[0051] 4. Enhanced Model Generalization Ability: By combining object detection with scene classification through multi-task learning, the model learns not only object features but also scene features during training, enhancing its generalization ability across different scenarios. Even in unfamiliar scenes, the model can adaptively adjust using the scene classification module, maintaining high performance.

[0052] 5. Efficient Model Evaluation and Inference: Through a hierarchical evaluation method, this application can perform refined evaluations for different scenario categories, detecting whether the model's performance in each scenario meets expectations, thereby enabling targeted optimization. During the inference phase, the model dynamically adjusts the inference path based on scenario labels, ensuring high inference efficiency and accuracy across different scenarios.

[0053] 6. Supports scene feature clustering and transfer learning: Through scene feature clustering and transfer learning strategies, the model can discover feature similarities between different scenes and share the learned features among these similar scenes, effectively improving the model's training efficiency and inference ability, and reducing the risk of repetitive learning and overfitting. Attached Figure Description

[0054] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0055] Figure 1 A flowchart illustrating the steps of the visual inspection optimization method provided in this application embodiment;

[0056] Figure 2 This is a block diagram of a visual inspection optimization system provided in an embodiment of this application. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] In the description of this invention, the terms "comprising," "having," and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are expressly listed, but may also include other steps or units that are not expressly listed but are inherent to these processes, methods, products, or apparatuses, or steps or units added based on further optimizations of the inventive concept.

[0059] This application applies to a multi-scene dataset-based visual detection optimization method. Its aim is to improve the multi-scene adaptability of visual detection models by utilizing dataset classification without relying on joint training of the model across multiple datasets. This primarily includes: introducing scene classification labels and standardizing scene features during the dataset annotation and preprocessing stage to provide scene information for subsequent training; introducing hierarchical training and scene adaptation modules during the model selection and training stage, and multi-task learning for scene classification, enabling the model to dynamically adjust feature learning under different scenarios, thus improving multi-scene adaptability; introducing scene-based hierarchical evaluation of model performance and multi-scene adaptation evaluation during the model evaluation and validation stage, facilitating the identification of performance deviations in certain specific scenarios and enabling corresponding optimizations; introducing dynamic scene-adaptive inference during the model deployment and inference stage, dynamically adjusting the model's inference path to adapt to the characteristics of the scene, thereby improving inference accuracy; and using scene feature clustering and transfer learning in later optimization stages to optimize the model's multi-scene performance. The advantage of this optimization method is that, by introducing dataset classification labels, hierarchical training, and multi-task learning, it can fully utilize dataset classification to improve the multi-scene adaptability of visual detection models without relying on joint training across multiple datasets. This method not only simplifies the training process and reduces problems caused by inconsistent labeling and complex data processing, but also significantly enhances the model's generalization ability and robustness while reducing computational resources and time costs, making it highly valuable for practical applications. This optimization method provides an innovative approach to solving multi-scene visual detection tasks, improving the model's adaptability in different scenarios, reducing the complexity of joint training, enhancing the model's generalization ability, and providing an efficient solution for practical applications. It has particularly broad application prospects in fields such as robot navigation (indoor / outdoor scene switching, etc.), autonomous driving (urban / rural streets, etc.), UAV vision (farmland detection, urban mapping, etc.), medical imaging (different imaging devices such as X-rays and CT scans), and intelligent monitoring (day / night switching, etc.).

[0060] Please refer to Figure 1 The diagram illustrates a flowchart of a visual detection optimization method based on dataset classification in one embodiment, which may include the following steps:

[0061] S1, Dataset Labeling and Preprocessing: Assign a corresponding scene label to each collected image sample, and perform preprocessing operations on the data according to the assigned scene label; wherein, the preprocessing operation includes standardization, which includes at least adjusting the lighting, color and geometric features of the image;

[0062] Specifically, by introducing scene classification labels for each sample, the data can be categorized according to different scene types, such as indoors and outdoors. This helps the model distinguish feature differences across different scenes during training. During preprocessing, the data can be standardized according to different scene categories, such as adjusting lighting, color, or geometric features, to ensure the model can adapt to various scene characteristics.

[0063] S2, Model Selection and Training: During model training, a hierarchical training approach is first adopted, gradually introducing data from different scenarios. The model parameters are then adjusted through a scenario adaptation module, enabling it to adaptively extract features across various scenarios. Furthermore, the model employs multi-task learning, combining scenario classification and object detection tasks to learn both target and scenario features, thereby enhancing its generalization ability across multiple scenarios.

[0064] S3, Model Evaluation and Validation: In the model evaluation phase, the system performs tiered evaluation of the model based on scene classification labels, that is, it evaluates the model's performance in different scenes separately to ensure that it can work stably in different scenarios. Through this multi-scene adaptive evaluation, the overall performance of the model in diverse environments can be verified, and potential scene-specific problems can be identified.

[0065] S4, Model Deployment and Inference: During the deployment phase, the model dynamically adjusts its inference strategy based on the scene labels of the input image. For example, when the model detects that the input image is from outdoors, it adjusts specific weights or feature extraction paths to better adapt to the characteristics of outdoor scenes, thereby ensuring the accuracy and efficiency of inference.

[0066] S5, Post-processing: In the post-processing stage, scene feature clustering is used to further explore the commonalities and characteristics between different scenes, helping the model optimize its multi-scene processing capabilities. Simultaneously, transfer learning methods are used to enable the model to quickly adapt to new scenes or environments, thereby improving the model's adaptability and robustness in unknown scenarios.

[0067] The specific technical solution is as follows:

[0068] 1. Dataset labeling

[0069] Introduce dataset classification labels: When labeling the dataset, add a "scene classification" label (such as "indoor", "outdoor", "city" etc.) to each image or sample, so that each data sample not only contains the object detection label, but also the scene category label, providing additional scene information during model training.

[0070] Label format extension: Modify the format of the data annotation file, such as expanding the label structure of datasets like COCO by adding a scene category field. This allows the model to dynamically adjust feature learning based on scene labels during subsequent training.

[0071] Assign an additional scene classification label to each image for each image data x. i , corresponding to the target detection label y i External and scene tags i Scene labels are used to distinguish different scene features, and the formula is described as follows:

[0072]

[0073] Where N is the number of training samples; L detection It is the loss function for object detection (e.g., cross-entropy loss); L secene λ is the scene classification loss function (e.g., classification loss); λ is the adjustment coefficient used to balance the loss of detection and scene classification.

[0074] 2. Data Preprocessing

[0075] Standardization of scene features: In the data preprocessing process, in addition to conventional data augmentation (such as flipping, cropping, etc.), data from different scene categories can be standardized. For example, data augmentation strategies can be adjusted according to different scene labels to make the data augmentation operations in each scene more suitable for the characteristics of that scene (such as lighting conditions, color style, etc.).

[0076] For example, for each scene label s i Applying different normalization methods g (si) :

[0077] Among them, g si It is a preprocessing operation based on scene characteristics, x i It is image data, x i It is normalized image data, with preprocessing operations such as normalization and enhancement.

[0078] 3. Model Selection and Training

[0079] Layered Training and Scene Adaptation Module: During model training, a layered training strategy can be used to first train the model on a single scene category, and then gradually introduce data from other scene categories. For data from different scene categories, the model can adaptively adjust weights through a scene adaptation module or a conditional network (such as introducing scene label conditions into the network structure) to improve performance in different scenes.

[0080] Suppose we first train the model on a single-scene dataset, and then gradually introduce data from other scenes. For each scene s... i The optimized loss function can be:

[0081]

[0082] Among them, D si Representing scenario s i The corresponding dataset; L detection It is the loss function for object detection.

[0083] To enable the model to adapt to different scenarios, a scenario adaptation module can be introduced. This can be achieved by utilizing scenario labels. i Through conditional network g (si) Dynamic adjustment features:

[0084] h' l (x i ) = h l (x i )⊙g(s i )

[0085] Among them, h l (x i ) is the feature map of the l-th layer of the network, g(s) i ) is a scene tag s i The encoding function.

[0086] Multi-task learning for scene classification: During training, the model learns not only object detection but also scene classification. By jointly optimizing the object detection loss function and the scene classification loss function, the model can simultaneously focus on both object features and scene features, thereby improving performance across multiple scenes.

[0087] In multi-task learning, the model must learn not only object detection but also scene classification. For each training sample (x) i y i s i Optimized joint loss function:

[0088] L multi-task =L detection (x i ,y i )+αL scene (x i ,s i )

[0089] Among them, L detection It is the loss function for object detection; L secene α is the scene classification loss function; α is a coefficient that controls the balance between the two tasks.

[0090] 4. Model Evaluation and Validation

[0091] Stratified evaluation by scenario: During the model evaluation phase, the model can be evaluated stratified according to scenario classification labels, that is, the model's performance is evaluated separately under different scenario categories (such as performance in indoor and outdoor scenarios). This can help identify whether the model has performance deviations in certain specific scenarios and make corresponding optimizations.

[0092] Category tags by scenario i The model's performance was evaluated in each scenario, with the evaluation metric being Metric. si Calculations can be made by scene category:

[0093]

[0094] in, It is the model's predicted output, y i It is the object detection label, D si It is scene s i The test set;

[0095] For multi-scenario adaptability assessment, new assessment metrics are introduced, through:

[0096]

[0097] Multi-scenario evaluation results were obtained, among which, w j It is scene s j The weights are k, where k is the number of scene categories.

[0098] 5. Model Deployment and Inference

[0099] Dynamic scene-adaptive inference: During the model deployment phase, the inference path can be dynamically adjusted based on the scene label of the input image. For example, the scene classification module can predict the scene category of the current input image and guide the model to select specific weights or network branches to adapt to the characteristics of the scene, thereby improving the accuracy of inference.

[0100] For the input image x i First, by predicting scene labels s i Then, adjust the model's inference path based on that label:

[0101] z i =f(x) i )+g(s i )

[0102] Where, f(x) i ) represents the image features extracted by the model, g(s) i ) is scene encoding, z i It is a feature that combines scene information.

[0103] In optional embodiments of this application, the method further includes: utilizing scene encoding of large-scale visual models during deployment. Large-scale visual models (such as ViT, SAM, etc.) can be combined with scene encoding vectors for inference, thereby achieving better feature extraction and detection in different scenarios.

[0104] 6. Post-launch optimization and enhancement

[0105] Scene feature clustering and transfer learning: After training, transfer learning is performed to optimize the model for specific scenes, further improving its performance in those scenarios. Furthermore, scene feature clustering can be used to discover similarities in scene features, enabling feature sharing and reducing training time and computational resource consumption.

[0106] For scene features f s Perform cluster analysis to optimize the model. The feature clustering formula is:

[0107]

[0108] Where, μ j It is the cluster center, f s (x i The data in each subclass is a scene feature. After clustering, transfer learning is performed on the data in each subclass to optimize the model's adaptability in different scenes.

[0109] In summary, the key points of this application are:

[0110] ① Introduction of dataset classification labels: By attaching scene classification labels (such as "indoor", "outdoor" etc.) to each image sample, the model can acquire scene-related feature information during training, thereby enhancing the model's performance in multiple scenes.

[0111] ② Layered training strategy: In model training, first train for a single scenario, and then gradually introduce other scenarios. By using layered training, the adaptability of the model in multiple scenarios is gradually enhanced, avoiding the data inconsistency problem caused by training multiple scenarios at once.

[0112] ③ Scene Adaptive Module: A scene adaptive module or conditional network is introduced into the model structure to dynamically adjust the model's feature extraction and inference path based on the scene label of the input image. This module can improve the model's detection capability by differentially processing different scenes.

[0113] ④ Multi-task learning: Combining object detection and scene classification tasks, a multi-task learning strategy is adopted to enable the model to learn both object features and scene features simultaneously, thereby improving its overall performance in multiple scene environments.

[0114] ⑤ Hierarchical evaluation and scenario-adaptive inference: The model evaluation and inference stages are based on scenario labels, and hierarchical evaluation or dynamic adjustment of the inference path is carried out to ensure the stability and accuracy of the model's performance in different scenarios.

[0115] In addition, the following technical points are specifically involved:

[0116] ① Visual detection method based on scene classification labels: a technical means and process that enables the model to acquire scene information during training and inference by adding scene classification labels to the dataset.

[0117] ② Multi-scenario adaptation method of hierarchical training: An innovative method that enhances the adaptability of the model by gradually introducing multi-scenario datasets through a hierarchical training strategy without joint training with multiple datasets.

[0118] ③ Implementation and application of scene adaptation module: In the visual detection model, the module design and optimization strategy of dynamically adjusting network feature extraction and inference path is implemented by using scene classification labels as input conditions.

[0119] ④ Combining scene classification and object detection in a multi-task learning framework: In a multi-task learning framework, scene classification and object detection tasks are combined to jointly optimize the implementation method of the loss function.

[0120] ⑤ Scene-based hierarchical evaluation and inference: During the model evaluation and inference process, a method of hierarchical or dynamic adjustment of the inference path based on scene labels is used to ensure the consistency of model performance in multiple scene tasks.

[0121] ⑥ Optimization methods for scene feature clustering and transfer learning: Improve the performance of the model on multi-scene datasets by using scene feature clustering and transfer learning strategies.

[0122] Compared with the prior art, the advantages of this application are mainly reflected in the following aspects:

[0123] 1. No need for joint training on multiple datasets

[0124] Existing technology: In multi-scenario tasks, it is often necessary to train models using multiple datasets to enhance their adaptability to various scenarios. However, this approach can easily lead to increased training complexity, labeling conflicts, and decreased model generalization ability when datasets have inconsistent labeling methods or different data distributions.

[0125] The advantages of this application are: By introducing scene classification labels and combining hierarchical training and scene adaptation modules, this application avoids the complexity and inconsistency issues of joint training with multiple datasets. The model can progressively learn and adapt to different scenes, thus achieving good adaptability to multiple scenes without jointly training multiple datasets.

[0126] 2. Enhanced scene adaptability

[0127] Existing technologies: Most existing visual detection models perform well in a single scene, or require multiple independent models to be trained separately to handle different scenes, resulting in low inference efficiency.

[0128] The advantages of this application are: by introducing scene classification labels and an adaptive module, the model can automatically adjust its feature extraction and inference path according to the scene information of the input image, achieving adaptation to different scenes. Compared with existing technologies, this application improves the model's flexibility and scene processing capabilities.

[0129] 3. Simplify the model training process

[0130] Existing technology: Training models in multiple scenarios often requires complex preprocessing of multiple datasets, such as standardizing data formats and annotation standards, resulting in high training costs. Furthermore, differences in dataset size or scenario complexity can easily lead to inefficient model convergence during training.

[0131] The advantages of this application are: by employing a hierarchical training strategy, this application gradually introduces data from different scenario categories for training, simplifying the training process and enabling the model to adapt more effectively to various scenarios. It eliminates the need for complex data preprocessing and format conversion, and also avoids annotation conflicts and data inconsistencies during model training.

[0132] 4. Enhance model generalization ability

[0133] Existing technology: Models typically perform well only in scenarios similar to the training dataset, and their performance often drops significantly when faced with new scenarios or environmental changes.

[0134] The advantages of this application are: by combining object detection with scene classification through multi-task learning, the model learns not only object features but also scene features during training, enhancing its generalization ability in different scenarios. Even in unfamiliar scenes, the model can adaptively adjust based on the scene classification module, maintaining high performance.

[0135] 5. Efficient model evaluation and inference

[0136] Existing technologies: Model evaluation in existing technologies is usually performed uniformly on the entire test set, making it difficult to evaluate the model's performance in different scenarios in detail. In addition, the ability to adjust the model for different scenarios during inference is limited, resulting in poor inference performance in some complex scenarios.

[0137] The advantages of this application are: By employing a hierarchical evaluation method, this application can perform refined evaluations for different scenario categories, detecting whether the model's performance in each scenario meets expectations, thereby enabling targeted optimization. During the inference phase, the model dynamically adjusts the inference path based on scenario labels, ensuring high inference efficiency and accuracy across different scenarios.

[0138] 6. Supports scene feature clustering and transfer learning

[0139] Existing technologies: For multi-scenario problems, existing models often lack flexible scene feature clustering and transfer learning mechanisms, which makes it impossible for models to effectively share features when dealing with similar scenarios.

[0140] Advantages of this application: This application enables the model to discover feature similarities between different scenes through scene feature clustering and transfer learning strategies, and to share the learned features among these similar scenes, effectively improving the training efficiency and inference ability of the model, and reducing the risk of repeated learning and overfitting.

[0141] In one embodiment of this application, such as Figure 2 Furthermore, a block diagram of a visual detection optimization system based on dataset classification is provided. The system includes:

[0142] The dataset labeling and preprocessing module is used to assign a corresponding scene label to each collected image sample and perform preprocessing operations on the data according to the assigned scene label; wherein, the preprocessing operation includes standardization processing, which includes at least adjusting the lighting, color and geometric features of the image;

[0143] The model training module is used to train the model by introducing data from different scenarios using a hierarchical training strategy, and dynamically adjusts the parameters according to the current data scenario being processed.

[0144] The stratified evaluation module is used to evaluate the model in stratified manner based on scene classification labels;

[0145] The deployment inference module is used to deploy the hierarchically evaluated model and dynamically adjust the model inference strategy based on the scene labels of the input image.

[0146] The post-processing module is used to optimize the model through scene feature clustering and transfer learning, and then use the optimized model for visual detection optimization.

[0147] Specific limitations regarding the dataset-based visual inspection optimization system can be found in the limitations of the dataset-based visual inspection optimization method described above, and will not be repeated here. Each module in the aforementioned dataset-based visual inspection optimization system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0148] In one embodiment, an electronic device, which may be a computer, is provided. The electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for visual inspection optimization data based on dataset classification. The network interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a visual inspection optimization method based on dataset classification.

[0149] In one embodiment of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described visual detection optimization method based on dataset classification.

[0150] In one embodiment of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned visual detection optimization method based on dataset classification.

[0151] The computer-readable storage medium and computer program product provided in this embodiment are similar in implementation principle and technical effect to the above method embodiments, and will not be repeated here.

[0152] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in M ​​forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0153] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A visual detection optimization method based on dataset classification, characterized in that, The method comprises: assigning a corresponding scene label to each collected image sample, and performing a preprocessing operation on the data according to the assigned scene label; wherein the preprocessing operation comprises standardization processing, and the standardization processing at least includes adjusting the illumination, color and geometric features of the image; adopting a hierarchical training strategy to introduce different scene data for model training, and dynamically adjusting parameters according to the current processed data scene; performing hierarchical evaluation on the model according to the scene classification label; deploying the model after hierarchical evaluation, and dynamically adjusting the model inference strategy according to the scene label of the input image; optimizing the model through scene feature clustering and transfer learning, and performing visual detection optimization using the optimized model; assigning a corresponding scene label to each collected image sample, and performing a preprocessing operation on the data according to the assigned scene label, including: An additional scene classification label is assigned to each image, for each image data x i , corresponding to the target detection label y i , and the scene label s i , which is used to distinguish different scene characteristics; adopting a hierarchical training strategy to introduce different scene data for model training, and dynamically adjusting parameters according to the current processed data scene, including: For each training sample (x i , y i , s i ), the optimized joint loss function L multi-task : ， wherein L detection is the loss function of target detection; L secene is the scene classification loss function; and a is a coefficient for controlling the balance between the two tasks. performing hierarchical evaluation on the model according to the scene classification label, including: According to the scene classification label s i The performance of the model in each scene is evaluated respectively, where Metric si Calculate by scene category: ， wherein, is the predicted output of the model, y i is the target detection label, D si is the scene s i test set; For multi-scene adaptability evaluation, a new evaluation index is introduced, which is ， The multi-scene evaluation result is obtained, wherein w j is the weight of the scene s j , and k is the number of scene categories. deploying the model after hierarchical evaluation, and dynamically adjusting the model inference strategy according to the scene label of the input image, including: For an input image x i , first a scene label s i is predicted, and then the model inference path is adjusted based on this label: ， where f(x i ) is the image feature extracted by the model, g(s i ) is the scene code, and z i is the feature combined with the scene information. The standardization processing further includes normalization operation, wherein: For each scene label s i , a different normalization method is applied: where g si is a pre-processing operation of the scene characteristics, x i is the image data, is the normalized image data.

2. A visual inspection optimization system based on data set classification, characterized by, The system comprises: a data set labeling and preprocessing module for assigning a corresponding scene label to each collected image sample, and performing a preprocessing operation on the data according to the assigned scene label; wherein the preprocessing operation comprises standardization processing, and the standardization processing at least includes adjusting the illumination, color and geometric features of the image; a model training module for adopting a hierarchical training strategy to introduce different scene data for model training, and dynamically adjusting parameters according to the current processed data scene; a hierarchical evaluation module for performing hierarchical evaluation on the model according to the scene classification label; a deployment and inference module for deploying the model after hierarchical evaluation, and dynamically adjusting the model inference strategy according to the scene label of the input image; a post-processing module for optimizing the model through scene feature clustering and transfer learning, and performing visual detection optimization using the optimized model; assigning a corresponding scene label to each collected image sample, and performing a preprocessing operation on the data according to the assigned scene label, including: An additional scene classification label is assigned to each image, for each image data x i , corresponding to the target detection label y i , and the scene label s i , which is used to distinguish different scene characteristics. adopting a hierarchical training strategy to introduce different scene data for model training, and dynamically adjusting parameters according to the current processed data scene, including: For each training sample (x i , y i , s i ), the optimized joint loss function L multi-task : ， wherein L detection is the loss function of target detection; L secene is the scene classification loss function; and a is a coefficient controlling the balance between the two tasks. performing hierarchical evaluation on the model according to the scene classification label, including: According to the scene classification label s i The performance of the model in each scene is evaluated respectively, where Metric si Calculated by scene category: ， wherein, is the predicted output of the model, y i is the target detection label, D si is the scene s i test set; For multi-scene adaptability evaluation, a new evaluation index is introduced, which is ， The multi-scene evaluation result is obtained, wherein w j is the weight of the scene s j , and k is the number of scene categories. deploying the model after hierarchical evaluation, and dynamically adjusting the model inference strategy according to the scene label of the input image, including: For an input image x i , first a scene label s i is predicted, and then the model inference path is adjusted based on this label: ， where f(x i ) is the image feature extracted by the model, g(s i ) is the scene code, and z i is the feature combined with the scene information. The standardization processing further includes normalization operation, wherein: For each scene label s i , a different normalization method is applied: where g si is a pre-processing operation of the scene characteristics, x i is the image data, is the normalized image data.

3. An electronic device, comprising: comprising a memory and a processor, the memory storing a computer program, the computer program being executed by the processor to realize the visual detection optimization method of claim 1.

4. A computer-readable storage medium, characterized in that, having a computer program stored thereon, the computer program being executed by a processor to realize the visual detection optimization method of claim 1.

Images