An industrial quality inspection scene and AI model matching method and device

Abstract

The application provides a method and device for matching an industrial quality inspection scene and an AI model. The method comprises obtaining scene characteristic information of the industrial quality inspection scene, wherein the scene characteristic information comprises a task type of the industrial quality inspection, characteristics of a quality inspection object, and performance requirements of the AI model for the industrial quality inspection; and matching a target AI model from a scene characteristic library based on the scene characteristic information, wherein the scene characteristic library comprises a plurality of scene characteristic information and a plurality of AI models corresponding to the scene characteristic information respectively, and the target AI model is an AI model corresponding to the industrial quality inspection scene. The application distinguishes different industrial quality inspection scenes by obtaining the scene characteristic information, and then matches the target AI model in the pre-constructed scene characteristic library according to the scene characteristic information, thereby automatically matching the appropriate AI model for the scene, quickly selecting the appropriate AI model, and simultaneously avoiding the dependence on the experience of algorithm engineers and reducing the uncertainty of selecting the AI model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for matching industrial quality inspection scenarios with AI models. Background Technology

[0002] In the field of quality inspection in intelligent manufacturing, a large number of tasks are concentrated in the vision domain, generally divided into detection, classification, and segmentation. However, due to industry differences, there are many products and processes involved, and the process requirements are not the same. The scenarios are fragmented, making it difficult to unify the algorithms for various scenarios.

[0003] Meanwhile, with the rise and development of artificial intelligence technology, more and more AI models are emerging, exhibiting different performance and accuracy for different visual tasks.

[0004] For example, for object detection tasks, there are more than 50 available detection models, such as YoLo v4, YoLo v5, YoLoX, SSD, Faster R-CNN, and M2Det; there are also many branches of classification models, such as ResNet50, ResNeXt, MobileNet, and ImageNet.

[0005] Currently, the only information available about these AI models is related academic papers and benchmark scores, which represent the characteristics and capabilities of the models. Moreover, the performance data is only valid for specific datasets.

[0006] Therefore, in the face of complex quality inspection scenarios, how to quickly select the appropriate AI model to solve quality inspection problems has become a real challenge for algorithm engineers. Summary of the Invention

[0007] The embodiments of this application provide a method and apparatus for matching industrial quality inspection scenarios with AI models, which enables the rapid selection of suitable AI models for industrial quality inspection scenarios and avoids reliance on the experience of algorithm engineers, thereby reducing the uncertainty of AI model selection.

[0008] In a first aspect, embodiments of this application provide a method for matching industrial quality inspection scenarios with AI models. The method includes obtaining scenario feature information of the industrial quality inspection scenario, including the task type of industrial quality inspection, the characteristics of the inspection object, and the performance requirements of industrial quality inspection for the AI ​​model; matching a target AI model from a scenario feature library based on the scenario feature information, the scenario feature library including multiple scenario feature information and AI models corresponding to the multiple scenario feature information, and the target AI model being the AI ​​model corresponding to the industrial quality inspection scenario.

[0009] This application embodiment obtains scene feature information to distinguish different industrial quality inspection scenarios, and then matches the target AI model in a pre-built scene feature library based on the scene feature information, automatically matching a suitable AI model for the scenario, quickly selecting a suitable AI model, while avoiding reliance on the experience of algorithm engineers and reducing the uncertainty of AI model selection.

[0010] In one possible implementation, the task type of industrial quality inspection is determined at least based on the standard operating procedure (SOP) for the industrial quality inspection scenario.

[0011] In some other examples, when the SOP information is not detailed or standardized enough, it is also necessary to obtain process flow analysis and determine the task type for the industrial quality inspection scenario based on the SOP information and process flow analysis information.

[0012] In one possible implementation, the features of the quality inspection object are obtained by feature extraction from positive and negative sample pairs of the quality inspection object. For example, the positive and negative sample pairs of the quality inspection object are input into a feature extractor to extract multiple features, which are the features of the quality inspection object. For instance, the positive and negative sample pairs of the quality inspection object include positive samples and negative samples, where positive samples are images of quality inspection objects that have passed quality inspection, and negative samples are images of quality inspection objects that have failed quality inspection.

[0013] The feature extractor can be implemented to extract specific features from an image, either from a trained feature extraction AI model or from a pre-written feature extraction template.

[0014] In another possible implementation, the features of the quality inspection object include several low-order features and several high-order features. The low-order features are selected from a set of low-order features, and the high-order features are high-order features related to the low-order features from the set of high-order features. The low-order feature set and the high-order feature set are obtained by feature extraction based on positive and negative sample pairs of the quality inspection object.

[0015] For example, the features extracted by the feature extractor are often numerous. It's necessary to select features highly relevant to the quality inspection object. For instance, based on the SOP information, highly relevant low-order feature vectors are selected from the extracted features, such as color, texture, and size feature vectors. For high-order features, high-order features highly relevant to the selected low-order features are filtered from the multiple high-order features extracted by the feature extractor. For example, the Euclidean distance between the selected low-order features and high-order features is calculated, and high-order features with a distance greater than a preset threshold are deleted to remove redundant high-order features. Then, a chi-square test is used to remove high-order features with low relevance to the selected low-order features. The selected low-order features and the filtered high-order features constitute the features of the quality inspection object.

[0016] In another possible implementation, performance requirements include one or more of the following: AI model training time requirements, inference latency requirements, and detection accuracy requirements.

[0017] In another possible implementation, the matching method between an industrial quality inspection scenario and an AI model provided in this application embodiment further includes: when no target AI model is matched from the scenario feature library based on scenario feature information, several candidate AI models are matched from the AI ​​model library based on scenario feature information, the AI ​​module library including multiple AI models and scenario feature information corresponding to the multiple AI models respectively; and the target AI model is determined from the several candidate AI models based on the preferred indicators of the candidate AI models.

[0018] In one example, the preferred metrics for a candidate AI model are related to one or more of the AI ​​model's generalization ability, sample labeling requirements, and AI resource requirements.

[0019] In another possible implementation, the selection metric for candidate AI models is determined based on a weighted average of the AI ​​model's generalization ability, sample labeling requirements, and AI resource requirements. The weight of the AI ​​model's generalization ability is greater than the weight of the sample labeling requirements, and the weight of the sample labeling requirements is greater than the weight of the AI ​​resource requirements.

[0020] For example, considering the characteristics of industrial quality inspection, the generalization ability of the AI ​​model is more important than the sample labeling requirement, and the sample labeling requirement is more important than the AI ​​resource requirement. Therefore, the weight of the generalization ability of the AI ​​model is set to 0.5, the weight of the sample labeling requirement is set to 0.4, and the AI ​​resource requirement is set to 0.1. The optimal index of the candidate AI model is p = 0.5m + 0.4s + 0.1r, where m represents the generalization ability of the AI ​​model, s represents the sample labeling requirement of the AI ​​model, and r represents the AI ​​resource requirement of the AI ​​model.

[0021] In another possible implementation, the method for matching industrial quality inspection scenarios with AI models provided in this application embodiment further includes updating the mapping relationship between the target AI model and scenario feature information to the scenario feature library, so that when the same industrial quality inspection scenario is encountered again, the target AI model can be quickly matched through the scenario feature library.

[0022] Secondly, this application provides an AI model reasoning method for industrial quality inspection scenarios, including collecting image information of the quality inspection object; extracting features from the image information; comparing the extracted features with features in a feature sample library to determine if there are features in the feature sample library that are similar to the extracted features; using the image information as input to the AI ​​model and outputting the quality inspection results.

[0023] This application provides an AI model inference method for industrial quality inspection scenarios. By comparing extracted features with features in a feature sample library, it identifies whether the acquired image is a difficult example. If not, the image information is input into the AI ​​model for quality inspection inference to obtain the quality inspection result. This ensures the accuracy of the quality inspection results output by the AI ​​model.

[0024] In another possible implementation, the features in the feature sample library are obtained by feature extraction from multiple pairs of positive and negative samples of the quality inspection objects.

[0025] In another possible implementation, the AI ​​model is the target AI model obtained by matching based on the method provided in the first aspect.

[0026] Thirdly, embodiments of this application provide a matching device for industrial quality inspection scenarios and AI models, including an acquisition module and a matching module; wherein the acquisition module is used to acquire scenario feature information of the industrial quality inspection scenario, the scenario feature information including the task type of industrial quality inspection, the characteristics of the inspection object, and the performance requirements of industrial quality inspection for the AI ​​model; the matching module is used to match a target AI model from a scenario feature library based on the scenario feature information, the scenario feature library including multiple scenario feature information and AI models corresponding to the multiple scenario feature information respectively, and the target AI model is the AI ​​model corresponding to the industrial quality inspection scenario.

[0027] In one possible implementation, the task types for industrial quality inspection are determined at least based on the standard operating procedures for industrial quality inspection scenarios.

[0028] In another possible implementation, the features of the quality inspection object are obtained by feature extraction from positive and negative sample pairs of the quality inspection object.

[0029] In another possible implementation, the features of the quality inspection object include several low-order features and several high-order features. The low-order features are selected from a set of low-order features, and the high-order features are high-order features related to the low-order features from the set of high-order features. The low-order feature set and the high-order feature set are obtained by feature extraction based on positive and negative sample pairs of the quality inspection object.

[0030] In another possible implementation, performance requirements include one or more of the following: AI model training time requirements, inference latency requirements, and detection accuracy requirements.

[0031] In another possible implementation, the matching module is also used to obtain several candidate AI models from the AI ​​model library based on the scene feature information when no target AI model is matched from the scene feature library based on the scene feature information. The AI ​​module library includes multiple AI models and the scene feature information corresponding to each of the multiple AI models.

[0032] The matching device between industrial quality inspection scenarios and AI models also includes a determination module, which is used to determine the target AI model from several candidate AI models based on the preferred indicators of candidate AI models.

[0033] In another possible implementation, the preferred metrics for candidate AI models are related to one or more of the following: the generalization ability of the AI ​​model, sample labeling requirements, and AI resource requirements.

[0034] In another possible implementation, the selection metric for candidate AI models is determined based on a weighted average of the AI ​​model's generalization ability, sample labeling requirements, and AI resource requirements. The weight of the AI ​​model's generalization ability is greater than the weight of the sample labeling requirements, and the weight of the sample labeling requirements is greater than the weight of the AI ​​resource requirements.

[0035] In another possible implementation, the matching device between the industrial quality inspection scenario and the AI ​​model also includes an update module, which updates the mapping relationship between the target AI model and the scenario feature information to the scenario feature library.

[0036] Fourthly, this application provides an AI model inference device for industrial quality inspection scenarios, including a data acquisition module, an extraction module, a comparison module, and an inference module. The data acquisition module is used to acquire image information of the quality inspection object; the extraction module is used to extract features from the image information; the comparison module is used to compare the extracted features with features in a feature sample library to determine whether there are features in the feature sample library that are similar to the extracted features; and the inference module is used to take the image information as input to the AI ​​model and output the quality inspection results.

[0037] In one possible implementation, the features in the feature sample library are obtained by feature extraction from multiple pairs of positive and negative samples of the quality inspection objects.

[0038] In another possible implementation, the AI ​​model is the target AI model obtained by matching based on the method provided in the first aspect.

[0039] Fifthly, embodiments of this application provide a computing device, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the methods provided in the first and / or second aspects of this application.

[0040] In a sixth aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the methods provided in the first and / or second aspects of this application.

[0041] In a seventh aspect, embodiments of this application provide a computer program or computer program product, the computer program or computer program product including instructions that, when executed, implement the methods provided in the first and / or second aspects of this application.

[0042] Eighthly, embodiments of this application provide a chip including at least one processor and an interface, wherein the at least one processor determines program instructions or data through the interface; the at least one processor is used to execute the program instructions to implement the methods provided in the first aspect and / or the second aspect. Attached Figure Description

[0043] Figure 1 A flowchart illustrating a method for matching an industrial quality inspection scenario with an AI model, provided in an embodiment of this application;

[0044] Figure 2 A flowchart illustrating another method for matching an industrial quality inspection scenario with an AI model, provided as an embodiment of this application;

[0045] Figure 3 A flowchart illustrating an AI model inference method for an industrial quality inspection scenario, provided as an embodiment of this application;

[0046] Figure 4 A schematic diagram of the structure of a matching device between an industrial quality inspection scenario and an AI model provided in an embodiment of this application;

[0047] Figure 5 A schematic diagram of the structure of an AI model inference device for an industrial quality inspection scenario provided in an embodiment of this application;

[0048] Figure 6 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0049] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0050] Currently, the question of which AI model to use for a specific industrial quality inspection scenario is generally addressed by algorithm engineers using their experience to judge and collect data for testing and verification, thus matching the quality inspection scenario with the AI ​​model. However, the accuracy of the matching is highly dependent on the experience and professional level of the algorithm engineers, and is therefore subject to significant uncertainty. Furthermore, this process is time-consuming and labor-intensive, requiring algorithm engineers to repeatedly fine-tune and optimize the model's parameters during data collection and testing.

[0051] To address the aforementioned issues, this application provides a method for matching industrial quality inspection scenarios with AI models. By acquiring scenario feature information, different industrial quality inspection scenarios are distinguished, and then the target AI model is matched in a pre-built scenario feature library based on the scenario feature information. This automatically matches a suitable AI model for the industrial quality inspection scenario, quickly determines the target AI model, and avoids reliance on the experience of algorithm engineers, reducing the uncertainty in selecting the AI ​​model.

[0052] Figure 1 This application provides a flowchart of a method for matching an industrial quality inspection scenario with an AI model. This method can be executed by any device, equipment, platform, or cluster of devices with computing capabilities. The present invention does not specifically limit the specific computing device for executing this method; appropriate computing devices can be selected as needed. Figure 1 As shown, the matching method between the industrial quality inspection scenario and the AI ​​model includes at least steps S101-S102.

[0053] In step S101, scene feature information of the industrial quality inspection scenario is obtained.

[0054] The scenario characteristics of industrial quality inspection include the task type of industrial quality inspection, the characteristics of the inspection objects, and the performance requirements of industrial quality inspection for AI models.

[0055] For example, SOP information corresponding to an industrial quality inspection scenario can be obtained, and the task type of the industrial quality inspection scenario can be obtained by analyzing the SOP. For instance, SOP information provided by the factory can be obtained, and the quality inspection requirements of the industrial quality inspection scenario to be matched can be analyzed. The quality inspection requirements can be abstracted and described as the task type of the AI ​​model. For example, the abstract description can be detection plus detection, that is, detecting the target in the image and detecting more subtle targets within the target. For example, detecting the network card on the server motherboard and detecting the chip inside the network card; or detection plus classification, that is, detecting the target in the image and distinguishing the type of the target; for example, detecting the network card on the server motherboard and distinguishing the network card type.

[0056] In another example, when the SOP information provided by the project is not standardized or detailed enough, it is necessary to obtain the SOP information and process flow analysis corresponding to the industrial quality inspection scenario (including each step in the product manufacturing process and quality requirements, etc.). Based on the SOP information and industrial process analysis information corresponding to the industrial quality inspection scenario to be matched, the task type of the quality inspection scenario is described, such as inspection plus inspection and inspection plus classification, etc.

[0057] SOPs are generally understood as describing the standard operating procedures and requirements for a particular event in a standardized format to guide and regulate daily work.

[0058] To identify the characteristics of the quality inspection object, features can be extracted from positive and negative sample pairs. Each positive and negative sample pair includes a positive sample and an image of a qualified object, while the negative sample represents an unqualified object. For example, if the quality inspection object is a network card, the positive sample would be an image of a network card with its chip installed, and the negative sample would be an image of a network card without its chip installed.

[0059] Multiple image features of positive and negative sample pairs of the quality inspection object are extracted by a feature extractor, and these multiple image features are used as the features of the quality inspection object.

[0060] In another example, to make the features of the quality inspection object more relevant to the quality inspection task, after extracting multiple image features in the feature extractor, the process also includes selecting low-order features from the multiple image features (e.g., manually selecting relevant low-order features, such as size, color, etc.) to determine low-order features that are more relevant to the quality inspection task. Then, the selected low-order features are used to filter out high-order features from the multiple image features that are somewhat related to the selected low-order features. The selected low-order features and the filtered high-order features are the features of the quality inspection object.

[0061] For example, a certain number of image samples of quality inspection objects are collected to ensure that each quality inspection information defined in the SOP is represented by at least two images, such as one image of a qualified quality inspection object and one image of a failed quality inspection object. Preferably, each quality inspection information can be represented by five or more images, such as one image of a qualified quality inspection object and four images of failed quality inspection objects, or one qualified quality inspection object combined with the four failed quality inspection objects to form four positive and negative sample images. The collected images are then preprocessed. For example, the completeness of the coverage of the quality inspection information by the collected images is first checked, and images that do not fully represent the quality inspection information are filtered out. Then, based on prior knowledge, the images are processed, registered, and regions of interest (ROIs) related to the quality inspection information are segmented. Deep learning feature extraction is then performed on these ROI images to reduce the number of irrelevant features. Since the extracted features are often still numerous, feature selection is required to obtain relevant feature vectors. In industrial quality inspection scenarios, low-order feature vectors, such as color, texture, and size, can be selected first based on SOP information. For high-order features, distance correlation is used to remove redundant features, and then the chi-square test method is used to remove high-order features with low correlation to low-order features.

[0062] Regarding the performance requirements of AI models in industrial quality inspection, for example, based on prior knowledge, specific performance requirements of AI models in industrial quality inspection scenarios can be summarized, such as one or more of the following: training time requirements, inference latency requirements, and detection accuracy requirements of AI models.

[0063] The task types, characteristics of the inspection objects, and performance requirements of the AI ​​model obtained above constitute the scenario feature information of the industrial quality inspection scenarios to be matched. This scenario feature information allows for rapid identification of similar industrial quality inspection scenarios, avoiding human error in identification.

[0064] In step S102, a target AI model is matched from the scene feature library based on scene feature information. The scene feature library includes multiple scene feature information and AI models corresponding to the multiple scene feature information. The target AI model is the AI ​​model corresponding to the industrial quality inspection scene.

[0065] The scene feature library stores the correspondence between known industrial quality inspection scenes and AI models. For example, the scene feature library includes the identifiers (e.g., numbers) of known industrial quality inspection scenes, scene feature information of known industrial quality inspection scenes, and the corresponding AI models for industrial quality inspection scenes. Based on the scene feature information obtained in step S101, the corresponding AI model is matched from the scene feature library. The matched target AI model is the AI ​​model corresponding to the industrial quality inspection scene. That is, after creating scene feature information for a new scene (i.e., the industrial quality inspection scene to be matched), scene feature matching is performed with the scene feature library. If the match is successful, it is identified as a known scene, and the AI ​​model corresponding to the known scene is matched for the industrial quality inspection scene to be matched.

[0066] For example, the matching rule is as follows: first, compare whether the task type in the scene feature information of the industrial quality inspection scenario to be matched is consistent with the task type in the scene feature information of each AI model; then, compare whether the features of the quality inspection object are similar among the AI ​​models that are consistent with the task type; and finally, match the AI ​​model that meets the performance requirements from the AI ​​models that are similar to the features of the quality inspection object. If all three are successfully matched, the matched target AI model is determined as the AI ​​model corresponding to the industrial quality inspection scenario.

[0067] This application embodiment obtains scene feature information to distinguish different industrial quality inspection scenarios, and then matches the target AI model in a pre-built scene feature library based on the scene feature information, automatically matching a suitable AI model for the scenario, quickly selecting a suitable AI model, while avoiding reliance on the experience of algorithm engineers and reducing the uncertainty of AI model selection.

[0068] In another example, the application embodiment provides a method for matching industrial quality inspection scenarios with AI models, such as... Figure 2 As shown, it also includes steps S201-S202.

[0069] In step S201, if no target AI model is matched from the scene feature library based on scene feature information, then several candidate AI models are matched from the AI ​​model library based on scene feature information.

[0070] In other words, if a known scenario cannot be matched in the scenario feature library for an industrial quality inspection scenario to be matched, then the AI ​​model will continue to be matched from the AI ​​model library.

[0071] The AI ​​model library stores multiple AI models and their corresponding scene feature information. Based on the scene feature information to be matched, several candidate AI models are obtained from the AI ​​model library. The scene feature information corresponding to the candidate AI models is the same as the scene feature information of the industrial quality inspection scene to be matched. If there is only one matched candidate AI model, then that candidate AI model is the target AI model. If there are multiple matched candidate AI models, then step S202 is executed to select the target AI model from the multiple candidate AI models.

[0072] Generally, when a new AI model is added to an AI model library, the types of tasks the model is applicable to are known, but the characteristics and performance of the objects to be tested are unknown and need to be determined through testing.

[0073] The characteristics of the quality inspection objects of the new AI model can be obtained from the training sample set of the AI ​​model. For example, the feature set of the training sample set of the AI ​​model can be extracted, and the features related to quality inspection (including low-order features and high-order features) can be selected from the feature set to form the characteristics of the quality inspection objects of the AI ​​model.

[0074] For the performance specifications of the new AI model, AutoML-HPO was used to test the best performance (including accuracy) of the AI ​​model and the model performance specifications (including training latency, inference latency and detection accuracy) and hyperparameter configurations were recorded in the AI ​​model library.

[0075] The system identifies the task type, applicable quality inspection object characteristics, and performance specifications corresponding to the AI ​​model, enabling a quantitative assessment of the AI ​​model's specifications and capabilities. This shortens the model selection time and improves the accuracy of matching and selection.

[0076] In step S202, the target AI model is determined from several candidate AI models based on the preferred indicators of the candidate AI models.

[0077] Based on the optimization indicators of multiple candidate AI models, the optimal AI model for the industrial quality inspection scenario to be matched is selected from multiple candidate AI models.

[0078] In one example, the preferred metrics for a candidate AI model are related to one or more of the AI ​​model's generalization ability, sample labeling requirements, and AI resource requirements.

[0079] In another example, the selection metric for candidate AI models is determined based on a weighted average of the AI ​​model's generalization ability, sample labeling requirements, and AI resource requirements. The weight of the AI ​​model's generalization ability is greater than the weight of the sample labeling requirements, and the weight of the sample labeling requirements is greater than the weight of the AI ​​resource requirements.

[0080] For example, data augmentation tools are used to process scene feature samples by occlusion, rotation, deformation, and color adjustment to generate test samples. The model's recognition accuracy is tested using these test samples, and the percentage of test samples that pass is denoted as the model's generalization ability index, denoted as m.

[0081] Adjust the bounding boxes of some samples by offsetting and scaling them proportionally, then retrain and check if the model's accuracy decreases. The allowable adjustment ratio for bounding boxes is set as 's', assuming no decrease in accuracy.

[0082] The inference performance of different models was tested using images of the same size and resolution, and under identical hardware conditions. The maximum inference time for each model was set as t. max The current model inference time is set to t. The AI ​​resource requirement metric for the model is calculated as r: r = t / t max .

[0083] Considering the characteristics of industrial quality inspection, the importance of AI model generalization ability > sample labeling requirements > AI resource requirements. Therefore, the weight of AI model generalization ability is set to 0.5, the weight of sample labeling requirements is set to 0.4, and the weight of AI resource requirements is set to 0.1. The optimal index of candidate AI models is p = 0.5m + 0.4s + 0.1r.

[0084] The AI ​​model with the highest performance index among multiple candidate AI models is selected as the target AI model for industrial quality inspection scenarios.

[0085] If no matching AI model is found from the AI ​​model library, a matching failure will be reported.

[0086] In another example, the AI ​​model inference method for industrial quality inspection scenarios provided in this application embodiment further includes updating the mapping relationship between the target AI model and scene feature information to the scene feature library. That is, the AI ​​model matched from the AI ​​model library, the industrial quality inspection scenario, and the corresponding scene feature information are updated to the scene feature library, so that when the same industrial quality inspection scenario is encountered again, the target AI model can be quickly matched through the scene feature library.

[0087] This application also provides an AI model inference method for industrial quality inspection scenarios, such as... Figure 3 As shown, the method includes at least steps S301-S304.

[0088] In step S301, image information of the quality inspection object is collected.

[0089] For example, a camera device can be installed at the quality inspection station to capture images of the products (i.e., the objects to be inspected, such as network cards) at the quality inspection station.

[0090] In step S302, features of the image information are extracted.

[0091] After the original images of the products at the workstations are collected, they first undergo data processing, including image registration, ROI cutout, and target annotation, completing basic feature sample preprocessing. Then, feature extraction is performed on the feature samples to obtain the features of the image information of the quality inspection objects.

[0092] In step S303, the extracted features are compared with the features in the feature sample library to determine whether there are features in the feature sample library that are similar to the extracted features.

[0093] The features in the feature sample library are derived from feature extraction of multiple pairs of positive and negative samples of the quality inspection objects. For example, raw images of products (including images of qualified and unqualified products) are collected at the direct workstations on the production line. After the raw product images at the workstations are collected, they first undergo data processing, including image registration, ROI cutout, and target annotation, completing basic feature sample preprocessing. Then, feature extraction is performed on the feature samples. Features with strong relevance to quality inspection are manually selected from the extracted features. The selected features are the features of the quality inspection objects at that workstation and are saved to form the feature sample library.

[0094] By comparing the extracted features with features in the feature sample library, it can be determined whether the captured image is a difficult example. For example, if the extracted features are compared with features in the feature sample library, and there are similar features in the feature sample library, then the image is a non-difficult example, and the detection accuracy of the target AI model is high. If there are no similar features in the feature sample library, then the image is a difficult example, and the detection accuracy of the target AI model is low.

[0095] In step S304, the image information is used as the input to the AI ​​model, and the quality inspection result is output.

[0096] The AI ​​model includes a feature extraction network and a classifier. After the image information of the quality inspection object is input into the AI ​​model, the image is used as the input of the feature extraction network of the AI ​​model, and the output is the feature vector of the image. This feature vector is used as the input of the classifier, and the output is the quality inspection result, such as 0 and 1, where 0 represents unqualified and 1 represents qualified.

[0097] In another example, the feature extraction of image information in step S302 can be achieved by inputting the image information into the same feature extraction network as the feature extraction network of the AI ​​model and outputting the feature vector of the image information.

[0098] Optionally, the AI ​​model can be a target AI model obtained by matching an industrial quality inspection scenario with an AI model, as provided in this application embodiment.

[0099] When the image features of the quality inspection object are identified as difficult examples through steps S302 and S303, the target AI model can be retrained using these difficult examples to enable it to achieve high-precision detection of such difficult examples. After the target AI model is retrained, the features of the difficult example are added to the feature sample library so that it will not be identified as a difficult example again when encountered.

[0100] Based on the same concept as the aforementioned embodiments of the matching method between industrial quality inspection scenarios and AI models, this application also provides a matching device 400 for industrial quality inspection scenarios and AI models. This matching device 400 includes components for implementing... Figure 1 and 2 The units or modules in the matching method between the industrial quality inspection scenario and the AI ​​model shown.

[0101] Figure 4 This is a schematic diagram of the structure of a device for matching industrial quality inspection scenarios with AI models, provided in an embodiment of this application. This device can be deployed on any computing-capable device, equipment, platform, or equipment cluster, such as a computer or server, to match suitable AI models for industrial quality inspection scenarios. Figure 4 As shown, the matching device 400 for the industrial quality inspection scenario and the AI ​​model includes at least: an acquisition module 401 and a matching module 402; wherein, the acquisition module 401 is used to acquire scenario feature information of the industrial quality inspection scenario, the scenario feature information including the task type of industrial quality inspection, the characteristics of the inspection object, and the performance requirements of industrial quality inspection for the AI ​​model; the matching module 402 is used to match the target AI model from the scenario feature library based on the scenario feature information, the scenario feature library including multiple scenario feature information and AI models corresponding to the multiple scenario feature information respectively, and the target AI model is the AI ​​model corresponding to the industrial quality inspection scenario.

[0102] In one possible implementation, the task types for industrial quality inspection are determined at least based on the standard operating procedures for industrial quality inspection scenarios.

[0103] In another possible implementation, the features of the quality inspection object are obtained by feature extraction from positive and negative sample pairs of the quality inspection object.

[0104] In another possible implementation, the features of the quality inspection object include several low-order features and several high-order features. The low-order features are selected from a set of low-order features, and the high-order features are high-order features related to the low-order features from the set of high-order features. The low-order feature set and the high-order feature set are obtained by feature extraction based on positive and negative sample pairs of the quality inspection object.

[0105] In another possible implementation, performance requirements include one or more of the following: AI model training time requirements, inference latency requirements, and detection accuracy requirements.

[0106] In another possible implementation, the matching module 402 is also used to obtain several candidate AI models from the AI ​​model library based on the scene feature information when no target AI model is matched from the scene feature library based on the scene feature information. The AI ​​module library includes multiple AI models and scene feature information corresponding to each of the multiple AI models.

[0107] The matching device 400 for industrial quality inspection scenarios and AI models also includes a determination module 403, which is used to determine the target AI model from a number of candidate AI models based on the preferred indicators of candidate AI models.

[0108] In another possible implementation, the preferred metrics for candidate AI models are related to one or more of the following: the generalization ability of the AI ​​model, sample labeling requirements, and AI resource requirements.

[0109] In another possible implementation, the selection metric for candidate AI models is determined based on a weighted average of the AI ​​model's generalization ability, sample labeling requirements, and AI resource requirements. The weight of the AI ​​model's generalization ability is greater than the weight of the sample labeling requirements, and the weight of the sample labeling requirements is greater than the weight of the AI ​​resource requirements.

[0110] In another possible implementation, the matching device 400 between the industrial quality inspection scenario and the AI ​​model also includes an update module 404, which updates the mapping relationship between the target AI model and the scenario feature information to the scenario feature library.

[0111] The industrial quality inspection scenario and AI model matching device 400 according to the embodiments of this application can correspondingly execute the method described in the embodiments of this application, and the above and other operations and / or functions of each module in the industrial quality inspection scenario and AI model matching device 400 are respectively for implementing Figure 1 and 2For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0112] Based on the same concept as the aforementioned embodiments of AI model inference methods for industrial quality inspection scenarios, this application also provides an AI model inference device 500 for industrial quality inspection scenarios. This AI model inference device 500 includes components for implementing... Figure 3 The units or modules shown are the steps in the AI ​​model inference method for industrial quality inspection scenarios.

[0113] Figure 5 This is a schematic diagram of the structure of an AI model inference device for industrial quality inspection scenarios provided in an embodiment of this application. This device can be deployed on any computing-capable device, equipment, platform, or cluster of devices, such as a computer or server, to perform inference of quality inspection results in industrial quality inspection scenarios, such as... Figure 5 As shown, the AI ​​model inference device 500 for industrial quality inspection scenarios includes at least an acquisition module 501, an extraction module 502, a comparison module 503, and an inference module 504. The acquisition module 501 is used to acquire image information of the quality inspection object; the extraction module 502 is used to extract features from the image information; the comparison module 503 is used to compare the extracted features with features in a feature sample library to determine whether there are features in the feature sample library that are similar to the extracted features; and the inference module 504 is used to take the image information as input to the AI ​​model and output the quality inspection results.

[0114] In one possible implementation, the features in the feature sample library are obtained by feature extraction from multiple pairs of positive and negative samples of the quality inspection objects.

[0115] In another possible implementation, the AI ​​model is the target AI model obtained by matching the industrial quality inspection scenario provided by the embodiments of this application with the AI ​​model.

[0116] The AI ​​model inference device 500 for industrial quality inspection scenarios according to the embodiments of this application can correspondingly execute the methods described in the embodiments of this application, and the above and other operations and / or functions of each module in the AI ​​model inference device 500 for industrial quality inspection scenarios are respectively for implementing Figure 3 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0117] Based on the same concept as the aforementioned method embodiments, this application also provides a computing device, which includes at least a processor and a memory. The memory stores a program, and when the processor reads the program, it can achieve... Figure 1-3 The units or modules of each step in the method shown.

[0118] Figure 6 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application.

[0119] like Figure 6 As shown, the computing device 600 includes at least one processor 601, a memory 602, and a communication interface 603. The processor 601, memory 602, and communication interface 603 are communicatively connected, which can be achieved via a wired (e.g., bus) or wireless connection. The communication interface 603 is used to receive data sent by other devices (e.g., image information acquired by an acquisition device); the memory 602 stores computer instructions, and the processor 601 executes these computer instructions to perform the method described in the foregoing method embodiments.

[0120] It should be understood that in the embodiments of this application, the processor 601 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0121] The memory 602 may include read-only memory and random access memory, and provides instructions and data to the processor 601. The memory 602 may also include non-volatile random access memory.

[0122] The memory 602 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0123] It should be understood that the computing device 600 according to the embodiments of this application can perform the implementation of the embodiments of this application. Figure 1-3 The method shown is described in detail above, and will not be repeated here for the sake of brevity.

[0124] Embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein when the computer instructions are executed by a processor, the aforementioned method is implemented.

[0125] An embodiment of this application provides a chip including at least one processor and an interface, wherein the at least one processor determines program instructions or data through the interface; the at least one processor is used to execute the program instructions to implement the method mentioned above.

[0126] Embodiments of this application provide a computer program or computer program product that includes instructions that, when executed, cause a computer to perform the methods mentioned above.

[0127] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0128] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, processor-executed software modules, or a combination of both. The software modules can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.

[0129] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for matching industrial quality inspection scenarios with AI models, characterized in that, include: The scenario feature information of the industrial quality inspection scenario is obtained. The scenario feature information includes the task type of industrial quality inspection, the characteristics of the inspection object, and the performance requirements of the industrial quality inspection for the AI ​​model. The task type of industrial quality inspection is determined at least based on the standard operating procedure of the industrial quality inspection scenario. The task type includes either inspection plus inspection or inspection plus classification. The characteristics of the inspection object are obtained by feature extraction from positive and negative sample pairs of the inspection object. The performance requirements include one or more of the following: training time requirements, inference latency requirements, and detection accuracy requirements of the AI ​​model. The target AI model is matched from the scene feature library based on the scene feature information. The scene feature library includes multiple scene feature information and AI models corresponding to the multiple scene feature information respectively. The target AI model is the AI ​​model corresponding to the industrial quality inspection scene. If no target AI model is matched from the scene feature library based on the scene feature information, then several candidate AI models are matched from the AI ​​model library based on the scene feature information. The AI ​​module library includes multiple AI models and scene feature information corresponding to each of the multiple AI models. The target AI model is determined from the candidate AI models based on the preferred indicators of the candidate AI models.

2. The method according to claim 1, characterized in that, The features of the quality inspection object include several low-order features and several high-order features. The several low-order features are selected from a set of low-order features, and the several high-order features are high-order features in the set of high-order features that are related to the low-order features. The low-order feature set and the high-order feature set are obtained by feature extraction from positive and negative sample pairs of the quality inspection object.

3. The method according to claim 1, characterized in that, The preferred metrics for the candidate AI model are related to one or more of the following: the AI ​​model's generalization ability, sample labeling requirements, and AI resource requirements.

4. The method according to claim 1, characterized in that, The selection criteria for the candidate AI model are determined based on a weighted average of the AI ​​model's generalization ability, sample labeling requirements, and AI resource requirements. The weight of the AI ​​model's generalization ability is greater than the weight of the sample labeling requirements, and the weight of the sample labeling requirements is greater than the weight of the AI ​​resource requirements.

5. The method according to any one of claims 1-4, characterized in that, Also includes: The mapping relationship between the target AI model and the scene feature information is updated in the scene feature library.

6. An AI model inference method for industrial quality inspection scenarios, characterized in that, include: Collect image information of the quality inspection objects; Extract the features of the image information; The feature is compared with the features in the feature sample library to determine that there are features similar to the feature in the feature sample library; The image information is used as input to the AI ​​model, and the output is the quality inspection result. The AI ​​model is the target AI model matched based on the method described in any one of claims 1-5.

7. The method according to claim 6, characterized in that, The features in the feature sample library are obtained by extracting features from multiple pairs of positive and negative samples of the quality inspection object.

8. A matching device for industrial quality inspection scenarios and AI models, characterized in that, include: The acquisition module is used to acquire scene feature information of industrial quality inspection scenarios. The scene feature information includes the task type of industrial quality inspection, the characteristics of the inspection objects, and the performance requirements of the industrial quality inspection for the AI ​​model. The task type of industrial quality inspection is determined at least based on the standard operating procedures of the industrial quality inspection scenario. The task type includes either inspection plus inspection or inspection plus classification. The characteristics of the inspection objects are obtained by feature extraction from positive and negative sample pairs of the inspection objects. The performance requirements include one or more of the following: training time requirements, inference latency requirements, and detection accuracy requirements of the AI ​​model. The matching module is used to match a target AI model from a scene feature library based on the scene feature information. The scene feature library includes multiple scene feature information and AI models corresponding to the multiple scene feature information respectively. The target AI model is the AI ​​model corresponding to the industrial quality inspection scene. If no target AI model is matched from the scene feature library based on the scene feature information, then several candidate AI models are matched from the AI ​​model library based on the scene feature information. The AI ​​module library includes multiple AI models and scene feature information corresponding to each of the multiple AI models. The target AI model is determined from the candidate AI models based on the preferred indicators of the candidate AI models.

9. The apparatus according to claim 8, characterized in that, The features of the quality inspection object include several low-order features and several high-order features. The several low-order features are selected from a set of low-order features, and the several high-order features are high-order features in the set of high-order features that are related to the low-order features. The low-order feature set and the high-order feature set are obtained by feature extraction from positive and negative sample pairs of the quality inspection object.

10. The apparatus according to claim 8, characterized in that, The preferred metrics for the candidate AI model are related to one or more of the following: the AI ​​model's generalization ability, sample labeling requirements, and AI resource requirements.

11. The apparatus according to claim 8, characterized in that, The selection criteria for the candidate AI model are determined based on a weighted average of the AI ​​model's generalization ability, sample labeling requirements, and AI resource requirements. The weight of the AI ​​model's generalization ability is greater than the weight of the sample labeling requirements, and the weight of the sample labeling requirements is greater than the weight of the AI ​​resource requirements.

12. The apparatus according to any one of claims 8-11, characterized in that, Also includes: The update module is used to update the mapping relationship between the target AI model and the scene feature information to the scene feature library.

13. An AI model inference device for industrial quality inspection scenarios, characterized in that, include: The acquisition module is used to acquire image information of the quality inspection objects; The extraction module is used to extract features from the image information; The comparison module is used to compare the feature with features in the feature sample library to determine that there are features similar to the feature in the feature sample library; The inference module is used to take the image information as input to the AI ​​model and output the quality inspection result, wherein the AI ​​model is the target AI model matched based on the method described in any one of claims 1-5.

14. The apparatus according to claim 13, characterized in that, The features in the feature sample library are obtained by extracting features from multiple pairs of positive and negative samples of the quality inspection object.

15. A computing device, comprising a memory and a processor, characterized in that, The memory stores executable code, and the processor executes the executable code to implement the method according to any one of claims 1-5 and / or any one of claims 6-7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed in a computer, it causes the computer to perform the method described in any one of claims 1-5 and / or any one of claims 6-7.

Images