Visual detection method and device and computer storage medium

Abstract

The invention discloses a visual detection method and device and a computer storage medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining additional information of a to-be-detected image, and determining a detection scene of the to-be-detected image according to the additional information; according to the detection scene and the to-be-detected image, selecting a target vision algorithm matched with the to-be-detected image; processing the to-be-detected image through the target vision algorithm, and determining a detection result of the to-be-detected image; and outputting the detection result, and executing corresponding automatic operation according to the detection result, so that the product detection efficiency and the product quality in the production process are improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a visual detection method, device and computer storage medium. Background Art

[0002] In modern industrial production, quality inspection is a key link in controlling product quality, and its accuracy directly affects the stable operation of the automated control system. However, many companies still use traditional manual visual inspection methods. This method is inefficient and relies on the subjective judgment of inspectors, which easily leads to inconsistent implementation of inspection standards, and thus fails to provide stable and reliable quality feedback for the automated control system. The lack of accurate quality inspection feedback limits the decision-making and adjustment capabilities of the automated control system, hinders the automated control of the production process, and also affects the stability of product quality. Summary of the invention

[0003] The main purpose of this application is to provide a visual inspection method, equipment and computer storage medium, aiming to solve the technical problem of how to provide stable and reliable quality feedback to achieve production process automation.

[0004] To achieve the above object, the present application provides a visual inspection method, which includes: Acquire additional information of the image to be detected, and determine the detection scene of the image to be detected according to the additional information; According to the detection scene and the image to be detected, selecting a target vision algorithm adapted to the image to be detected; Processing the image to be detected by the target vision algorithm to determine a detection result of the image to be detected; The detection result is output, and corresponding automated operations are performed according to the detection result.

[0005] In one embodiment, the step of obtaining additional information of the image to be detected and determining the detection scene of the image to be detected according to the additional information includes: Based on the additional information, obtaining an identification of a shooting device of the image to be detected; Determine the installation location or preset activity range of the shooting device according to the shooting device identifier; Querying the processing plan information of the installation position or the preset activity range to determine the processing link the photographing device is currently in; According to the processing task corresponding to the processing link, the detection scene of the image to be detected is determined.

[0006] In one embodiment, after the step of obtaining the additional information of the image to be detected and determining the detection scene of the image to be detected according to the additional information, the method further includes: Selecting an algorithm set according to the detection scenario; Processing the image to be detected respectively by using a plurality of the target vision algorithms in the algorithm set to generate a plurality of intermediate results, and determining the confidence level of each of the intermediate results; The multiple intermediate results are fused according to the confidence level to generate a final detection result of the image to be detected.

[0007] In one embodiment, the step of selecting a target vision algorithm adapted to the image to be detected according to the detection scene and the image to be detected includes: Selecting a set of candidate algorithms according to the detection task of the detection scenario; Acquire image feature parameters of the image to be detected, and determine the applicable feature range of each visual algorithm in the candidate algorithm set; The image feature parameters are matched with the applicable feature range, and the visual algorithm with the highest matching degree in the candidate algorithm set is selected as the target visual algorithm.

[0008] In one embodiment, the step of processing the image to be detected by the target vision algorithm to determine the detection result of the image to be detected includes: Calculate the gradient magnitude and direction of each pixel in the image to be detected, and mark the pixel whose gradient magnitude is greater than a preset threshold as a feature point; Performing cluster analysis on the feature points, and determining key features of the image to be detected according to the clustering results; The key features of the image to be detected are compared with the target feature template, and the detection result of the image to be detected is determined according to the comparison result.

[0009] In one embodiment, before the step of comparing the key features of the image to be detected with the target feature template and determining the detection result of the image to be detected according to the comparison result, the method further includes: Extracting edge information of the image to be detected, and locating a target area of ​​the target object based on the edge information; Using a pre-trained target classification model to classify and identify the target area, and determine the target object in the image to be detected; According to the identified target object, a target feature template corresponding to the image to be detected is determined.

[0010] In one embodiment, the step of comparing the key features of the image to be detected with the target feature template and determining the detection result of the image to be detected according to the comparison result further includes: Converting the key feature and the target feature template into a first feature vector and a second feature vector respectively, calculating the similarity between the first feature vector and the second feature vector, and judging whether the target object in the image to be detected meets a preset quality standard or state condition based on the similarity calculation result; If the target object meets the preset quality standard or status condition, the image to be detected is qualified; If the target object does not meet the preset quality standard or status condition, the image to be detected is unqualified, and detection information of the target object is obtained according to the processing result of the target vision algorithm, and the detection information includes defect information or abnormality information; The conclusion on whether the product is qualified or not is integrated with the test information to generate the final test result.

[0011] In one embodiment, the step of outputting the detection result and performing corresponding automated operations according to the detection result includes: When the inspection result of the image to be inspected is qualified, a qualified signal is output to the automatic control system, and the qualified product corresponding to the image to be inspected is transmitted to the next production process or storage area; If the detection result of the image to be detected is unqualified, an alarm signal is issued or an alarm prompt is output, and the detection result is classified, and the corresponding equipment is controlled to perform corresponding automatic operations according to the classification result.

[0012] An embodiment of the present application also provides a visual inspection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the visual inspection method described above.

[0013] An embodiment of the present application also provides a computer storage medium, which is a computer-readable storage medium. A computer program is stored on the computer storage medium. When the computer program is executed by a processor, the steps of the visual inspection method described above are implemented.

[0014] The embodiment of the present application discloses a visual inspection method, which obtains additional information of an image to be inspected, and determines the inspection scene of the image to be inspected according to the additional information; selects a target vision algorithm adapted to the image to be inspected according to the inspection scene and the image to be inspected; processes the image to be inspected by the target vision algorithm to determine the inspection result of the image to be inspected; outputs the inspection result, and performs corresponding automated operations according to the inspection result, thereby improving the product inspection efficiency and product quality in the production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of the first embodiment of the visual inspection method involved in the embodiment scheme of the present application; Figure 2 This is a flow chart of a second embodiment of a visual inspection method according to an embodiment of the present application; Figure 3 This is a flow chart of a third embodiment of the visual inspection method according to the embodiment of the present application; Figure 4 This is a flow chart of a fourth embodiment of the visual inspection method according to the embodiment of the present application; Figure 5 This is a flow chart of a fifth embodiment of a visual inspection method according to an embodiment of the present application; Figure 6 This is a schematic diagram of the structure of the visual inspection equipment involved in the embodiment of the present application.

[0016] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0017] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0018] In modern industrial production, quality inspection is a key link in controlling product quality, and its accuracy directly affects the stable operation of the automated control system. However, many companies still use traditional manual visual inspection methods. This method is inefficient and relies on the subjective judgment of inspectors, which easily leads to inconsistent implementation of inspection standards, and thus fails to provide stable and reliable quality feedback for the automated control system. The lack of accurate quality inspection feedback limits the decision-making and adjustment capabilities of the automated control system, hinders the automated control of the production process, and also affects the stability of product quality.

[0019] In order to solve the above-mentioned defects existing in the related art, an embodiment of the present application proposes a visual inspection method, which obtains additional information of the image to be inspected, and determines the inspection scene of the image to be inspected according to the additional information; selects a target vision algorithm adapted to the image to be inspected according to the detection scene and the image to be inspected; processes the image to be inspected by the target vision algorithm to determine the inspection result of the image to be inspected; outputs the inspection result, and performs corresponding automated operations according to the inspection result, thereby improving the product inspection efficiency and product quality in the production process.

[0020] It should be noted that the execution subject of this embodiment may be an automated control system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a visual inspection device capable of realizing the above functions, etc. The following takes an automated control system (hereinafter referred to as the "system") as an example to illustrate this embodiment and the following embodiments.

[0021] The visual inspection method of the first embodiment proposed in this application, please refer to Figure 1 The method comprises steps S10 to S40: Step S10: Acquire additional information of the image to be detected, and determine the detection scene of the image to be detected according to the additional information.

[0022] The images to be inspected refer to the image data of objects that need to undergo quality inspection, target detection and other operations. These objects can be products, components on a production line, materials in a warehouse, etc.

[0023] The additional information includes the shooting time, shooting location, associated production batch, product number, and identification of the shooting device used to shoot the image to be detected, which can be used to identify the source of the image to be detected, the detection scene, and the detection object.

[0024] The image acquisition device itself has the function of recording the shooting time, shooting location and shooting device identification. Therefore, in the process of image acquisition, additional information such as shooting time, shooting location and shooting device identification will be automatically embedded in the metadata of the image file when the image to be detected is captured.

[0025] In addition, in environments such as production lines or warehouses, there is usually a production management system to track and manage product production batches, product numbers and other information. When the image acquisition device triggers image acquisition, the image device can obtain the production batch, product number and other information of the current object through the interface or communication protocol with the production management system, and use it as part of the additional information.

[0026] After receiving the image to be detected, the system will first pre-process the image to be detected, such as denoising, enhancement, cropping, and adjusting brightness and contrast, etc. Then, the system will extract additional information and perform recognition by parsing the metadata corresponding to the image to be detected.

[0027] The inspection scenarios are pre-divided based on factors such as production process, product type, quality standard or inspection requirements. The inspection scenarios reflect the inspection tasks that need to be performed on the current production line. For example, in a large factory, the production line will be divided into different inspection scenarios, each of which is responsible for inspecting different parts of a product or different quality indicators.

[0028] It should be noted that in the actual production environment, the processing plan, product type or inspection standard of the production line may be changed according to market demand, production scheduling or quality improvement. Therefore, if the additional information of the image to be inspected is simply fixedly mapped directly to the target vision algorithm, this fixed mapping method will be difficult to adapt to the ever-changing inspection needs on the production line.

[0029] Therefore, this embodiment first determines the detection scene of the image to be detected based on the additional information of the image to be detected, and then selects a target visual algorithm adapted to the detection scene from a preset visual algorithm library based on the determined detection scene, thereby providing accurate and efficient detection results for specific detection scenes.

[0030] In an optional implementation scheme, a description vector is constructed in advance for each detection scene, and the description vector represents the additional information features under the corresponding detection scene. Specifically, by statistically analyzing the additional information in the historical image data under a large number of different detection scenes, the feature combination corresponding to different detection scenes is determined to obtain a description vector. Then, one or more content combinations of the additional information in the image to be detected, such as shooting location, production batch, product number, and shooting equipment identification, are subjected to feature extraction and encoding processing to convert the additional information into a feature vector. For example, the production batch and product number are mapped to the vector space by hash coding. Then, the similarity between the feature vector of the additional information in the image information to be detected and the description vector corresponding to each detection scene is calculated by using methods such as cosine similarity and Euclidean distance. Finally, the scene with the highest similarity result is used as the monitoring scene corresponding to the image to be detected.

[0031] In the above implementation scheme, a description vector is constructed based on the additional information of historical image data under a large number of different detection scenarios, so that the additional information features corresponding to the detection scene can effectively reflect the key characteristics of each scene. And through statistical analysis of historical image data under a large number of different detection scenarios, the common features and laws under the same detection scene can be excavated, thereby providing an accurate reference basis for the detection scene recognition of the image to be detected. In addition, by extracting and encoding one or more content combinations of additional information in the image to be detected, the feature vector corresponding to the additional information can more accurately reflect the characteristics of the image to be detected, which helps to improve the accuracy of detection scene recognition.

[0032] In another optional scheme, a large amount of historical image data with annotated detection scenes and corresponding additional information can also be collected as training data to train the classification model. In the training process of the model, the mapping relationship between the additional information and the detection scene can be learned. When a new image to be detected is collected, the additional information of the image to be detected is input into the trained classification model, thereby directly outputting the predicted detection scene. This embodiment can automatically learn the mapping relationship between the additional information and the detection scene, and has a good effect on scene classification in a large-scale production environment. And with the increase of training data and the continuous optimization of the classification model, the accuracy of the classification will continue to improve, thereby being able to adapt to the dynamic changes in the detection requirements on the production line.

[0033] In another optional scheme, a knowledge graph is constructed, and the various contents in the additional information and the preset detection scenes are used as nodes, and the relationship between the additional information and the preset detection scenes is used as edges. For example, the association between production batches and product types, detection scenes, the association between shooting positions and production line links, and detection tasks, etc. After obtaining the additional information of the image to be detected, query and reasoning are performed in the knowledge graph. By traversing the nodes and edges in the knowledge graph, the detection scenes associated with the additional information are found. This implementation scheme can use domain knowledge to visualize and structure the complex relationship between additional information and detection scenes, so as to perform multi-dimensional reasoning and analysis. At the same time, the knowledge graph has good scalability, and when new detection scenes or associations appear in the production environment or production line, the knowledge graph can be easily updated and maintained.

[0034] In addition, the detection scene corresponding to the additional information can be determined by accessing a pre-built detection scene mapping table, which stores the correspondence between the additional information and the detection scene. For example, when the shooting location in the additional information is accurate to specific coordinates or area names, the system can directly find the detection scene that meets the corresponding production process from the detection scene mapping table according to the production process corresponding to the shooting location. Or, according to the identification of the shooting device that shoots the image to be detected, first determine the location information of the shooting device, and then determine the detection scene of the image to be detected according to the processing plan corresponding to the location information of the shooting device. Although this solution can match quickly, when some parameters in the additional information, such as production batches, change slightly, it may not be possible to find a completely matching result in the detection scene mapping table, resulting in the inability to accurately identify the detection scene. In addition, the detection scene mapping table requires long-term manual maintenance, and the maintenance cost will increase significantly with changes in the production process and the emergence of new detection scenes.

[0035] Therefore, the implementation scheme of constructing description vectors and using similarity calculation to identify detection scenarios will have high flexibility and adaptability. This scheme can not only adapt to subtle changes in parameters, but also continuously optimize the construction of feature vectors through a self-learning mechanism. When encountering new combinations of additional information, users can be automatically reminded to add new detection scenarios, or feature vectors can be continuously learned and updated through a self-learning mechanism. This self-learning mechanism makes the entire process more automated, reduces the need for manual intervention, and improves the adaptability and flexibility of the system.

[0036] The solution based on classification model recognition of detection scenes also has high flexibility and applicability. The classification model can automatically learn and adapt to the mapping relationship between additional information and detection scenes, so that the classification model can capture the subtle feature differences in different detection scenes, so as to quickly and accurately output the corresponding detection scene when facing new images to be detected. In addition, with the continuous accumulation of training data and the continuous optimization of the classification model, the recognition accuracy of the classification model will gradually improve, and it can better cope with the dynamic changes in detection requirements on the production line.

[0037] The knowledge graph has good scalability. When the production environment or detection requirements change, the graph can be easily updated and maintained. If you need to introduce new detection scenarios, update production processes, or adjust detection tasks, you only need to add new nodes to the knowledge graph or update the relationship between nodes. This structured knowledge representation method can also enable the system to flexibly adapt to dynamic changes in the production environment while reducing maintenance costs.

[0038] In this embodiment, the image to be detected is acquired by an image acquisition device. The image acquisition device can be selected and deployed according to the specific application scenario or detection requirements. Image acquisition devices include but are not limited to industrial cameras, fixed cameras, and AGV (Automated Guided Vehicle) mounted cameras. Industrial cameras have the characteristics of high resolution, high frame rate, high stability, etc., and are suitable for industrial detection scenarios with high requirements for accuracy and speed. Fixed cameras are usually installed in specific locations for continuous detection of objects in fixed areas. The camera mounted on the AGV can capture images of objects in different positions as the AGV moves, increasing the flexibility and coverage of the detection. These image acquisition devices are connected to the system by wire or wireless means to ensure that when the object to be detected reaches the detection point, the image acquisition device can automatically or according to instructions trigger image acquisition.

[0039] For example, in the scenario of product quality inspection, image acquisition can be performed by means of production line triggering photography, robot automatic photography, etc. In the scenario of warehouse material inspection, image acquisition can be performed by means of fixed cameras, AGV-mounted cameras, etc.

[0040] In this embodiment, different detection scenes correspond to different detection tasks and quality standards. Only by determining the detection scene of the image to be detected can the adapted visual algorithm be called and the detection standard of the image to be detected be determined to ensure the accuracy of the detection result corresponding to the image to be detected.

[0041] Step S20: selecting a target vision algorithm adapted to the image to be detected according to the detection scene and the image to be detected.

[0042] Different inspection scenarios have different objects to be inspected, inspection tasks, and inspection standards. For example, the inspection task in the product quality inspection scenario is to detect whether there are scratches, cracks, stains, and other defects on the surface of the object to be inspected on the production line, and the inspection task in the warehouse material inspection scenario may be to detect whether the warehouse material is empty. Therefore, in order to accurately analyze and process the inspection image, it is necessary to select an adaptive target vision algorithm based on the characteristics of the inspection scenario.

[0043] In an optional implementation, step S20 may include steps S210 to S230: Step S210: selecting a set of candidate algorithms according to the detection task of the detection scene.

[0044] Step S220: Acquire image feature parameters of the image to be detected, and determine the applicable feature range of each visual algorithm in the candidate algorithm set.

[0045] Step S230: Match the image feature parameters with the applicable feature range, and select the visual algorithm with the highest matching degree in the candidate algorithm set as the target visual algorithm.

[0046] In this embodiment, the functional description and applicable scene information of the visual algorithm are stored in advance, and the corresponding relationship between the detection scene and the visual algorithm is established. After the detection scene of the image to be detected is determined, according to the detection task corresponding to the detection scene, such as the type, position or defect of the detection target, a suitable candidate algorithm set is selected from the algorithm library by matching and searching.

[0047] Next, the image feature parameters of the image to be detected are obtained, including the brightness, contrast, texture, edge and other information of the image to be detected, which are used to describe the characteristic attributes of the image to be detected. At the same time, the applicable feature range of each visual algorithm in the candidate algorithm set is determined, that is, the value range of the image feature parameters that each visual algorithm can effectively process.

[0048] The acquisition of image feature parameters can be achieved through image processing technology, including extracting color features using color histograms, extracting texture features using local binary patterns or gray-level co-occurrence matrices, extracting edge features through Canny edge detection, and extracting shape features through contour detection. In addition, the applicable feature range of each visual algorithm is determined based on the performance requirements of the visual algorithm. For example, deep learning algorithms have higher requirements for the texture and edge features of images, while traditional image processing algorithms rely more on the brightness and contrast of images.

[0049] In an optional implementation scheme, an image data set is collected, and the image data set contains a variety of image feature data, such as different brightness, contrast, texture complexity, edge clarity, etc. Then the image data set is annotated to determine the image feature parameters of each image (such as brightness value range, contrast value range, texture complexity level, edge clarity level, etc.). For each visual algorithm, training and testing are performed on the image data set, and the performance of different visual algorithms under different feature parameters is recorded, including evaluation indicators such as accuracy, recall rate, and F1 score. Finally, statistical tools are used to perform a correlation analysis on the performance of the visual algorithm and the image feature parameters, determine the feature parameter interval where the algorithm performance reaches or exceeds the predetermined threshold, and confirm it as the applicable feature range of the visual algorithm.

[0050] Since different visual algorithms have different processing effects on the images to be detected, the visual algorithms in the candidate algorithm set may not be completely applicable to the current image to be detected. Therefore, it is necessary to calculate the similarity between the image feature parameters of the image to be detected and the applicable feature range of the visual algorithm, so as to select the most suitable algorithm for the current image to be detected from the candidate algorithm set.

[0051] In another optional embodiment, step S20 may further include steps S240 to S250: Step S240: according to the detection scene, searching for a visual algorithm identifier corresponding to the detection scene from a preset mapping table, and the detection scene may be associated with one or more of the visual algorithm identifiers.

[0052] Step S250: Select and apply the target vision algorithm corresponding to the vision algorithm identifier from the algorithm library.

[0053] The mapping table stores the mapping relationship between different detection scenarios and visual algorithm identifiers. Considering that multiple visual algorithms may need to be combined in different detection scenarios to meet the quality detection standards. Therefore, each detection scenario can be associated with one or more visual algorithm identifiers. Then the system can select the target visual algorithm corresponding to the visual algorithm identifier from the algorithm library based on the visual algorithm identifier. In this way, the system can ensure that the selected target visual algorithm can meet the detection task of the corresponding detection scenario.

[0054] The algorithm library stores multiple different types of visual algorithms, each of which has its own applicable detection scenarios and detection targets. For example, the edge detection algorithm is suitable for identifying the outline of an object and is often used to detect defects such as cracks, scratches, and stains on the surface of a product. The threshold segmentation algorithm can separate the target object from the background in the image, making it easier to analyze the characteristics of the target object.

[0055] Exemplarily, if the inspection task corresponding to the inspection scene is to detect whether the size of the product meets the quality standards, then a size measurement algorithm based on feature matching, template matching or deep learning can be selected. Specifically, when using the feature matching algorithm, the feature points in the image to be inspected are first extracted, and then matched with the pre-stored standard feature points of the same target object, and the size of the product is determined based on the matching results. Whether it meets the quality standards. When using the template matching algorithm, the image to be inspected is compared with the image template of the standard size to find the best matching area, and then determine whether the product size meets the quality standards. When selecting a size measurement algorithm based on deep learning, the image to be inspected can be input into a pre-trained neural network model, and the neural network model can directly output the detection results.

[0056] In another optional implementation scheme, for a detection scene associated with multiple visual algorithm identifiers, the execution order of the associated multiple visual algorithm identifiers can be pre-set according to the detection task of the detection scene or the processing logic of the image to be processed. Then, based on the execution order, the system can call the target visual algorithm corresponding to the visual algorithm identifier from the algorithm library in sequence to process the image to be detected. Each target visual algorithm can analyze and calculate the image to be processed according to its own function, so as to obtain a more accurate detection result.

[0057] For example, if the inspection task corresponding to the inspection scene is to detect whether the product has defects, multiple algorithms such as edge detection and threshold segmentation can be selected for defect detection, and then the execution order of multiple defect detection algorithms associated with defect detection can be pre-set according to the image processing order for defect detection to ensure efficient and accurate defect detection.

[0058] Step S30: Processing the image to be detected by the target vision algorithm to determine the detection result of the image to be detected.

[0059] In this embodiment, the system first inputs the preprocessed image to be detected into the selected target vision algorithm to extract key features in the image to be detected, and the key features include but are not limited to texture features, color features and shape features, such as shape, texture, color distribution and edge information. At the same time, the target object in the image to be detected can be identified by the target detection algorithm or the pre-trained target classification model, and then the key features are compared with the target feature template corresponding to the target object. Through the comparison results, it is judged whether the target object in the image to be detected meets the preset quality standard or state condition, thereby obtaining the detection result of the image to be detected.

[0060] While judging whether the comparison results meet the preset quality standards or status conditions, the detection information of the target object in the image to be detected will be output based on the processing results of the target vision algorithm on the image to be detected. The detection information includes defect information or abnormal information, such as surface defects, dimensional deviations, color differences, position, size, category, and empty status of the product.

[0061] The detection result includes the conclusion of whether the image to be detected is qualified, and the detection information of the target object in the image to be detected.

[0062] In an optional implementation scheme, a pre-trained quality detection model is used to automatically determine the target visual algorithm that is suitable for the image to be detected, and the image to be detected is processed to obtain the detection result of the image to be detected. The quality detection model is trained based on a large amount of annotated image data. Through the detection scene of the image data and the detection task corresponding to the detection scene, the mapping relationship between different image data and the visual algorithm can be learned, so that the appropriate target visual algorithm can be selected when processing the image to be detected to achieve quality detection of the image to be detected.

[0063] Exemplarily, the visual algorithms include target detection algorithms, image classification algorithms, image segmentation algorithms, etc. Among them, target detection algorithms include YOLO (You Only Look Once, a target detection algorithm based on deep learning), SSD (Single Shot MultiBox Detector), Faster R-CNN (Faster Region-based Convolutional Neural Networks), etc., which are used to detect product defects or material locations. Image classification includes ResNet (Residual Network), VGG (Visual Geometry Group Network), Inception, etc., which are used to identify product types or material types. Image segmentation includes U-Net (Convolutional Neural Network), Mask R-CNN (Mask Region-based Convolutional Neural Networks), etc., which are used to accurately segment defective areas or material contours.

[0064] After completing the training of the quality inspection model, in the model deployment phase, the deployment environment needs to choose cloud deployment or edge deployment based on actual production needs.

[0065] Cloud deployment is to deploy the trained quality inspection model to the cloud server. This method is suitable for scenarios with large data volumes and high computing resource requirements. In large-scale industrial production or logistics warehousing environments, a large amount of image data is generated every day. Processing and analyzing these image data requires powerful computing power and storage resources. Cloud deployment can take advantage of the cloud server cluster to meet the needs of large-scale data processing.

[0066] Edge deployment is to deploy the quality inspection model on devices close to the data source, such as smart cameras on the production line, edge computing gateways in the warehouse, etc. This deployment method is suitable for scenarios with high real-time requirements and high data security requirements. In some automated production lines with extremely high production efficiency requirements, it is necessary to conduct quality inspections on products and make decisions in a very short time. Edge deployment can avoid the delay in transmitting the image data to be inspected to the cloud and achieve real-time inspection.

[0067] In another optional implementation, the historically collected image data may be annotated, and the annotated content includes defect type, material type, location information, etc. The annotated content includes information such as the name, color, shape, size, defect type, and defect location of the object to be detected in the image data. For example, an annotation tool is used to mark the defect area in the image data, and the defect type, such as scratches, cracks, stains, etc., is annotated, and the defect location is recorded at the same time.

[0068] The annotated image data includes different objects to be inspected, different inspection scenes, multiple defect types or abnormal types, and image data in normal state, which can be used for training and performance evaluation of quality inspection models. Based on the annotated image data, the quality inspection model can learn the categories and features of different objects to be inspected in the image data, thereby realizing functions such as quality inspection and target detection of the images to be inspected, and obtaining corresponding inspection results.

[0069] Furthermore, the trained quality inspection model can be optimized. The optimization of the quality inspection model mainly includes compression, acceleration and other operations on the quality inspection model to improve the reasoning speed and efficiency of the quality inspection model.

[0070] Specifically, model compression can be achieved by reducing the number of parameters in the quality detection model and reducing the complexity of the quality detection model. For example, pruning technology can be used to remove parameters in the quality detection model that have little impact on the detection results, or quantization technology can be used to convert the parameters of the quality detection model from high-precision data types to low-precision data types. The calculation speed of the quality detection model can be improved by optimizing the calculation structure of the quality detection model and using hardware acceleration libraries, such as using GPU (Graphics Processing Unit) acceleration libraries to perform parallel calculations on the quality detection model.

[0071] When the image to be inspected is input, the quality inspection model first analyzes the feature information of the image to be inspected, such as texture, color distribution, shape, etc., and then automatically matches and selects the target vision algorithm suitable for the current image to be inspected. Next, the image to be inspected is processed using the selected target vision algorithm to extract the key features of the target object in the image to be inspected, and based on the key features, the surface defects, size deviation, color difference, position, size and category of the target object are determined, and finally a test result is generated containing a conclusion on whether it is qualified and the detection information of the target object in the image to be inspected.

[0072] Step S40: output the detection result, and perform corresponding automated operations according to the detection result.

[0073] In this embodiment, the test results are first converted from the data format of the algorithm processing stage to a format suitable for output and subsequent automated operations. The test results include not only the conclusion of whether the product is qualified, but also any defect information or abnormal information of the detected product, such as defect type, location, size, category, whether the storage location material is empty, and product quality grade. These result information needs to be presented in a structured manner, such as JSON or XML format, to facilitate subsequent parsing and processing. Then, the test results after format conversion are output to the corresponding device or system. For example, in product quality inspection, the test results can be output to a display screen, an alarm or a PLC (Programmable Logic Controller). These test results can also be saved as text files, image files or database records through a file system, or the test results can be sent to a remote server or client application through a network communication protocol.

[0074] For example, if scratches are detected on the surface of a product, the unqualified conclusion and defect information (such as the length of the scratch, the location coordinates of the scratch, etc.) will be sent to the display screen for display, and the alarm will be triggered to send out an alarm signal, and the detection result will be transmitted to the PLC to control the subsequent production process.

[0075] While outputting the test results, the system will trigger corresponding automated operations according to preset rules or logic, including sending alarm signals to operators or outputting alarm prompts, adjusting the parameters or status of the production line, removing unqualified products from the production line, etc. The way to trigger automated operations can be to directly call functions or methods within the system, or to send requests or instructions to external systems or services.

[0076] For example, in the material detection of the storage location, the material identification and positioning results are updated to the warehouse management system, and the corresponding operations are triggered according to the storage status of the material. For example, if it is detected that the material is out of stock, the system will automatically send a replenishment instruction. In product quality inspection, if the test results show that the product is unqualified and there are defects in the product, an instruction is sent to the PLC on the production line to trigger the sorting equipment to remove the unqualified products from the production line. If the abnormal information in the test results is that the material position is abnormal, an instruction is sent to the AGV to adjust the material position. In this way, the system can flexibly output the test results and trigger the corresponding automated operations according to different detection tasks, thereby realizing the automated management of the production process.

[0077] Based on the above embodiments, please refer to Figure 2 In the visual inspection method of the second embodiment proposed in the present application, step S10 includes steps S110 to S140: Step S110: Based on the additional information, obtain the identification of the shooting device of the image to be detected.

[0078] In industrial production environments, it is usually necessary to determine the detection scene corresponding to the image to be detected based on the source and background information of the image to be detected in order to select the appropriate visual algorithm. For example, in an automated production line, different processing links may require the detection of different features or defects, so the detection strategy needs to be adjusted based on the source and processing link information of the image to be detected.

[0079] Additional information refers to metadata associated with the image to be detected, such as the shooting time, shooting location, shooting equipment identification, and environmental parameters of the image to be detected. This additional information is usually embedded in the header of the image file or stored together with the image file.

[0080] The camera identification is a number or code used to identify the camera, which can be used to trace the source of the image to be detected. In practical applications, the camera identification field can be extracted by parsing the metadata of the image file to obtain the camera identification. For example, if the image file is in JPEG format, its EXIF ​​information may contain the model and serial number of the camera, and the camera identification corresponding to the image to be captured can be obtained by reading this information.

[0081] Step S120: Determine the installation location or preset activity range of the shooting device according to the shooting device identifier.

[0082] In industrial production scenarios, camera equipment is usually installed in a fixed position or moves along a preset path. The installation position refers to the fixed coordinates of the camera equipment on the production line, while the preset activity range refers to the range within which the camera equipment can move within a certain area. By matching the camera equipment identification with the pre-stored device location information or activity range information, the specific location or activity range of the camera equipment can be determined. For example, the system can maintain a database to record the installation position coordinates or activity range parameters corresponding to each camera equipment identification.

[0083] For example, the range of movement of the camera on the AGV will change as the AGV moves. The AGV travels along a preset path at the production site, and its camera can take pictures of workpieces, equipment or warehouse environment along the way. In this case, the range of movement of the camera on the AGV is determined by the AGV's travel path and the camera's field of view. By presetting the AGV's travel path and the camera's parameters, it can be ensured that the AGV's camera can cover the key areas that need to be inspected on the production line.

[0084] Step S130: query the processing plan information of the installation position or the preset activity range to determine the processing link in which the photographing device is currently located.

[0085] Processing plan information refers to the production process arrangement on the production line, including the order of each processing link, the content of the processing task, and the corresponding equipment location. By matching the installation position or activity range of the camera with the position parameters in the processing plan information, the processing link that the camera is currently in can be determined. For example, if the camera is installed at a fixed position A on the production line, the processing link corresponding to the fixed position A can be determined based on the processing plan information, such as welding, assembly, or quality inspection. The technical means to achieve this function include database query, position matching algorithm, etc.

[0086] Step S140: determining the detection scene of the image to be detected according to the processing task corresponding to the processing link.

[0087] Inspection scenarios refer to the inspection targets and inspection parameters set according to different processing tasks. For example, in the welding process, the inspection scenarios include the integrity of the weld and the quality of the welding. In the assembly process, the inspection scenarios involve the confirmation of the installation position of the parts. By analyzing the processing task content corresponding to the processing link, the system can determine the inspection task corresponding to the image to be inspected and select the corresponding visual algorithm. For example, if the processing task is to detect the dimensional accuracy of the parts, the system can call the dimensional measurement algorithm and set the measurement accuracy range and allowable error parameters according to the requirements of the processing task.

[0088] Based on the above embodiments, please refer to Figure 3 The visual inspection method of the third embodiment proposed in the present application may further include steps S11 to S13 after step S10: Step S11: selecting an algorithm set according to the detection scenario.

[0089] Step S12: Processing the image to be detected respectively by using the plurality of target vision algorithms in the algorithm set to generate a plurality of intermediate results, and determining the confidence level of each of the intermediate results.

[0090] Step S13: fusing the multiple intermediate results according to the confidence level to generate a final detection result of the image to be detected.

[0091] In this embodiment, different target vision algorithms have different advantages and limitations when processing the image to be detected. Therefore, in order to improve the accuracy of the detection result, an algorithm set can be selected according to the detection task of the detection scene. By processing the same image to be detected by multiple target vision algorithms in the selected algorithm set, image information can be extracted from different processing angles, thereby improving the accuracy and reliability of the detection.

[0092] Each target vision algorithm generates its own corresponding intermediate result and calculates the confidence for each intermediate result. The intermediate result refers to the detection result obtained by each target vision algorithm after processing the image to be detected, such as the image classification algorithm identifying the product type or material type in the image to be detected, and the target detection algorithm determining the defects of the product in the image to be detected. Confidence refers to the measure of the credibility of the intermediate result, usually expressed as a probability value, indicating the credibility of the detection result. For example, in target detection, the confidence indicates the probability that the detected target object belongs to a certain category.

[0093] Furthermore, there may be some differences among the multiple intermediate results obtained by the system, so it is necessary to fuse the intermediate results to obtain a more accurate final detection result. By fusing multiple intermediate detection results according to the confidence level, the advantages of each visual algorithm can be fully utilized to reduce the error and uncertainty of a single visual algorithm. During the fusion process, a weighted average method can be used to assign weights to each intermediate result according to the confidence level, and then calculate the comprehensive result. The weight distribution can be dynamically adjusted according to the confidence level, and the results with high confidence levels are given higher weights.

[0094] Based on the above embodiments, please refer to Figure 4 In the visual inspection method of the fourth embodiment proposed in the present application, step S30 includes steps S310 to S330: Step S310: Calculate the gradient magnitude and direction of each pixel in the image to be detected, and mark the pixel whose gradient magnitude is greater than a preset threshold as a feature point.

[0095] In this embodiment, the gradient magnitude and direction of the image to be detected are obtained by calculating the horizontal and vertical gradients of the image to be detected. For example, a Sobel operator or a Prewitt operator can be used to perform a convolution operation on the image to be detected, thereby calculating the horizontal gradient and vertical gradient of each pixel point, and then obtaining the gradient magnitude and direction. Then, the pixel points whose gradient magnitude is greater than a preset threshold are marked as feature points.

[0096] The preset threshold is a pre-set value used to filter out pixels with sufficiently large gradient amplitudes. By using the preset threshold, noise points can be filtered out while retaining pixels that contribute to image features.

[0097] Step S320: performing cluster analysis on the feature points, and determining key features of the image to be detected according to the clustering results.

[0098] Cluster analysis is a method of dividing feature points into multiple clusters, so that feature points in the same cluster have high similarity, while feature points between different clusters have great differences. Cluster analysis of feature points can group feature points with similar features (such as spatial position, gradient direction, etc.) into one cluster.

[0099] In this embodiment, the purpose of cluster analysis is to divide the feature points into several groups, each group representing a key feature area in the image to be detected.

[0100] There are many technical means of cluster analysis, the common ones include K-Means clustering algorithm, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm, etc. The above algorithms can divide feature points into different categories according to their attributes (such as coordinates, gradient direction, etc.).

[0101] Step S330: comparing the key features of the image to be detected with the target feature template, and determining the detection result of the image to be detected according to the comparison result.

[0102] The target feature template is predefined and used to describe the feature information of the target object. The target feature templates of different target objects are determined by analyzing and extracting a large number of sample images.

[0103] By comparing the key features of the image to be detected with the target feature template, it can be determined whether the product corresponding to the image to be detected is qualified. The comparison is achieved by calculating the similarity between the key features and the target feature template, for example, using methods such as Euclidean distance and cosine similarity. According to the comparison results, the detection result of the image to be detected can be determined, such as whether the product in the image to be detected has defects, whether the target object is correct, and whether the material in the warehouse is in an empty state. If the similarity is higher than the set threshold, the product corresponding to the image to be detected is considered to be qualified or meets the condition; otherwise, the product corresponding to the image to be detected is unqualified or does not meet the condition.

[0104] In an optional implementation, step S327 to S329 may be included before step S330: Step S327: extracting edge information of the image to be detected, and locating a target area of ​​the target object based on the edge information.

[0105] Edge information indicates the boundary of the target object in the image to be detected, and reflects the segmentation of different regions in the image to be detected. By extracting the edge information of the image to be detected, the approximate outline of the target object in the image to be detected can be found. The edge information of the image to be detected can be obtained by edge detection algorithms, such as the Canny edge detection algorithm and the Sobel operator.

[0106] Step S328: using a pre-trained target classification model to classify and identify the target area, and determine the target object in the image to be detected.

[0107] The target classification model can determine which category the target object in the target area belongs to based on the target area of ​​the input image to be detected. Common target classification models include convolutional neural networks, such as AlexNet and VGG.

[0108] The target classification model is trained with a large amount of sample data to learn the characteristic patterns of target objects of different categories. During the model training process, the model adjusts its own parameters so that the category labels can be accurately output for sample images of known categories.

[0109] In this embodiment, the target area is input into a pre-trained target classification model, which classifies and identifies the target area according to the learned feature pattern and outputs the category to which the target object belongs, thereby determining the target object in the image to be detected.

[0110] Step S329: Determine a target feature template corresponding to the image to be detected based on the identified target object.

[0111] Different target objects have different features, so different target feature templates are needed for comparison and detection. According to the target object identified in step S328, a corresponding target feature template is selected from a target feature template library. The target feature template library stores feature templates of various types of target objects, and the feature templates can be obtained by extracting and analyzing features of a large number of sample images of different types of target objects.

[0112] In another optional implementation, step S330 may further include steps S3310 to S3340: Step S3310: Convert the key feature and the target feature template into a first feature vector and a second feature vector respectively, calculate the similarity between the first feature vector and the second feature vector, and based on the similarity calculation result, determine whether the target object in the image to be detected meets the preset quality standard or status condition.

[0113] The key feature is the feature information that can represent the target object extracted from the image to be detected, and the target feature template is a pre-set feature set of the target object that meets the quality standard or a specific state. In order to facilitate mathematical calculation and comparison of the key features and the target feature template, the SIFT (Scale-Invariant Feature Transform) or ORB (Oriented Fast and Rotated Brief) algorithm is first used to convert the key features and the target feature template into the first feature vector and the second feature vector respectively.

[0114] Then, the similarity between the two feature vectors is calculated, and commonly used methods include cosine similarity, Euclidean distance, etc. Based on the similarity calculation result, it is determined whether the target object in the image to be detected meets the preset quality standard or state condition.

[0115] Quality standards or status conditions are rules predefined according to specific application scenarios. For example, a similarity threshold is set. If the similarity reaches or exceeds the similarity threshold, the target object is judged to meet the preset quality standards or status conditions. Among them, quality standards are usually used to evaluate whether there are defects, flaws or other quality problems on the surface of the target object in the image to be detected; while status conditions are used to determine the specific state of the target object, such as whether the material in the warehouse is full or empty.

[0116] Step S3320: If the target object meets the preset quality standard or status condition, the image to be detected is qualified.

[0117] When the target object meets the preset quality standard or status condition, it means that the target object in the image to be detected meets expectations, and therefore the image to be detected is determined to be qualified.

[0118] Step S3330: If the target object does not meet the preset quality standard or status condition, the image to be detected is unqualified, and the detection information of the target object is obtained according to the processing result of the target vision algorithm, and the detection information includes defect information or abnormality information.

[0119] When the target object does not meet the preset quality standards or status conditions, the image to be inspected is judged as unqualified. At this time, the detection information of the target object is obtained according to the processing results of the selected target vision algorithm. When processing the image, the target vision algorithm analyzes and identifies various features of the target object to generate an intermediate result. The intermediate result contains possible defect information or abnormal information of the target object. For example, when detecting surface defects of a product, scratches are identified and information such as the length and width of the scratches is obtained.

[0120] Step S3340: Integrate the conclusion of whether the product is qualified with the test information to generate the final test result.

[0121] In this embodiment, based on the judgment results of the aforementioned steps, the conclusion on whether the product is qualified is integrated with the detection information detected by the target visual algorithm to generate a detection result containing a clear conclusion and a specific description of the defect or abnormality, so that the operator can intuitively understand the detection situation.

[0122] Based on the above embodiments, please refer to Figure 5 In the visual inspection method of the fifth embodiment proposed in the present application, step S40 includes steps S410 to S420: Step S410: When the detection result of the image to be detected is qualified, a qualified signal is output to the automatic control system, and the qualified product corresponding to the image to be detected is transmitted to the next production process or storage area.

[0123] In this embodiment, when the detection result of the image to be detected is qualified, it means that the product corresponding to the image to be detected meets the preset quality standard or status condition. Then the system will output a qualified signal to the automatic control system to notify the automatic control system of the status of the current product. Then the automatic control system can continue the next production process according to the received qualified signal, for example, through a conveyor belt, a robotic arm and other equipment, to transmit the corresponding qualified product to the next production process or storage area.

[0124] The qualified signal is an electrical signal or digital signal used to convey product qualified information. The automated control system refers to a system that can automatically perform corresponding operations based on input signals. It can be built based on hardware devices such as programmable logic controllers and industrial computers.

[0125] Step S420: If the detection result of the image to be detected is unqualified, an alarm signal is issued or an alarm prompt is output, and the detection result is classified, and the corresponding device is controlled to perform corresponding automatic operations according to the classification result.

[0126] If the detection result of the image to be detected is unqualified, the system will send out an alarm signal or output an alarm prompt. The alarm signal can be a sound or light prompt from an audible and visual alarm, or it can be an alarm message automatically generated by the system and transmitted to the display screen of the monitoring terminal or the operator's mobile device through the network.

[0127] Since different types of non-conforming situations may require different treatment methods, the test results can be classified, and the corresponding equipment can be controlled to perform corresponding automated operations or adjust the parameters of the corresponding equipment in the production process according to the classification results. The equipment may include sorting equipment, robotic arms, marking equipment, destruction equipment, etc.

[0128] In an optional implementation, the detection results are classified based on the type of defect or abnormal state detected. According to the classification results, the corresponding equipment is controlled to perform the corresponding automated operation. For example, if a surface scratch is detected, it is classified as an "appearance defect"; if a material is missing, it is classified as an "abnormal state". For products with appearance defects, the sorting equipment is triggered to remove the unqualified products from the current production line or transfer them to the rework area. For abnormal states, such as insufficient materials, an alarm signal or alarm prompt can be given, or the production line can be suspended and the replenishment personnel can be notified.

[0129] After determining the inspection result of the image to be inspected, it is also possible to record in detail the relevant information of the product corresponding to the image to be inspected, such as the product's production batch, production time, inspection time, the visual algorithm used and related parameters, etc., to facilitate subsequent quality traceability and production data analysis.

[0130] An embodiment of the present application provides a visual inspection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the visual inspection method in the above-mentioned embodiment one.

[0131] Reference below Figure 6 , which shows a schematic diagram of the structure of a visual inspection device suitable for implementing the embodiment of the present application. The visual inspection device in the embodiment of the present application may include various hardware and software components for implementing the visual inspection method. Figure 6 The visual inspection device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0132] like Figure 6As shown, the visual inspection device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM) 1004. In the random access memory 1004, various programs and data required for the operation of the visual inspection device are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, etc.; an output device 1008 including, for example, a liquid crystal display (LCD, Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. Communication device 1009 can allow the visual inspection device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a visual inspection device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0133] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0134] The visual inspection device provided by the present application adopts the visual inspection method in the above embodiment, which can solve the technical problem of how to provide stable and reliable quality feedback to realize the automation of the production process. Compared with the prior art, the beneficial effects of the visual inspection device provided by the present application are the same as the beneficial effects of the visual inspection method provided by the above embodiment, and the other technical features in the visual inspection device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0135] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0136] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0137] An embodiment of the present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the visual inspection method in the above-mentioned embodiment.

[0138] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM, CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, radio frequencies (RF, Radio Frequency), etc., or any suitable combination of the above.

[0139] The computer-readable storage medium may be included in the visual inspection device; or may exist independently without being assembled into the visual inspection device.

[0140] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the visual inspection device, the visual inspection device: obtains additional information of the image to be inspected, and determines the inspection scene of the image to be inspected based on the additional information; selects a target vision algorithm adapted to the image to be inspected based on the detection scene and the image to be inspected; processes the image to be inspected through the target vision algorithm to determine the inspection result of the image to be inspected; outputs the inspection result, and performs corresponding automated operations based on the inspection result.

[0141] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0142] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0143] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0144] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned visual inspection method, and can solve the technical problem of how to provide stable and reliable quality feedback to achieve production process automation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the visual inspection method provided in the above-mentioned embodiment, and will not be repeated here.

[0145] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the above-mentioned visual inspection method when executed by a processor.

[0146] The computer program product provided in this application can solve the technical problem of how to provide stable and reliable quality feedback to achieve production process automation. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiment of this application are the same as the beneficial effects of the visual inspection method provided in the above embodiment, which will not be repeated here.

[0147] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

[0148] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0149] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0150] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A visual inspection method, characterized in that: The visual detection method comprises: Acquire additional information of the image to be detected, and determine the detection scene of the image to be detected according to the additional information; According to the detection scene and the image to be detected, selecting a target vision algorithm adapted to the image to be detected; Processing the image to be detected by the target vision algorithm to determine a detection result of the image to be detected; The detection result is output, and corresponding automated operations are performed according to the detection result.

2. The visual inspection method according to claim 1, characterized in that: The step of obtaining additional information of the image to be detected and determining the detection scene of the image to be detected according to the additional information includes: Based on the additional information, obtaining an identification of a shooting device of the image to be detected; Determine the installation location or preset activity range of the shooting device according to the shooting device identifier; Querying the processing plan information of the installation position or the preset activity range to determine the processing link the photographing device is currently in; According to the processing task corresponding to the processing link, the detection scene of the image to be detected is determined.

3. The visual inspection method according to claim 1, characterized in that: After the step of obtaining the additional information of the image to be detected and determining the detection scene of the image to be detected according to the additional information, the method further includes: Selecting an algorithm set according to the detection scenario; Processing the image to be detected respectively by using a plurality of the target vision algorithms in the algorithm set to generate a plurality of intermediate results, and determining the confidence level of each of the intermediate results; The multiple intermediate results are fused according to the confidence level to generate a final detection result of the image to be detected.

4. The visual inspection method according to claim 1, characterized in that: The step of selecting a target vision algorithm adapted to the image to be detected according to the detection scene and the image to be detected comprises: Selecting a set of candidate algorithms according to the detection task of the detection scenario; Acquire image feature parameters of the image to be detected, and determine the applicable feature range of each visual algorithm in the candidate algorithm set; The image feature parameters are matched with the applicable feature range, and the visual algorithm with the highest matching degree in the candidate algorithm set is selected as the target visual algorithm.

5. The visual inspection method according to claim 1, characterized in that: The step of processing the image to be detected by the target vision algorithm to determine the detection result of the image to be detected includes: Calculate the gradient magnitude and direction of each pixel in the image to be detected, and mark the pixel whose gradient magnitude is greater than a preset threshold as a feature point; Performing cluster analysis on the feature points, and determining key features of the image to be detected according to the clustering results; The key features of the image to be detected are compared with the target feature template, and the detection result of the image to be detected is determined according to the comparison result.

6. The visual inspection method according to claim 5, characterized in that: Before the step of comparing the key features of the image to be detected with the target feature template and determining the detection result of the image to be detected according to the comparison result, the method further includes: Extracting edge information of the image to be detected, and locating a target area of ​​the target object based on the edge information; Using a pre-trained target classification model to classify and identify the target area, and determine the target object in the image to be detected; According to the identified target object, a target feature template corresponding to the image to be detected is determined.

7. The visual inspection method according to claim 4, characterized in that: The step of comparing the key features of the image to be detected with the target feature template and determining the detection result of the image to be detected according to the comparison result also includes: Converting the key feature and the target feature template into a first feature vector and a second feature vector respectively, calculating the similarity between the first feature vector and the second feature vector, and judging whether the target object in the image to be detected meets a preset quality standard or state condition based on the similarity calculation result; If the target object meets the preset quality standard or status condition, the image to be detected is qualified; If the target object does not meet the preset quality standard or status condition, the image to be detected is unqualified, and detection information of the target object is obtained according to the processing result of the target vision algorithm, and the detection information includes defect information or abnormality information; The conclusion on whether the product is qualified or not is integrated with the test information to generate the final test result.

8. The visual inspection method according to claim 1, characterized in that: The step of outputting the detection result and performing corresponding automated operations according to the detection result includes: When the inspection result of the image to be inspected is qualified, a qualified signal is output to the automatic control system, and the qualified product corresponding to the image to be inspected is transmitted to the next production process or storage area; If the detection result of the image to be detected is unqualified, an alarm signal is issued or an alarm prompt is output, and the detection result is classified, and the corresponding equipment is controlled to perform corresponding automatic operations according to the classification result.

9. A visual inspection device, characterized in that: The visual inspection device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the visual inspection method according to any one of claims 1 to 8.

10. A computer storage medium, characterized in that: The computer storage medium is a computer-readable storage medium, and a computer program is stored on the computer storage medium. When the computer program is executed by a processor, the steps of the visual inspection method according to any one of claims 1 to 8 are implemented.

Images