Visual detection method, electronic device, and computer-readable storage medium

Abstract

This application discloses a visual inspection method, electronic device, and computer-readable storage medium, relating to the field of industrial visual inspection technology. The visual inspection method includes: after detecting a visual inspection request based on a target product, obtaining an image of the target product to be inspected; extracting features from the image to be inspected based on a preset image processing algorithm to obtain target features of the image to be inspected, and performing visual inspection on the target product based on the target features to obtain a first visual inspection result of the target product; inputting the image to be inspected into a pre-trained visual inspection deep learning model to obtain a second visual inspection result of the target product output by the visual inspection deep learning model; and performing weighted fusion on the first and second visual inspection results to obtain a third visual inspection result of the target product. This application can improve the accuracy of visual inspection results.

Description

Technical Field

[0001] The present application relates to the technical field of industrial visual inspection, and in particular to a visual inspection method, an electronic device, and a computer-readable storage medium. Background Art

[0002] In industrial manufacturing, visual inspection technology is a core means of ensuring product quality and improving production automation. With the rapid development of modern manufacturing towards high precision and high efficiency, machine vision-based applications such as defect detection, dimensional measurement, and blob (connected domain) analysis have become indispensable in high-end industries such as semiconductors, automotive, and electronics. Visual inspection, with its non-contact, high-speed, and highly consistent features, can identify product defects, geometric parameter deviations, and other issues in real time, significantly reducing manual inspection costs and improving product yield. This technology is crucial for intelligent production processes and quality control.

[0003] Currently, industrial visual inspection relies primarily on two types of technologies: rule-driven visual inspection (hereinafter referred to as traditional visual inspection) and data-driven visual inspection (hereinafter referred to as deep learning-based visual inspection). However, traditional visual inspection typically relies on manually designed feature extraction rules and is less robust to changes in illumination, noise interference, and complex backgrounds. This can lead to dimensional measurement deviations or blob segmentation errors in scenarios such as blurred feature edges, low contrast, or object adhesion. While deep learning-based visual inspection can adaptively learn complex features, its detection performance is highly dependent on the scale and quality of training data. When the amount of data is insufficient or the sample distribution is uneven, the model is prone to overfitting or insufficient generalization. Furthermore, the model's decision-making process lacks interpretability, making it difficult to accurately correlate the setting of key parameters (such as dimensional error thresholds and connected domain morphological features) with actual physical properties, further affecting the reliability of the inspection results.

[0004] To sum up, how to effectively improve the accuracy of visual inspection results has become a technical problem that needs to be solved urgently in the industry. Summary of the Invention

[0005] The main purpose of this application is to provide a visual inspection method, aiming to solve the technical problem of low accuracy of visual inspection results in industrial visual inspection.

[0006] To achieve the above objectives, the present application provides a visual inspection method, which comprises:

[0007] After detecting a visual inspection request based on a target product, obtaining an image of the target product to be inspected;

[0008] Performing feature extraction on the image to be inspected based on a preset image processing algorithm to obtain target features of the image to be inspected, and performing visual inspection on the target product based on the target features to obtain a first visual inspection result of the target product, wherein the image processing algorithm includes an edge detection algorithm and / or a connected component labeling algorithm, and the first visual inspection result includes a first size measurement result and / or a first blob analysis result;

[0009] Inputting the image to be inspected into a pre-trained visual inspection deep learning model to obtain a second visual inspection result of the target product output by the visual inspection deep learning model, wherein the visual inspection deep learning model includes a dimensional measurement deep learning model and / or a blob analysis deep learning model, and the second visual inspection result includes a second dimensional measurement result and / or a second blob analysis result;

[0010] The first visual inspection result and the second visual inspection result are weightedly fused to obtain a third visual inspection result of the target product, wherein the third visual inspection result includes a third size measurement result and / or a third blob analysis result.

[0011] In one embodiment, the first visual detection result includes a first size measurement result and a normalized average edge gradient amplitude corresponding to the first size measurement result, and the second visual detection result includes a second size measurement result and a confidence level of the second size measurement result;

[0012] The step of performing weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product includes:

[0013] Calculating a confidence level of the first size measurement result according to the normalized average edge gradient amplitude corresponding to the first size measurement result;

[0014] Determining, according to the confidence level of the first size measurement result and the confidence level of the second size measurement result, a first weight corresponding to the first size measurement result and a second weight corresponding to the second size measurement result;

[0015] The first size measurement result and the second size measurement result are weightedly fused according to the first weight and the second weight to obtain a third size measurement result.

[0016] In one embodiment, the step of performing weighted fusion on the first size measurement result and the second size measurement result according to the first weight and the second weight to obtain a third size measurement result includes:

[0017] Acquiring shooting environment information of the image to be detected, and adjusting the first weight and the second weight according to the shooting environment information;

[0018] The first size measurement result and the second size measurement result are weightedly fused according to the adjusted first weight and the adjusted second weight to obtain a third size measurement result.

[0019] In one embodiment, the first visual detection result includes a first blob analysis result and a normalized average edge gradient amplitude corresponding to the first blob analysis result, and the second visual detection result includes a second blob analysis result and a confidence level of the second blob analysis result;

[0020] The step of performing weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product includes:

[0021] Calculating a confidence level of the first Blob analysis result according to the normalized average edge gradient amplitude corresponding to the first Blob analysis result;

[0022] Determining, according to the confidence level of the first blob analysis result and the confidence level of the second blob analysis result, a third weight corresponding to the first blob analysis result and a fourth weight corresponding to the second blob analysis result;

[0023] The first blob analysis result and the second blob analysis result are weightedly fused according to the third weight and the fourth weight to obtain a third blob analysis result.

[0024] In one embodiment, the first visual detection result includes a first blob analysis result and a morphological closed area change rate corresponding to the first blob analysis result, and the second visual detection result includes a second blob analysis result and a confidence level of the second blob analysis result;

[0025] The step of performing weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product includes:

[0026] Calculating the confidence level of the first Blob analysis result according to the morphological closed area change rate corresponding to the first Blob analysis result;

[0027] Determining, according to the confidence level of the first blob analysis result and the confidence level of the second blob analysis result, a third weight corresponding to the first blob analysis result and a fourth weight corresponding to the second blob analysis result;

[0028] The first blob analysis result and the second blob analysis result are weightedly fused according to the third weight and the fourth weight to obtain a third blob analysis result.

[0029] In one embodiment, the first visual detection result includes a first blob analysis result and a convex hull filling rate corresponding to the first blob analysis result, and the second visual detection result includes a second blob analysis result and a confidence level of the second blob analysis result;

[0030] The step of performing weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product includes:

[0031] Calculating the confidence level of the first Blob analysis result according to the convex hull filling rate corresponding to the first Blob analysis result;

[0032] Determining, according to the confidence level of the first blob analysis result and the confidence level of the second blob analysis result, a third weight corresponding to the first blob analysis result and a fourth weight corresponding to the second blob analysis result;

[0033] The first blob analysis result and the second blob analysis result are weightedly fused according to the third weight and the fourth weight to obtain a third blob analysis result.

[0034] In one embodiment, the first visual detection result includes a first blob analysis result, and a normalized average edge gradient amplitude, a morphological closed area change rate, and a convex hull filling rate corresponding to the first blob analysis result; the second visual detection result includes a second blob analysis result, and a confidence level of the second blob analysis result;

[0035] The step of performing weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product includes:

[0036] Calculating the confidence level of the first Blob analysis result based on the normalized average edge gradient amplitude, the morphological closed area change rate, and the convex hull filling rate corresponding to the first Blob analysis result;

[0037] Determining, according to the confidence level of the first blob analysis result and the confidence level of the second blob analysis result, a third weight corresponding to the first blob analysis result and a fourth weight corresponding to the second blob analysis result;

[0038] The first blob analysis result and the second blob analysis result are weightedly fused according to the third weight and the fourth weight to obtain a third blob analysis result.

[0039] In one embodiment, the third visual inspection result includes a third size measurement result and a third blob analysis result;

[0040] After the step of performing weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product, the method further includes:

[0041] According to the third size measurement result and the third blob analysis result, defect type detection is performed on the target product to obtain a defect type detection result of the target product.

[0042] In addition, to achieve the above-mentioned purpose, the present application also provides an electronic device, which includes: a memory, a processor, and a computer program stored on the memory and runnable on the processor, and when the computer program is executed by the processor, the steps of the visual detection method as described above are implemented.

[0043] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the visual inspection method as described above are implemented.

[0044] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the above-mentioned visual inspection method.

[0045] Embodiments of the present application provide a visual inspection method, an electronic device, and a computer-readable storage medium. The visual inspection method includes: after detecting a visual inspection request based on a target product, obtaining an image of the target product to be inspected; performing feature extraction on the image to be inspected based on a preset image processing algorithm to obtain target features of the image to be inspected, and performing visual inspection on the target product based on the target features to obtain a first visual inspection result of the target product, wherein the image processing algorithm includes an edge detection algorithm and / or a connected component labeling algorithm, and the first visual inspection result includes a first size measurement result and / or a first blob analysis result; inputting the image to be inspected into a pre-trained visual inspection deep learning model to obtain a second visual inspection result of the target product output by the visual inspection deep learning model, wherein the visual inspection deep learning model includes a size measurement deep learning model and / or a blob analysis deep learning model, and the second visual inspection result includes a second size measurement result and / or a second blob analysis result; and performing weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product, wherein the third visual inspection result includes a third size measurement result and / or a third blob analysis result.

[0046] The embodiment of the present application uses a weighted decision-making mechanism to integrate the heterogeneous results of two visual inspection technologies, effectively solving the problems of traditional visual inspection being overly sensitive to the environment and limited by manual experience, as well as the problem of deep learning-based visual inspection being overly dependent on sample data, thereby significantly improving the accuracy and robustness of industrial visual inspection. Specifically, the embodiment of the present application first uses traditional image processing algorithms such as edge detection and connected domain labeling to extract the geometric features of the target product (such as edge contours and regional connectivity), thereby generating a first visual inspection result. At the same time, a dedicated deep learning model for dimensional measurement or blob analysis is used to perform adaptive feature learning and complex pattern recognition on the same image to be inspected, thereby outputting a second visual inspection result. Finally, based on the preset weight coefficient or dynamic weight adjustment strategy, the two types of results are weightedly fused (such as linear weighting, confidence weighting). Through complementary decision optimization, the accuracy of traditional visual inspection in rule features is retained, and the strong representation ability of deep learning for nonlinear features is utilized to compensate for the detection deviation of traditional visual inspection in complex scenarios such as blurred edges and noise interference. In this way, in variable industrial scenarios such as lighting fluctuations, target adhesion, and data scarcity, the dimensional measurement error is reduced and the accuracy of Blob analysis is improved, thereby solving the technical problem of low accuracy of visual inspection results in industrial visual inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0049] Figure 1 A schematic diagram of a flow chart of the first embodiment of the visual inspection method of the present application;

[0050] Figure 2 A schematic diagram of a flow chart of the second embodiment of the visual inspection method of the present application;

[0051] Figure 3 A schematic diagram of a flow chart provided for the third embodiment of the visual inspection method of the present application;

[0052] Figure 4 A system flow chart provided for a specific implementation of this application;

[0053] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment of the electronic device involved in the visual detection method in the embodiment of the present application.

[0054] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0055] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0056] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0057] In industrial visual inspection, the current mainstream visual inspection technologies are: traditional visual inspection and deep learning-based visual inspection.

[0058] Traditional visual inspection relies on manually designed feature extraction rules, and the accuracy and reliability of its detection results are limited by environmental interference (such as lighting changes and background noise) and the subjectivity of human experience (such as deviations in the parameter settings of feature extraction rules). Although deep learning-based visual inspection can automatically extract abstract features through data-driven analysis, the accuracy and reliability of its detection results are limited by the scale and quality of training data. When training data is insufficient or unevenly distributed, the model may not be able to accurately identify target features, resulting in unstable detection results.

[0059] In response to the above problems, the main solution of the embodiment of the present application is: after detecting a visual inspection request based on a target product, obtaining an image of the target product to be inspected; performing feature extraction on the image to be inspected based on a preset image processing algorithm to obtain target features of the image to be inspected, and performing visual inspection on the target product based on the target features to obtain a first visual inspection result of the target product, wherein the image processing algorithm includes an edge detection algorithm and / or a connected component labeling algorithm, and the first visual inspection result includes a first size measurement result and / or a first blob analysis result; inputting the image to be inspected into a pre-trained visual inspection deep learning model to obtain a second visual inspection result of the target product output by the visual inspection deep learning model, wherein the visual inspection deep learning model includes a size measurement deep learning model and / or a blob analysis deep learning model, and the second visual inspection result includes a second size measurement result and / or a second blob analysis result; performing weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product, wherein the third visual inspection result includes a third size measurement result and / or a third blob analysis result.

[0060] The embodiment of the present application uses a weighted decision-making mechanism to integrate the heterogeneous results of two visual inspection technologies, effectively solving the problems of traditional visual inspection being overly sensitive to the environment and limited by manual experience, as well as the problem of deep learning-based visual inspection being overly dependent on sample data, thereby significantly improving the accuracy and robustness of industrial visual inspection. Specifically, the embodiment of the present application first uses traditional image processing algorithms such as edge detection and connected domain labeling to extract the geometric features of the target product (such as edge contours and regional connectivity), thereby generating a first visual inspection result. At the same time, a dedicated deep learning model for dimensional measurement or blob analysis is used to perform adaptive feature learning and complex pattern recognition on the same image to be inspected, thereby outputting a second visual inspection result. Finally, based on the preset weight coefficient or dynamic weight adjustment strategy, the two types of results are weightedly fused (such as linear weighting, confidence weighting). Through complementary decision optimization, the accuracy of traditional visual inspection in rule features is retained, and the strong representation ability of deep learning for nonlinear features is utilized to compensate for the detection deviation of traditional visual inspection in complex scenarios such as blurred edges and noise interference. In this way, in variable industrial scenarios such as lighting fluctuations, target adhesion, and data scarcity, the dimensional measurement error is reduced and the accuracy of Blob analysis is improved, thereby solving the technical problem of low accuracy of visual inspection results in industrial visual inspection.

[0061] It should be noted that the execution entities of the embodiments of this application may include, but are not limited to, terminal devices such as mobile phones, laptops, tablet computers, desktop computers, visual inspection systems, or any other electronic device capable of implementing the above functions. The following embodiments of this application are described using the visual inspection system as an example.

[0062] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0063] This application proposes a visual inspection method according to a first embodiment.

[0064] Please refer to Figure 1 , Figure 1 A flowchart of the first embodiment of the visual inspection method of this application is provided.

[0065] In this embodiment, the visual inspection method may include steps S100 to S300:

[0066] Step S100: After detecting a visual inspection request based on a target product, an image of the target product to be inspected is obtained;

[0067] It should be noted that target products refer to items or components that require visual inspection, such as visual defect detection, dimensional measurement, and morphological analysis, during industrial production or quality control. Target products may include, but are not limited to, electronic components, mechanical parts, food packaging, textiles, and automotive parts. The image to be inspected is a digitized image of the target product captured by an industrial camera, linear array sensor, or 3D scanning device. It contains information such as the target's surface texture, geometric outline, and defect characteristics.

[0068] In this embodiment, the visual inspection request refers to an instruction that triggers the visual inspection system to perform a visual inspection task, which can be generated by an external device (such as a programmable logic controller), a preset trigger condition (such as a conveyor position sensor signal), or a user interface interaction (such as a touch screen operation).

[0069] It is worth mentioning that the visual inspection request can specify which visual inspection tasks to perform on the target product, such as size measurement, blob analysis, etc.

[0070] In this embodiment, the visual inspection system can parse the visual inspection request, determine the target product, and obtain the image to be inspected of the target product, thereby performing the visual inspection task specified in the visual inspection request on the target product.

[0071] Step S200: extracting features from the image to be inspected based on a preset image processing algorithm to obtain target features of the image to be inspected, and performing visual inspection on the target product based on the target features to obtain a first visual inspection result of the target product;

[0072] The image processing algorithm includes an edge detection algorithm and / or a connected component labeling algorithm, and the first visual detection result includes a first size measurement result and / or a first blob analysis result;

[0073] Those skilled in the art will recognize that edge detection algorithms are specialized techniques for identifying object boundaries in images. They locate edges by calculating the rate of change of pixel values ​​in an image, highlighting areas of significant grayscale changes, i.e., object outlines. Connected component labeling algorithms are used to segment independent regions in an image and assign unique identifiers to each region. By scanning an image, adjacent pixels with the same or similar attributes (such as color or brightness) are grouped and each group is assigned a unique label. Common connected component labeling algorithms include two-pass scanning algorithms.

[0074] In this embodiment, target features refer to features extracted from the image to be inspected that represent the characteristics of the target product. It is readily apparent that different image processing algorithms extract different target features. Edge detection algorithms extract target features representing the boundaries or outlines of objects in the image to be inspected, while connected component labeling algorithms extract target features representing connected components within the image to be inspected.

[0075] It should be noted that, in this embodiment, the image to be detected needs to be preprocessed before feature extraction is performed on the image to be detected by the image processing algorithm. It is not difficult to understand that different image processing algorithms require different image preprocessing operations, which have been deeply studied by those skilled in the art, and will not be elaborated on in this embodiment.

[0076] In this embodiment, the first visual inspection result refers to the visual inspection result obtained by performing a traditional visual inspection on the target product. The first visual inspection result may include a first dimensional measurement result and / or a first blob analysis result. The first dimensional measurement result refers to the physical dimensional parameters of the target product obtained through geometric calculations based on the target features extracted using the edge detection algorithm (i.e., the boundary or outline information of the object in the image to be inspected). The first blob analysis result refers to the attribute information (such as area, center of mass, shape, etc.) of each connected domain on the target product obtained through geometric calculations based on the target features extracted using the connected domain labeling algorithm (i.e., the connected domain in the image to be inspected).

[0077] It is worth mentioning that, when performing size measurement, this embodiment can combine a sub-pixel edge detection algorithm to further improve the accuracy of the first size measurement result. When performing Blob analysis, the algorithm can be improved by connecting domain labeling. For example, a tree structure is used to represent the relationship between connected areas, and a tree merging strategy of dynamically merging these trees during the labeling process can be used to reduce repeated calculations and improve merging efficiency.

[0078] After detecting a visual inspection request based on a target product, this embodiment determines what types of visual inspections need to be performed on the target product based on the analysis results of the visual inspection request. Therefore, when the visual inspection request specifies dimensional measurement of the target product, feature extraction is performed on the image to be inspected using a preset edge detection algorithm to obtain boundary or contour information of the object in the image to be inspected as a target feature, and geometric calculations are performed based on the target feature to complete the visual inspection of dimensional measurement and obtain a first dimensional measurement result as a first visual inspection result of the target product.

[0079] Correspondingly, when the visual inspection request specifies Blob analysis of the target product, this embodiment performs feature extraction on the image to be inspected through a preset connected domain labeling algorithm, obtains the connected domain in the image to be inspected as the target feature, and performs geometric calculations based on the target feature to complete the visual inspection of the Blob analysis, and obtains the first Blob analysis result as the first visual inspection result of the target product.

[0080] It is not difficult to understand that when the visual inspection request specifies size measurement and blob analysis of the target product, this embodiment calls the corresponding image processing algorithm to extract the required target features to complete the corresponding visual inspection, thereby obtaining the first size measurement result and the first blob analysis result as the first visual inspection result of the target product.

[0081] Step S300: inputting the image to be inspected into a pre-trained visual inspection deep learning model to obtain a second visual inspection result of the target product output by the visual inspection deep learning model;

[0082] The visual inspection deep learning model includes a size measurement deep learning model and / or a blob analysis deep learning model, and the second visual inspection result includes a second size measurement result and / or a second blob analysis result;

[0083] It should be noted that the visual inspection deep learning model is a model pre-trained based on the deep learning model and is used to perform visual inspection of the target product. In this embodiment, according to the type of visual inspection task, a dedicated visual inspection deep learning model is trained accordingly.

[0084] For example, for the visual inspection task of measuring the size of the target product, this embodiment can pre-photograph a large number of products of the same type as the target product to obtain corresponding images, and obtain the actual physical size parameters of these products through manual measurement, and mark them on the corresponding images to form training data, so as to select a suitable deep learning model and loss function, and train with these training data to finally obtain a trained deep learning model for size measurement.

[0085] Correspondingly, for the visual inspection task of Blob analysis of the target product, this embodiment can pre-photograph a large number of products of the same type as the target product to obtain corresponding images, and obtain the attribute information of the connected domains in these images through manual measurement, and mark them on the corresponding images to form training data, so as to select a suitable deep learning model and loss function, and train with these training data to finally obtain a trained Blob analysis deep learning model.

[0086] It is worth mentioning that when selecting a suitable deep learning model to train the above-mentioned visual detection deep learning model, a deep learning model with a lightweight network architecture can be selected to facilitate the deployment of the trained visual detection deep learning model on resource-constrained devices, thereby integrating it with traditional visual detection without having to be deployed separately, resulting in data barriers in the detection process, and avoiding the need for users to frequently switch between different devices or applications to detect the visual detection results output by the two visual detection methods.

[0087] In this embodiment, the second visual inspection result refers to the visual inspection result obtained by performing a deep learning-based visual inspection on the target product. This second visual inspection result may include a second dimensional measurement result and / or a second blob analysis result. The second dimensional measurement result refers to the physical dimensional parameters of the target product predicted by the dimensional measurement deep learning model. The second blob analysis result refers to the attribute information (such as area, centroid, shape, etc.) of each connected domain on the target product predicted by the blob analysis deep learning model.

[0088] It is understandable that when performing visual inspection on a target product through a visual inspection deep learning model, the image to be inspected of the target product can be preprocessed first to improve the image quality, and then converted into a form that is convenient for input into the visual inspection deep learning model, such as a tensor form.

[0089] After detecting a visual inspection request based on a target product, this embodiment selects a corresponding visual inspection deep learning model according to the visual inspection task specified by the visual inspection request, performs visual inspection on the target product, and thus obtains a second visual inspection result of the target product.

[0090] Step S400: performing weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product;

[0091] The third visual inspection result includes a third size measurement result and / or a third blob analysis result.

[0092] In this embodiment, the third visual detection result refers to a visual detection result obtained by weighted fusion of the first visual detection result and the second visual detection result. The third visual detection result may include a third size measurement result and / or a third blob analysis result. The third size measurement result refers to a size measurement result obtained by weighted fusion of the first size measurement result and the second size measurement result, and the third blob analysis result refers to a blob analysis result obtained by weighted fusion of the first blob analysis result and the second blob analysis result.

[0093] This embodiment uses a weighted decision-making mechanism to integrate the heterogeneous results of two visual inspection technologies. This effectively addresses the issues of traditional visual inspection being overly sensitive to the environment and limited by manual experience, as well as the over-reliance on sample data in deep learning-based visual inspection. This significantly improves the accuracy and robustness of industrial visual inspection. Specifically, this embodiment first uses traditional image processing algorithms such as edge detection and connected domain labeling to extract the geometric features of the target product (such as edge contours and region connectivity) to generate a first visual inspection result. Simultaneously, a dedicated deep learning model for dimensional measurement or blob analysis performs adaptive feature learning and complex pattern recognition on the same image to be inspected, thereby outputting a second visual inspection result. Finally, based on the preset weight coefficient or dynamic weight adjustment strategy, the two types of results are weightedly fused (such as linear weighting, confidence weighting). Through complementary decision optimization, the accuracy of traditional visual inspection in rule features is retained, and the strong representation ability of deep learning for nonlinear features is utilized to compensate for the detection deviation of traditional visual inspection in complex scenarios such as blurred edges and noise interference. In this way, in variable industrial scenarios such as lighting fluctuations, target adhesion, and data scarcity, the dimensional measurement error is reduced and the accuracy of Blob analysis is improved, thereby solving the technical problem of low accuracy of visual inspection results in industrial visual inspection.

[0094] In this embodiment, the weighted fusion method can be fixed weight fusion or dynamic weight fusion. In the fixed weight fusion method, the user or the system can pre-set the weight coefficients of the two visual detection results to perform fixed weight weighted fusion.

[0095] Regarding dynamic weight fusion, the following is a detailed explanation using size measurement as an example.

[0096] Based on the above first embodiment, a feature extraction method of the second embodiment of the present application is proposed.

[0097] In the second embodiment of the present application, for the same or similar contents as those in the above embodiments, please refer to the above introduction and will not be repeated hereafter.

[0098] Please refer to Figure 2 , Figure 2 A flowchart of the second embodiment of the feature extraction method of this application is provided.

[0099] In this embodiment, the first visual detection result includes a first size measurement result and a normalized average edge gradient amplitude corresponding to the first size measurement result, and the second visual detection result includes a second size measurement result and a confidence level of the second size measurement result;

[0100] Step S400 performs weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product, which may include steps S411 to S413:

[0101] Step S411, calculating the confidence level of the first size measurement result according to the normalized average edge gradient amplitude corresponding to the first size measurement result;

[0102] In this embodiment, the normalized average edge gradient amplitude is an indicator for quantifying edge clarity, and its calculation formula is as follows:

[0103] ;

[0104] in, is the normalized average edge gradient amplitude, N is the total number of edge pixels, Edge pixels The gradient amplitude of is the maximum gradient amplitude in the image to be detected.

[0105] In this embodiment, when measuring the size of a target product, it is necessary to first perform edge detection on the image to be detected of the target product to determine the edge pixels in the image to be detected, that is, pixels whose gradient amplitude is greater than a preset value. Then, the normalized average edge gradient amplitude corresponding to the first size measurement result can be calculated using the above calculation formula.

[0106] This embodiment notes that when performing size measurement, the higher the gradient amplitude of each edge pixel extracted by the edge detection algorithm (that is, the clearer the edge), the more accurate the size measurement result. Therefore, a solution is proposed to reflect the confidence level of the first size measurement result by calculating the normalized average edge gradient amplitude corresponding to the first size measurement result (that is, normalizing the gradient amplitude of each edge pixel extracted by the edge detection algorithm and then taking the average value).

[0107] Specifically, this embodiment can directly use the normalized average edge gradient amplitude corresponding to the first size measurement result as the confidence level of the first size measurement result, or perform certain mathematical operations based on the normalized average edge gradient amplitude corresponding to the first size measurement result to obtain the confidence level of the first size measurement result.

[0108] In addition, the ratio of the error between the first size measurement result and the actual physical size parameter that is less than a preset value under different normalized average edge gradient amplitudes can be pre-calibrated, so that this ratio is used as the confidence of the normalized average edge gradient amplitude mapping, and a corresponding mapping relationship is constructed, so that the confidence of the first size measurement result can be directly determined based on the mapping relationship in actual applications.

[0109] Step S412: determining a first weight corresponding to the first size measurement result and a second weight corresponding to the second size measurement result according to the confidence level of the first size measurement result and the confidence level of the second size measurement result;

[0110] It should be noted that the first weight refers to the weight ratio of the first size measurement result when performing weighted fusion, and the second weight refers to the weight ratio of the second size measurement result when performing weighted fusion.

[0111] In this embodiment, the dimension measurement deep learning model outputs the confidence level of the predicted second dimension measurement result while outputting the predicted second dimension measurement result.

[0112] In this embodiment, when determining the weight based on the confidence level, for example, if the confidence level of the first size measurement result is 0.9 and the confidence level of the second size measurement result is 0.6, the first weight of the first size measurement result may be 0.9 / (0.9+0.6)=0.6, and correspondingly, the weight of the second size measurement result may be 0.6 / (0.9+0.6)=0.4.

[0113] In addition, when the confidence of the first size measurement result is greater than 0.9, it can be determined that the first weight of the first size measurement result is equal to the confidence of the first size measurement result, and when the confidence of the first size measurement result is less than 0.6 and the confidence of the second size measurement result is greater than 0.8, it can be determined that the second weight of the second size measurement result is equal to the confidence of the second size measurement result.

[0114] It is not difficult to understand that the sum of the first weight and the second weight is 1, and the confidence value range is 0 to 1.

[0115] The specific confidence weighting rule can be flexibly set according to actual needs. The above example is for reference only and is not specifically limited in this embodiment.

[0116] Step S413 : performing weighted fusion on the first size measurement result and the second size measurement result according to the first weight and the second weight to obtain a third size measurement result.

[0117] This embodiment maps the normalized average edge gradient magnitude to a confidence level, transforming the environmental adaptability limitations of traditional visual inspection methods into quantifiable weight parameters for the first time. This allows the weighted fusion process to dynamically respond to changes in image quality. For example, when low light levels cause blurred edges, the first weight of the first dimension measurement result is automatically reduced to prevent error propagation.

[0118] After calculating the confidence level of the first dimension measurement result, this embodiment can dynamically adjust the weight ratios of the first dimension measurement result and the second dimension measurement result during weighted fusion based on the confidence levels of the first dimension measurement result and the second dimension measurement result, thereby achieving complementary decision optimization. This not only retains the accuracy of traditional visual inspection in regular features, but also utilizes the strong representation capability of deep learning for nonlinear features to compensate for the detection deviations of traditional visual inspection in complex scenarios such as blurred edges and noise interference, thereby reducing dimension measurement errors in variable industrial scenarios such as lighting fluctuations, target adhesion, and data scarcity.

[0119] Furthermore, in a feasible implementation manner, step S413 may include steps A10 to A20:

[0120] Step A10: obtaining shooting environment information of the image to be detected, and adjusting the first weight and the second weight according to the shooting environment information;

[0121] It should be noted that shooting environment information refers to a set of external condition parameters that affect the accuracy of visual inspection during the image acquisition process, which may include but is not limited to light intensity, light source type, camera gain and exposure time, relative movement speed between the camera and the target product, etc.

[0122] Step A20: performing weighted fusion on the first size measurement result and the second size measurement result according to the adjusted first weight and the adjusted second weight to obtain a third size measurement result.

[0123] Based on dynamic weight adjustment, this embodiment further introduces shooting environment information as a weight correction factor to achieve a more refined weight allocation strategy. Specifically, step A10 dynamically corrects the initial weights obtained in step S412 by real-time perception or calculation of interference factors in the shooting environment (such as light fluctuations and motion blur). Step A20 then completes weighted fusion based on the corrected weights, so that the third dimension measurement result takes into account both algorithm confidence and environmental adaptability, thus breaking through the limitations of relying solely on the confidence of dimension measurement results. For example, when the light intensity is known to be less than 50 Lux, even if the confidence of the first dimension measurement result is high, the first weight is actively reduced to prevent potential risks of sudden changes in lighting. For another example, when the camera gain is too high, resulting in significant image noise, even if the confidence of the first dimension measurement result is high, the second weight is actively increased, leveraging the noise suppression capabilities of the deep learning model to compensate for the measurement deviations of traditional visual inspection.

[0124] Regarding dynamic weight fusion, the following is a detailed explanation using Blob analysis as an example.

[0125] Based on the above first embodiment, a feature extraction method of the third embodiment of the present application is proposed.

[0126] In the third embodiment of the present application, for the same or similar contents as those in the above embodiments, please refer to the above introduction and will not be repeated hereafter.

[0127] Please refer to Figure 3 , Figure 3 A flowchart of the third embodiment of the feature extraction method of this application is provided.

[0128] In this embodiment, the first visual detection result includes a first blob analysis result and a normalized average edge gradient amplitude corresponding to the first blob analysis result; the second visual detection result includes a second blob analysis result and a confidence level of the second blob analysis result;

[0129] Step S400 performs weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product, which may include steps S421 to S423:

[0130] Step S421, calculating the confidence level of the first Blob analysis result according to the normalized average edge gradient amplitude corresponding to the first Blob analysis result;

[0131] In this embodiment, when performing Blob analysis on the target product, it is necessary to first mark the connected domains of the image to be detected of the target product, so as to obtain the connected domains in the image to be detected, and then determine the contour information of each connected domain through contour extraction, that is, determine the edge pixels of each connected domain, and then calculate the normalized average edge gradient amplitude corresponding to the first Blob analysis result through the calculation formula of the normalized average edge gradient amplitude in the second embodiment.

[0132] Accordingly, when performing Blob analysis, the higher the gradient amplitude of the edge pixels of each connected domain obtained by marking, the more accurate the Blob analysis result. Therefore, the confidence of the first Blob analysis result can be reflected by the normalized average edge gradient amplitude corresponding to the first Blob analysis result.

[0133] The specific confidence calculation method can be referred to the second embodiment, and this embodiment does not make repeated limitations on this.

[0134] Step S422: determining a third weight corresponding to the first blob analysis result and a fourth weight corresponding to the second blob analysis result according to the confidence level of the first blob analysis result and the confidence level of the second blob analysis result;

[0135] It should be noted that the third weight refers to the weight ratio of the first Blob analysis result when performing weighted fusion, and the fourth weight refers to the weight ratio of the second Blob analysis result when performing weighted fusion.

[0136] In this embodiment, the blob analysis deep learning model outputs the confidence level of the second blob analysis result while outputting the predicted second blob analysis result.

[0137] The specific confidence weighting rules can be referred to the second embodiment above, and will not be elaborated in detail in this embodiment.

[0138] Step S423: Perform weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain a third Blob analysis result.

[0139] Similar to the second embodiment, this embodiment can also dynamically adjust the weight ratio of the first Blob analysis result and the second Blob analysis result during weighted fusion through the confidence level of each Blob analysis result, thereby achieving complementary decision optimization, retaining the accuracy of traditional visual detection in rule features, and using the strong representation ability of deep learning for nonlinear features to make up for the detection deviation of traditional visual detection in complex scenarios such as blurred edges and noise interference, and thus improve the accuracy of Blob analysis in variable industrial scenarios such as lighting fluctuations, target adhesion, and data scarcity.

[0140] It is easy to understand that, in this embodiment, the third weight and the fourth weight can also be adjusted according to the shooting environment information.

[0141] In a feasible implementation, the first visual detection result includes a first blob analysis result and a morphological closed area change rate corresponding to the first blob analysis result, and the second visual detection result includes a second blob analysis result and a confidence level of the second blob analysis result;

[0142] Step S400 performs weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product, which may include steps S431 to S433:

[0143] Step S431, calculating the confidence level of the first Blob analysis result according to the morphological closed area change rate corresponding to the first Blob analysis result;

[0144] In this embodiment, the morphological closed area change rate refers to the change rate of the area of ​​the connected domain after the morphological closing operation is performed on the connected domain extracted by the connected domain labeling algorithm.

[0145] This embodiment notes that when performing Blob analysis, the smaller the rate of change of the area of ​​the connected domain before and after the morphological closing operation is performed on the connected domain, the more complete the structure of the connected domain is, and the more reliable the first Blob analysis result obtained based on this is. Therefore, it is proposed that after determining the connected domain in the image to be detected through the connected domain labeling algorithm, the area of ​​the connected domain before and after the closing operation is determined through the morphological closing operation, so as to calculate the morphological closing area change rate corresponding to the first Blob analysis result, and then use it to reflect the confidence of the first Blob analysis result.

[0146] The specific confidence calculation method is similar to the above embodiment. The ratio of the error between the first blob analysis result and the actual connected domain attribute information under different morphological closed area change rates can be pre-calibrated, and this ratio can be used as the confidence of the morphological closed area change rate mapping. A corresponding mapping relationship can be constructed to facilitate direct determination of the confidence of the first blob analysis result based on this mapping relationship in actual applications. A calculation formula between the morphological closed area change rate and the confidence level can also be pre-designed, and the confidence of the first blob analysis result can be determined using this calculation formula.

[0147] Step S432: Determine a third weight corresponding to the first blob analysis result and a fourth weight corresponding to the second blob analysis result according to the confidence level of the first blob analysis result and the confidence level of the second blob analysis result;

[0148] Step S433: Perform weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain a third Blob analysis result.

[0149] This embodiment evaluates the confidence of the first Blob analysis result from another dimension through the morphological closed area change rate, thereby achieving confidence-weighted Blob analysis result fusion to obtain a third Blob analysis result.

[0150] It is not difficult to understand that this embodiment can be combined with the above-mentioned second embodiment, so as to comprehensively determine the confidence of the first Blob analysis result through the morphological closed area change rate and the normalized average edge gradient amplitude corresponding to the first Blob analysis result, thereby achieving more accurate confidence weighted fusion and further improving the accuracy of the third Blob analysis result.

[0151] In a feasible implementation, the first visual detection result includes a first blob analysis result and a convex hull filling rate corresponding to the first blob analysis result, and the second visual detection result includes a second blob analysis result and a confidence level of the second blob analysis result;

[0152] Step S400 performs weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product, which may include steps S441 to S443:

[0153] Step S441, calculating the confidence level of the first Blob analysis result according to the convex hull filling rate corresponding to the first Blob analysis result;

[0154] Those skilled in the art will appreciate that the convex hull refers to the smallest convex polygon that contains all the pixels of the target connected domain.

[0155] In this embodiment, each connected domain corresponds to a convex hull. The convex hull filling rate refers to the ratio of the area of ​​the connected domain to the area of ​​the convex hull corresponding to the connected domain. The convex hull filling rate corresponding to the first blob analysis result refers to the ratio of the total area of ​​all connected domains involved in the first blob analysis result to the total area of ​​the convex hulls corresponding to each connected domain.

[0156] Exemplarily, when the first Blob analysis result involves connected domains A, B and C, the area of ​​the connected domain A is 10, and the area of ​​the corresponding convex hull a is 12; the area of ​​the connected domain B is 5, and the area of ​​the corresponding convex hull b is 6; the area of ​​the connected domain C is 1, and the area of ​​the corresponding convex hull c is 2. Then the total area of ​​the connected domains A, B and C is 10+5+1=16, and the total area of ​​the convex hulls corresponding to each connected domain is 12+6+2=20. The convex hull filling rate corresponding to the first Blob analysis result is 16 / 20=80%.

[0157] This embodiment notes that when performing Blob analysis, the larger the convex hull filling rate corresponding to the first Blob analysis result, the more complete the connected domain structure, and the more reliable the first Blob analysis result obtained based on this. Therefore, it is proposed that after determining the connected domain in the image to be detected through the connected domain labeling algorithm, the convex hull of each connected domain can be determined to calculate the convex hull filling rate corresponding to the first Blob analysis result, and then the convex hull filling rate can be used to reflect the confidence of the first Blob analysis result.

[0158] The specific confidence calculation method is similar to the above embodiment. The ratio of the error between the first blob analysis result and the actual connected domain attribute information less than a preset value under different convex hull filling rates can be pre-calibrated, and this ratio can be used as the confidence of the convex hull filling rate mapping. A corresponding mapping relationship can be constructed to facilitate direct determination of the confidence of the first blob analysis result based on this mapping relationship in actual applications. A calculation formula between the convex hull filling rate and the confidence level can also be pre-designed, and the confidence of the first blob analysis result can be determined using this calculation formula.

[0159] Step S442: Determine a third weight corresponding to the first blob analysis result and a fourth weight corresponding to the second blob analysis result according to the confidence level of the first blob analysis result and the confidence level of the second blob analysis result;

[0160] Step S443: Perform weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain a third Blob analysis result.

[0161] This embodiment uses the convex hull filling rate to evaluate the confidence of the first blob analysis result from another dimension, thereby achieving confidence-weighted blob analysis result fusion to obtain a third blob analysis result.

[0162] It is not difficult to understand that this embodiment can be combined with the above-mentioned embodiments and implementation methods, so as to comprehensively determine the confidence of the first Blob analysis result through the morphological closed area change rate, normalized average edge gradient amplitude and convex hull filling rate corresponding to the first Blob analysis result, thereby achieving more accurate confidence weighted fusion and further improving the accuracy of the third Blob analysis result.

[0163] In a feasible implementation, the first visual detection result includes a first Blob analysis result, and a normalized average edge gradient amplitude, a morphological closed area change rate, and a convex hull filling rate corresponding to the first Blob analysis result; the second visual detection result includes a second Blob analysis result, and a confidence level of the second Blob analysis result;

[0164] Step S400 performs weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product, which may include steps S451 to S453:

[0165] Step S451, calculating the confidence level of the first Blob analysis result based on the normalized average edge gradient amplitude, morphological closed area change rate, and convex hull filling rate corresponding to the first Blob analysis result;

[0166] Step S452: Determine a third weight corresponding to the first blob analysis result and a fourth weight corresponding to the second blob analysis result according to the confidence level of the first blob analysis result and the confidence level of the second blob analysis result;

[0167] Step S453: Perform weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain a third Blob analysis result.

[0168] In this embodiment, step S451 can first determine the confidence of each mapping based on the mapping relationship between each parameter (i.e., the normalized average edge gradient amplitude, morphological closed area change rate and convex hull filling rate corresponding to the first Blob analysis result) and the confidence, and then determine the confidence of the final first Blob analysis result by taking the average or weighted average.

[0169] In addition, a mathematical model can be established between the normalized average edge gradient amplitude, morphological closed area change rate and convex hull filling rate corresponding to the first Blob analysis result and the confidence level of the first Blob analysis result, so as to calculate the confidence level of the first Blob analysis result through the mathematical model.

[0170] This implementation provides a more accurate confidence assessment by introducing a confidence assessment mechanism based on multiple feature indicators (normalized average edge gradient amplitude, morphological closed area change rate, and convex hull filling rate). Combined with a dynamic weight adjustment strategy, it ensures that high-confidence results are given priority in the fusion process, thereby effectively responding to challenges in various complex scenarios and realizing the intelligent fusion of traditional image processing algorithms and deep learning model output results. It not only gives full play to the efficiency of traditional image processing algorithms under specific conditions, but also makes full use of the powerful representation capabilities of deep learning models, realizing the complementary advantages of the two and greatly improving the overall efficiency of the visual inspection system.

[0171] Based on the above embodiments, a feature extraction method according to a fourth embodiment of the present application is proposed.

[0172] In the fourth embodiment of the present application, for the same or similar contents as those in the above embodiments, please refer to the above introduction and will not be repeated hereafter.

[0173] In this embodiment, the third visual inspection result includes a third size measurement result and a third blob analysis result;

[0174] After step S400 performs weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product, the visual inspection method may further include step S500:

[0175] Step S500 : performing defect type detection on the target product according to the third size measurement result and the third blob analysis result to obtain a defect type detection result of the target product.

[0176] Those skilled in the art will appreciate that defect type detection refers to visual inspection to detect the type of product defects.

[0177] After obtaining the fused third visual inspection result, this embodiment performs more accurate defect type detection on the target product based on the more accurate third dimension measurement result and third blob analysis result in the third visual inspection result, thereby outputting a more accurate defect type detection result and improving the intelligence level and quality control capability of the industrial visual inspection system.

[0178] In order to facilitate understanding of the technical concept of the above embodiments of the present application, a specific embodiment is provided:

[0179] In this specific embodiment, the industrial visual inspection system integrates a hybrid inspection engine, which includes a traditional vision module, a deep learning module, and a result fusion module. The traditional vision module is used for traditional visual inspection, while the deep learning module is used for deep learning-based visual inspection. The result fusion module is used to fuse the results of the two visual inspections using a confidence-weighted method. The traditional vision module integrates dimensional measurement using sub-pixel edge detection, blob analysis using an improved connected domain labeling algorithm, and QR (Quick Response) decoding using a morphologically optimized positioning algorithm. The deep learning module uses a lightweight network architecture to implement dimensional measurement, blob analysis, defect type detection, and character recognition.

[0180] In this specific embodiment, after the industrial visual inspection system detects a visual inspection request based on the target product, it obtains the image to be inspected of the target product, and then uses the traditional visual module in the hybrid detection engine to perform feature extraction on the image to be inspected based on a preset image processing algorithm to obtain the target features of the image to be inspected, and performs visual inspection on the target product based on the target features to obtain a first visual inspection result of the target product. At the same time, through the deep learning module in the hybrid detection engine, the image to be inspected is input into a pre-trained visual inspection deep learning model to obtain a second visual inspection result of the target product output by the visual inspection deep learning model. Finally, through the result fusion module in the hybrid detection engine, the first visual inspection result and the second visual inspection result are weightedly fused to obtain a third visual inspection result of the target product.

[0181] This industrial visual inspection system utilizes a hot-swappable communication protocol configuration method, integrating a protocol template library, a field mapping configurator, and a heartbeat detection module. The protocol template library includes templates for various communication protocols, including TCP / IP (Transmission Control Protocol / Internet Protocol), serial communication, and Modbus communication. The field mapping configurator allows users to define the mapping between data packet structures and device register addresses using an XML (eXtensible Markup Language) configuration file. The heartbeat detection module supports a configurable keepalive mechanism.

[0182] The industrial visual inspection system also includes a code-free debugging mechanism and is equipped with a parameter visual editor. The parameters required for communication protocols or visual inspection can be input through a visual interface, and an XML configuration file can be automatically generated for the system to directly load the corresponding XML configuration file when configuring the communication protocol or performing visual inspection. The operator does not need to have code editing capabilities and does not need to modify the underlying code to flexibly debug the configuration parameters.

[0183] The industrial visual inspection system also supports users to dynamically adjust the area of ​​interest of the image to be inspected through a visual interface, so that visual inspection is only performed on the area of ​​interest set by the user, and the final visual inspection results are superimposed on the area of ​​interest for users to preview.

[0184] The industrial visual inspection system also integrates a protocol simulator, which can be used to pre-simulate whether the configured communication protocol parameters are correct when a new device is connected. The protocol simulator also supports abnormal injection testing.

[0185] like Figure 4 In this specific embodiment, when the user needs to perform industrial visual inspection on a product, he only needs to complete the camera configuration, model configuration, project management and parameter setting in sequence to realize the complete business process setting for industrial visual inspection of the product, so that the industrial vision system installs the business process setting to automatically perform the visual inspection task of the product and output the corresponding visual inspection results.

[0186] It should be noted that the above embodiments / implementations are only used to assist in understanding the present application and do not constitute a limitation on the visual inspection method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0187] In addition, please refer to Figure 5 , Figure 5This is a schematic diagram of the device structure of the hardware operating environment of the electronic device involved in the visual detection method in the embodiment of the present application.

[0188] The present application also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the visual detection method in the above embodiment.

[0189] Reference below Figure 5 , which shows a structural schematic diagram of an electronic device suitable for implementing the embodiments of the present application. The electronic device may include but is not limited to terminal devices such as mobile phones, laptops, tablet computers, desktop computers, or any electronic device that can achieve the above functions. Figure 5 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0190] like Figure 5 As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory 1002 or programs loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the electronic device. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems may be connected to the input / output interface 1006: an input device 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; a storage device 1003 including, for example, a magnetic tape or hard disk; and a communication device 1009. The communication device 1009 may allow the electronic device to communicate with other devices wirelessly or wired to exchange data. Although the figures show electronic devices with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have instead.

[0191] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are performed.

[0192] The electronic device provided in this application, using the visual inspection method described in the above embodiments, can resolve the technical problem of low accuracy of visual inspection results in industrial visual inspection. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the visual inspection method described in the above embodiments, and the other technical features of the electronic device are the same as those disclosed in the above embodiments, which will not be elaborated here.

[0193] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0194] The above are merely specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the above claims.

[0195] In addition, the present application also provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the steps of the visual inspection method in the above embodiment.

[0196] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory (erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0197] The computer-readable storage medium may be an electronic device or included in the electronic device; or may exist independently without being assembled into the electronic device or the electronic device.

[0198] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: after detecting a visual inspection request based on the target product, obtains an image of the target product to be inspected; extracts features of the image to be inspected based on a preset image processing algorithm to obtain target features of the image to be inspected, and performs visual inspection of the target product based on the target features to obtain a first visual inspection result of the target product, wherein the image processing algorithm includes an edge detection algorithm and / or a connected domain labeling algorithm, and the first visual inspection result includes a first size measurement result and / or a first blob analysis result; inputs the image to be inspected into a pre-trained visual inspection deep learning model to obtain a second visual inspection result of the target product output by the visual inspection deep learning model, wherein the visual inspection deep learning model includes a size measurement deep learning model and / or a blob analysis deep learning model, and the second visual inspection result includes a second size measurement result and / or a second blob analysis result; and performs weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product, wherein the third visual inspection result includes a third size measurement result and / or a third blob analysis result.

[0199] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0200] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0201] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0202] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the steps of the aforementioned visual inspection method. This can address the technical issue of low accuracy in industrial visual inspection results. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the visual inspection method provided in the aforementioned embodiments and are not further elaborated here.

[0203] In addition, an embodiment of the present application also provides a computer program product, including a computer program, which implements the steps of the visual detection method in the above embodiment when executed by a processor.

[0204] The computer program product provided in this application can solve the technical problem of low accuracy of visual inspection results in industrial visual inspection. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of this application are the same as the beneficial effects of the visual inspection method provided in the above embodiments, and will not be repeated here.

[0205] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A visual inspection method, characterized in that: The method comprises: After detecting a visual inspection request based on a target product, obtaining an image of the target product to be inspected; Performing feature extraction on the image to be inspected based on a preset image processing algorithm to obtain target features of the image to be inspected, and performing visual inspection on the target product based on the target features to obtain a first visual inspection result of the target product, wherein the image processing algorithm includes an edge detection algorithm and / or a connected component labeling algorithm, and the first visual inspection result includes a first size measurement result and / or a first blob analysis result; Inputting the image to be inspected into a pre-trained visual inspection deep learning model to obtain a second visual inspection result of the target product output by the visual inspection deep learning model, wherein the visual inspection deep learning model includes a dimensional measurement deep learning model and / or a blob analysis deep learning model, and the second visual inspection result includes a second dimensional measurement result and / or a second blob analysis result; Perform confidence-weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product, wherein the third visual inspection result includes a third size measurement result and / or a third blob analysis result.

2. The visual inspection method according to claim 1, wherein: The first visual detection result includes a first size measurement result and a normalized average edge gradient amplitude corresponding to the first size measurement result; the second visual detection result includes a second size measurement result and a confidence level of the second size measurement result; The step of performing confidence-weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product includes: Calculating a confidence level of the first size measurement result according to the normalized average edge gradient amplitude corresponding to the first size measurement result; Determining, according to the confidence level of the first size measurement result and the confidence level of the second size measurement result, a first weight corresponding to the first size measurement result and a second weight corresponding to the second size measurement result; The first size measurement result and the second size measurement result are weightedly fused according to the first weight and the second weight to obtain a third size measurement result.

3. The visual inspection method according to claim 2, wherein: The step of performing weighted fusion on the first size measurement result and the second size measurement result according to the first weight and the second weight to obtain a third size measurement result includes: Acquiring shooting environment information of the image to be detected, and adjusting the first weight and the second weight according to the shooting environment information; The first size measurement result and the second size measurement result are weightedly fused according to the adjusted first weight and the adjusted second weight to obtain a third size measurement result.

4. The visual inspection method according to claim 1, wherein: The first visual detection result includes a first blob analysis result and a normalized average edge gradient amplitude corresponding to the first blob analysis result; the second visual detection result includes a second blob analysis result and a confidence level of the second blob analysis result; The step of performing confidence-weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product includes: Calculating a confidence level of the first Blob analysis result according to the normalized average edge gradient amplitude corresponding to the first Blob analysis result; Determining, according to the confidence level of the first blob analysis result and the confidence level of the second blob analysis result, a third weight corresponding to the first blob analysis result and a fourth weight corresponding to the second blob analysis result; The first blob analysis result and the second blob analysis result are weightedly fused according to the third weight and the fourth weight to obtain a third blob analysis result.

5. The visual inspection method according to claim 1, wherein: The first visual detection result includes a first blob analysis result and a morphological closed area change rate corresponding to the first blob analysis result; the second visual detection result includes a second blob analysis result and a confidence level of the second blob analysis result; The step of performing confidence-weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product includes: Calculating the confidence level of the first Blob analysis result according to the morphological closed area change rate corresponding to the first Blob analysis result; Determining, according to the confidence level of the first blob analysis result and the confidence level of the second blob analysis result, a third weight corresponding to the first blob analysis result and a fourth weight corresponding to the second blob analysis result; The first blob analysis result and the second blob analysis result are weightedly fused according to the third weight and the fourth weight to obtain a third blob analysis result.

6. The visual inspection method according to claim 1, wherein: The first visual detection result includes a first blob analysis result and a convex hull filling rate corresponding to the first blob analysis result, and the second visual detection result includes a second blob analysis result and a confidence level of the second blob analysis result; The step of performing confidence-weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product includes: Calculating the confidence level of the first Blob analysis result according to the convex hull filling rate corresponding to the first Blob analysis result; Determining, according to the confidence level of the first blob analysis result and the confidence level of the second blob analysis result, a third weight corresponding to the first blob analysis result and a fourth weight corresponding to the second blob analysis result; The first blob analysis result and the second blob analysis result are weightedly fused according to the third weight and the fourth weight to obtain a third blob analysis result.

7. The visual inspection method according to claim 1, wherein: The first visual detection result includes a first blob analysis result, and a normalized average edge gradient amplitude, a morphological closed area change rate, and a convex hull filling rate corresponding to the first blob analysis result; the second visual detection result includes a second blob analysis result, and a confidence level of the second blob analysis result; The step of performing confidence-weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product includes: Calculating the confidence level of the first Blob analysis result based on the normalized average edge gradient amplitude, the morphological closed area change rate, and the convex hull filling rate corresponding to the first Blob analysis result; Determining, according to the confidence level of the first blob analysis result and the confidence level of the second blob analysis result, a third weight corresponding to the first blob analysis result and a fourth weight corresponding to the second blob analysis result; The first blob analysis result and the second blob analysis result are weightedly fused according to the third weight and the fourth weight to obtain a third blob analysis result.

8. The visual inspection method according to any one of claims 1 to 7, wherein: The third visual inspection result includes a third size measurement result and a third Blob analysis result; After the step of performing confidence-weighted fusion on the first visual inspection result and the second visual inspection result to obtain a third visual inspection result of the target product, the method further includes: According to the third size measurement result and the third blob analysis result, defect type detection is performed on the target product to obtain a defect type detection result of the target product.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the visual inspection method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the visual inspection method according to any one of claims 1 to 8.

Images