Visual inspection method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a visual inspection method, electronic equipment and a computer readable storage medium, and relates to the technical field of industrial visual inspection.The visual inspection method comprises the steps that after a visual inspection request based on a target product is detected, a to-be-detected image of the target product is obtained; performing feature extraction on the to-be-detected image based on a preset image processing algorithm to obtain a target feature of the to-be-detected image, and performing visual detection on the target product based on the target feature to obtain a first visual detection result of the target product; inputting the to-be-detected image into a pre-trained visual detection deep learning model to obtain a second visual detection result of the target product output by the visual detection deep learning model; and performing weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product. The accuracy of a visual detection result can be improved.

Description

Technical Field

[0001] This application relates to the field of industrial vision inspection technology, and particularly to a vision inspection method, an electronic device, and a computer-readable storage medium. Background Art

[0002] In the field of industrial manufacturing, vision inspection technology is one of the core means to ensure product quality and improve the level of production automation. With the rapid development of modern manufacturing towards high precision and high efficiency, applications such as defect detection, dimension measurement, and Blob (Binary Large Object, connected domain) analysis based on machine vision have become indispensable in high-end industries such as semiconductors, automobiles, and electronics. Vision inspection can, through the characteristics of non-contact, high speed, and high consistency, identify problems such as appearance defects and geometric parameter deviations of products in real time, significantly reduce the cost of manual inspection, and improve the product yield rate, which is of great significance to the intelligentization and quality control of the production process.

[0003] Currently, industrial vision inspection mainly relies on two types of technologies: one is rule-driven vision inspection (hereinafter referred to as traditional vision inspection), and the other is data-driven vision inspection (hereinafter referred to as deep learning-based vision inspection). However, traditional vision inspection usually relies on manually designed feature extraction rules and has poor robustness to light changes, noise interference, and complex backgrounds, resulting in size measurement deviations or Blob region segmentation errors in scenarios such as blurred feature edges, low contrast, or target adhesion. Although deep learning-based vision inspection can adaptively learn complex features, its detection performance highly depends on the scale and quality of training data. When the data volume is insufficient or the sample distribution is uneven, problems such as model overfitting or insufficient generalization ability are likely to occur, and the model decision-making process lacks interpretability, resulting in difficulties in accurately associating the setting of key parameters (such as size error thresholds and connected domain morphological features) with actual physical characteristics, further affecting the reliability of the detection results.

[0004] In summary, how to effectively improve the accuracy of vision inspection results has become an urgent technical problem in the industry. Summary of the Invention

[0005] The main purpose of this application is to provide a vision inspection method, aiming to solve the technical problem of low accuracy of vision inspection results in industrial vision inspection.

[0006] To achieve the above purpose, this application provides a vision inspection method, which includes: After detecting a vision inspection request based on a target product, obtain a to-be-inspected image of the target product; Feature extraction is performed on the image to be detected based on a preset image processing algorithm to obtain the target features of the image to be detected, and visual inspection is performed on the target product based on the target features to obtain the first visual inspection result of the target product. Among them, the image processing algorithm includes an edge detection algorithm and / or a connected component labeling algorithm, and the first visual inspection result includes a first size measurement result and / or a first Blob analysis result; The image to be detected is input into a pre-trained visual inspection deep learning model to obtain the second visual inspection result of the target product output by the visual inspection deep learning model. Among them, the visual inspection deep learning model includes a size measurement deep learning model and / or a Blob analysis deep learning model, and the second visual inspection result includes a second size measurement result and / or a second Blob analysis result; The first visual inspection result and the second visual inspection result are weighted and fused to obtain the third visual inspection result of the target product. Among them, the third visual inspection result includes a third size measurement result and / or a third Blob analysis result.

[0007] In an embodiment, the first visual inspection result includes a first size measurement result and the normalized average edge gradient amplitude corresponding to the first size measurement result, and the second visual inspection result includes a second size measurement result and the confidence level of the second size measurement result; The step of weighting and fusing the first visual inspection result and the second visual inspection result to obtain the third visual inspection result of the target product includes: According to the normalized average edge gradient amplitude corresponding to the first size measurement result, the confidence level of the first size measurement result is calculated; According to the confidence level of the first size measurement result and the confidence level of the second size measurement result, a first weight corresponding to the first size measurement result and a second weight corresponding to the second size measurement result are determined; According to the first weight and the second weight, the first size measurement result and the second size measurement result are weighted and fused to obtain a third size measurement result.

[0008] In an embodiment, the step of weighting and fusing the first size measurement result and the second size measurement result according to the first weight and the second weight to obtain a third size measurement result includes: Obtain the shooting environment information of the image to be detected, and adjust the first weight and the second weight according to the shooting environment information; Based on the adjusted first weight and the adjusted second weight, perform weighted fusion on the first dimension measurement result and the second dimension measurement result to obtain a third dimension measurement result.

[0009] In one embodiment, the first visual detection result includes a first Blob analysis result and the normalized average edge gradient magnitude corresponding to the first Blob analysis result, and the second visual detection result includes a second Blob analysis result and the confidence level of the second Blob analysis result. The step of performing weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product includes: Calculate the confidence level of the first Blob analysis result based on the normalized average edge gradient magnitude corresponding to the first Blob analysis result. Determine the third weight corresponding to the first Blob analysis result and the fourth weight corresponding to the second Blob analysis result based on the confidence level of the first Blob analysis result and the confidence level of the second Blob analysis result. Based on the third weight and the fourth weight, perform weighted fusion on the first Blob analysis result and the second Blob analysis result to obtain a third Blob analysis result.

[0010] In one embodiment, the first visual detection result includes a first Blob analysis result and the morphological closing area change rate corresponding to the first Blob analysis result, and the second visual detection result includes a second Blob analysis result and the confidence level of the second Blob analysis result. The step of performing weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product includes: Calculate the confidence level of the first Blob analysis result based on the morphological closing area change rate corresponding to the first Blob analysis result. Determine the third weight corresponding to the first Blob analysis result and the fourth weight corresponding to the second Blob analysis result based on the confidence level of the first Blob analysis result and the confidence level of the second Blob analysis result. Based on the third weight and the fourth weight, perform weighted fusion on the first Blob analysis result and the second Blob analysis result to obtain a third Blob analysis result.

[0011] In one embodiment, the first visual detection result includes a first Blob analysis result and the convex hull filling rate corresponding to the first Blob analysis result, and the second visual detection result includes a second Blob analysis result and the confidence level of the second Blob analysis result; The step of performing weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product includes: Calculate the confidence level of the first Blob analysis result according to the convex hull filling rate corresponding to the first Blob analysis result; Determine a third weight corresponding to the first Blob analysis result and a fourth weight corresponding to the second Blob analysis result according to the confidence level of the first Blob analysis result and the confidence level of the second Blob analysis result; Perform weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain a third Blob analysis result.

[0012] In one embodiment, the first visual detection result includes a first Blob analysis result, the normalized average edge gradient magnitude, the morphological closing area change rate, and the convex hull filling rate corresponding to the first Blob analysis result, and the second visual detection result includes a second Blob analysis result and the confidence level of the second Blob analysis result; The step of performing weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product includes: Calculate the confidence level of the first Blob analysis result according to the normalized average edge gradient magnitude, the morphological closing area change rate, and the convex hull filling rate corresponding to the first Blob analysis result; Determine a third weight corresponding to the first Blob analysis result and a fourth weight corresponding to the second Blob analysis result according to the confidence level of the first Blob analysis result and the confidence level of the second Blob analysis result; Perform weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain a third Blob analysis result.

[0013] In one embodiment, the third visual detection result includes a third dimension measurement result and a third Blob analysis result; After the step of performing weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product, the method further includes: Based on the third dimension measurement result and the third Blob analysis result, defect type detection is performed on the target product to obtain the defect type detection result of the target product.

[0014] In addition, to achieve the above object, the present application further provides an electronic device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps of the visual detection method as described above are implemented.

[0015] In addition, to achieve the above object, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the visual detection method as described above are implemented.

[0016] In addition, to achieve the above object, the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the visual detection method as described above are implemented.

[0017] The embodiment of the present application provides a visual detection method, an electronic device, and a computer-readable storage medium. The visual detection method includes: after detecting a visual detection request based on a target product, obtaining a to-be-detected image of the target product; performing feature extraction on the to-be-detected image based on a preset image processing algorithm to obtain target features of the to-be-detected image, and performing visual detection on the target product based on the target features to obtain a first visual detection result of the target product, where the image processing algorithm includes an edge detection algorithm and / or a connected component labeling algorithm, and the first visual detection result includes a first dimension measurement result and / or a first Blob analysis result; inputting the to-be-detected image into a pre-trained visual detection deep learning model to obtain a second visual detection result of the target product output by the visual detection deep learning model, where the visual detection deep learning model includes a dimension measurement deep learning model and / or a Blob analysis deep learning model, and the second visual detection result includes a second dimension measurement result and / or a second Blob analysis result; performing weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product, where the third visual detection result includes a third dimension measurement result and / or a third Blob analysis result.

[0018] In the embodiments of the present application, through a weighted decision-making mechanism, heterogeneous results of two visual detection technologies are integrated, effectively solving the problems that traditional visual detection is too sensitive to the environment and limited by manual experience, as well as the problem that visual detection based on deep learning is overly dependent on sample data, and significantly improving the accuracy and robustness of industrial visual detection. Specifically, in the embodiments of the present application, first, traditional image processing algorithms such as edge detection and connected component labeling are used to extract geometric features (such as edge contours and regional connectivity) of the target product, thereby generating the first visual detection result. At the same time, an adaptive feature learning and complex pattern recognition are performed on the same image to be detected through a dedicated deep learning model for size measurement or Blob analysis, thereby outputting the second visual detection result. Finally, based on a preset weight coefficient or a dynamic weight adjustment strategy, the two types of results are weighted and fused (such as linear weighting, confidence weighting), and through complementary decision optimization, both the accuracy of traditional visual detection in regular features is retained, and the strong representation ability of deep learning for non-linear features is utilized to make up for the detection deviation of traditional visual detection in complex scenarios such as fuzzy edges and noise interference, so as to reduce the size measurement error and improve the accuracy of Blob analysis in variable industrial scenarios such as light fluctuations, target adhesion, and data scarcity, and further solve the technical problem of low accuracy of visual detection results in industrial visual detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0020] To more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0021] Figure 1 It is a schematic flowchart provided for the first embodiment of the visual detection method of the present application; Figure 2 It is a schematic flowchart provided for the second embodiment of the visual detection method of the present application; Figure 3 It is a schematic flowchart provided for the third embodiment of the visual detection method of the present application; Figure 4 It is a system flowchart provided for a specific implementation of the present application; Figure 5 It is a schematic diagram of the device structure of the hardware operating environment of the electronic device involved in the visual detection method in the embodiments of the present application.

[0022] The implementation, functional features, and advantages of the present application will be further described in conjunction with embodiments and with reference to the accompanying drawings. Detailed implementation manners

[0023] Here, exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0024] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0025] In industrial vision inspection, the current mainstream vision inspection technologies are: traditional vision inspection and deep learning-based vision inspection.

[0026] Traditional vision inspection relies on feature extraction rules designed manually, and the accuracy and reliability of its inspection results are limited by environmental interferences (such as light changes, background noise) and the subjectivity of manual experience (such as deviations in parameter settings of feature extraction rules). Although deep learning-based vision inspection can automatically extract abstract features through data driving, the accuracy and reliability of its inspection results are limited by the scale and quality of training data. In the case of insufficient or unevenly distributed training data, the model may not be able to accurately identify target features, resulting in unstable inspection results.

[0027] For the above problems, the main solution of the embodiments of this application is as follows: after detecting a visual detection request based on a target product, obtain the image to be detected of the target product; extract features from the image to be detected based on a preset image processing algorithm to obtain the target features of the image to be detected, and perform visual detection on the target product based on the target features to obtain the first visual detection result of the target product, where the image processing algorithm includes an edge detection algorithm and / or a connected component labeling algorithm, and the first visual detection result includes a first dimension measurement result and / or a first blob analysis result; input the image to be detected into a pre-trained visual detection deep learning model to obtain the second visual detection result of the target product output by the visual detection deep learning model, where the visual detection deep learning model includes a dimension measurement deep learning model and / or a blob analysis deep learning model, and the second visual detection result includes a second dimension measurement result and / or a second blob analysis result; perform weighted fusion on the first visual detection result and the second visual detection result to obtain the third visual detection result of the target product, where the third visual detection result includes a third dimension measurement result and / or a third blob analysis result.

[0028] The embodiments of this application effectively solve the problems that traditional visual detection is too sensitive to the environment and limited by manual experience, and the visual detection based on deep learning is too dependent on sample data by using a weighted decision-making mechanism to fuse the heterogeneous results of two visual detection technologies, significantly improving the accuracy and robustness of industrial visual detection. Specifically, the embodiments of this application first use traditional image processing algorithms such as edge detection and connected component labeling to extract the geometric features (such as edge contours and regional connectivity) of the target product, thereby generating the first visual detection result. At the same time, the same image to be detected is subjected to adaptive feature learning and complex pattern recognition through a dedicated deep learning model for dimension measurement or blob analysis, thereby outputting the second visual detection result. Finally, based on a preset weight coefficient or a dynamic weight adjustment strategy, weighted fusion (such as linear weighting and confidence weighting) is performed on the two types of results. Through complementary decision optimization, the accuracy of traditional visual detection in regular features is retained, and the strong representation ability of deep learning for non-linear features is utilized to make up for the detection deviation of traditional visual detection in complex scenarios such as blurred edges and noise interference. Thus, in variable industrial scenarios such as light fluctuations, target adhesion, and data scarcity, the dimension measurement error is reduced, the accuracy of blob analysis is improved, and further the technical problem of low accuracy of visual detection results in industrial visual detection is solved.

[0029] It should be noted that the execution subject of the embodiments of the present application may include, but is not limited to, terminal devices such as mobile phones, laptop computers, tablet computers, desktop computers, etc., vision detection systems, or any electronic device capable of implementing the above functions. The following takes the vision detection system as the execution subject as an example to illustrate the following embodiments of the present application.

[0030] To better understand the technical solution of the present application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0031] The present application proposes a vision detection method for the first embodiment.

[0032] Please refer to Figure 1 , Figure 1 which is a schematic flowchart provided for the first embodiment of the vision detection method of the present application.

[0033] In this embodiment, the vision detection method may include steps S100 to S300: Step S100, after detecting a vision detection request based on a target product, obtain a to-be-detected image of the target product; It should be noted that the target product refers to an item or component that needs to be visually inspected for appearance defects, size measurement, shape analysis, etc. through a vision detection system during industrial production or quality control. The target product may include, but is not limited to, electronic components, mechanical parts, food packaging, textiles, automotive parts, etc. The to-be-detected image refers to a digital image of the target product collected by an industrial camera, a linear array sensor, or a 3D scanning device, and contains information such as the target surface texture, geometric contour, and defect characteristics.

[0034] In this embodiment, the vision detection request refers to an instruction that triggers the vision detection system to execute a vision detection task, and can be generated by an external device (such as a programmable logic controller), a preset trigger condition (such as a conveyor belt position sensor signal), or user interface interaction (such as a touch screen operation).

[0035] It is worth mentioning that the vision detection request can specify which vision detection tasks to perform on the target product, such as size measurement, Blob analysis, etc.

[0036] In this embodiment, the vision detection system can parse the vision detection request to determine the target product, and obtain the to-be-detected image of the target product, so as to perform the vision detection tasks specified by the vision detection request on the target product.

[0037] Step S200, perform feature extraction on the to-be-detected image based on a preset image processing algorithm to obtain the target features of the to-be-detected image, and perform vision detection on the target product based on the target features to obtain the first vision detection result of the target product; Among them, the image processing algorithms include edge detection algorithms and / or connected component labeling algorithms, and the first visual detection result includes a first dimension measurement result and / or a first blob analysis result; As known to those skilled in the art, the edge detection algorithm is a technique specifically used to identify the boundaries of objects in an image. By calculating the rate of change of pixel values in the image, it locates the positions of the edges, and can highlight the areas where significant changes in gray levels occur in the image, that is, the object contours. The connected component labeling algorithm is a technique used to segment independent regions in an image and assign a unique identifier to each region. By scanning the image, pixels with the same or similar attributes (such as color, brightness) and adjacent to each other are grouped, and a unique label is assigned to each group. Common connected component labeling algorithms include the two-pass scanning algorithm.

[0038] In this embodiment, the target feature refers to the feature extracted from the image to be detected that can represent the characteristics of the target product. It is not difficult to understand that different target features are extracted by different image processing algorithms. The target feature extracted by the edge detection algorithm is the boundary or contour information of the object in the image to be detected, and the target feature extracted by the connected component labeling algorithm is the connected components in the image to be detected.

[0039] It should be noted that before extracting features from the image to be detected through the image processing algorithm in this embodiment, it is necessary to preprocess the image to be detected first. It is not difficult to understand that different image preprocessing operations are required for different image processing algorithms. Those skilled in the art have conducted in-depth research on this, and this embodiment will not elaborate too much on this.

[0040] In this embodiment, the first visual detection result refers to the visual detection result obtained by performing traditional visual detection on the target product. This first visual detection result may include a first dimension measurement result and / or a first blob analysis result. Among them, the first dimension measurement result refers to the physical dimension parameters of the target product obtained through geometric calculation based on the target feature (i.e., the boundary or contour information of the object in the image to be detected) extracted by the edge detection algorithm. The first blob analysis result refers to the attribute information (such as area, centroid, shape, etc.) of each connected component on the target product obtained through geometric calculation based on the target feature (i.e., the connected components in the image to be detected) extracted by the connected component labeling algorithm.

[0041] It is worth mentioning that in this embodiment, when performing dimension measurement, the sub-pixel edge detection algorithm can be combined to further improve the accuracy of the first dimension measurement result. When performing blob analysis, the connected component labeling improvement algorithm can be used, for example, using a tree structure to represent the relationship between connected regions, and a tree merging strategy that dynamically merges these trees during the labeling process, so as to reduce repeated calculations and improve the merging efficiency.

[0042] After detecting a visual inspection request for a target product, according to the parsing result of the visual inspection request, this embodiment determines which types of visual inspections need to be performed on the target product. Thus, when the visual inspection request specifies performing dimensional measurement on the target product, feature extraction is performed on the image to be inspected through a preset edge detection algorithm, and the boundary or contour information of the object in the image to be inspected is obtained as the target feature. Based on this target feature, geometric calculations are performed to complete the visual inspection of dimensional measurement, and the first dimensional measurement result is obtained as the first visual inspection result of the target product.

[0043] Correspondingly, when the visual inspection request specifies performing Blob analysis on the target product, this embodiment performs feature extraction on the image to be inspected through a preset connected component labeling algorithm, and the connected components in the image to be inspected are obtained as the target feature. Based on this target feature, geometric calculations are performed to complete the visual inspection of Blob analysis, and the first Blob analysis result is obtained as the first visual inspection result of the target product.

[0044] It is not difficult to understand that when the visual inspection request specifies performing dimensional measurement and Blob analysis on the target product, this embodiment calls the corresponding image processing algorithms to extract the required target features to complete the corresponding visual inspections, so as to obtain the first dimensional measurement result and the first Blob analysis result as the first visual inspection result of the target product.

[0045] Step S300: Input the image to be inspected into a pre-trained deep learning model for visual inspection, and obtain the second visual inspection result of the target product output by the deep learning model for visual inspection; Among them, the deep learning model for visual inspection includes a deep learning model for dimensional measurement and / or a deep learning model for Blob analysis, and the second visual inspection result includes a second dimensional measurement result and / or a second Blob analysis result; It should be noted that the deep learning model for visual inspection is a model pre-trained based on a deep learning model for performing visual inspection on a target product. In this embodiment, according to the type division of the visual inspection task, a dedicated deep learning model for visual inspection is correspondingly trained.

[0046] Exemplarily, for the visual inspection task of dimensional measurement of a target product, this embodiment can pre-take a large number of products of the same type as the target product to obtain corresponding images, and through manual measurement, obtain the actual physical dimension parameters of these products and mark them on the corresponding images to form training data. Thus, a suitable deep learning model and loss function are selected, and training is performed through these training data to finally obtain a trained deep learning model for dimensional measurement.

[0047] Accordingly, for the visual inspection task of Blob analysis of the target product, in this embodiment, a large number of products of the same type as the target product can be pre-shot to obtain corresponding images, and the attribute information of the connected regions in these images can be obtained by manual measurement and marked on the corresponding images to form training data. Then, a suitable deep learning model and loss function are selected and trained with these training data to finally obtain a trained deep learning model for Blob analysis.

[0048] It is worth mentioning that when selecting a suitable deep learning model to train the above-mentioned visual inspection deep learning model, a deep learning model with a lightweight network architecture can be selected to facilitate the deployment of the trained visual inspection deep learning model to resource-constrained devices, so as to be integrated with traditional visual inspection without the need for separate deployment, which may lead to data barriers in the detection process and avoid the user having to frequently switch between different devices or applications to detect the visual inspection results output by the two visual inspection methods.

[0049] In this embodiment, the second visual inspection result refers to the visual inspection result obtained by performing deep learning-based visual inspection on the target product. The second visual inspection result may include a second dimension measurement result and / or a second Blob analysis result. Among them, the second dimension measurement result refers to the physical dimension parameters of the target product predicted by the dimension measurement deep learning model. The second Blob analysis result refers to the attribute information (such as area, centroid, shape, etc.) of each connected region on the target product predicted by the Blob analysis deep learning model.

[0050] It can be understood that when performing visual inspection on the target product through the visual inspection deep learning model, the image to be inspected of the target product can be preprocessed first to improve the image quality, and then converted into a form convenient for inputting into the visual inspection deep learning model, such as a tensor form.

[0051] After detecting a visual inspection request based on the target product, in this embodiment, the corresponding visual inspection deep learning model is selected according to the visual inspection task specified in the visual inspection request to perform visual inspection on the target product, thereby obtaining the second visual inspection result of the target product.

[0052] Step S400, perform weighted fusion on the first visual inspection result and the second visual inspection result to obtain the third visual inspection result of the target product; Among them, the third visual inspection result includes a third dimension measurement result and / or a third Blob analysis result.

[0053] In this embodiment, the third visual detection result refers to the visual detection result obtained by weighted fusion of the first visual detection result and the second visual detection result. The third visual detection result may include a third dimension measurement result and / or a third Blob analysis result. Among them, the third dimension measurement result refers to the dimension measurement result obtained by weighted fusion of the first dimension measurement result and the second dimension measurement result, and the third Blob analysis result refers to the Blob analysis result obtained by weighted fusion of the first Blob analysis result and the second Blob analysis result.

[0054] In this embodiment, through a weighted decision-making mechanism, heterogeneous results of two visual detection technologies are fused, effectively solving the problems that traditional visual detection is too sensitive to the environment and limited by manual experience, and the visual detection based on deep learning is too dependent on sample data, and significantly improving the accuracy and robustness of industrial visual detection. Specifically, in this embodiment, traditional image processing algorithms such as edge detection and connected component labeling are first used to extract the geometric features (such as edge contours and regional connectivity) of the target product, thereby generating the first visual detection result. At the same time, an adaptive feature learning and complex pattern recognition are performed on the same image to be detected through a dedicated deep learning model for dimension measurement or Blob analysis, thereby outputting the second visual detection result. Finally, based on a preset weight coefficient or a dynamic weight adjustment strategy, the two types of results are weighted and fused (such as linear weighting, confidence weighting), and through complementary decision optimization, both the accuracy of traditional visual detection on regular features and the strong representation ability of deep learning on non-linear features are retained, making up for the detection deviation of traditional visual detection in complex scenarios such as blurred edges and noise interference, so as to reduce the dimension measurement error and improve the Blob analysis accuracy in variable industrial scenarios such as light fluctuations, target adhesion, and data scarcity, and further solve the technical problem of low accuracy of visual detection results in industrial visual detection.

[0055] In this embodiment, the method of weighted fusion may be fixed-weight fusion or dynamic-weight fusion. Among them, in the method of fixed-weight fusion, the weight coefficients of the two visual detection results can be preset by the user or the system, so as to perform weighted fusion with fixed weights.

[0056] Regarding dynamic-weight fusion, the following takes dimension measurement as an example for detailed description.

[0057] Based on the above first embodiment, a feature extraction method according to the second embodiment of the present application is proposed.

[0058] In the second embodiment of the present application, the same or similar content as that in the above embodiment can be referred to the above introduction and will not be repeated hereinafter.

[0059] Please refer to Figure 2 , Figure 2Schematic flowchart provided for the second embodiment of the feature extraction method of this application.

[0060] In this embodiment, the first visual detection result includes a first size measurement result and the normalized average edge gradient magnitude corresponding to the first size measurement result, and the second visual detection result includes a second size measurement result and the confidence level of the second size measurement result. Step S400 performs weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product, which may include steps S411 to S413: Step S411 calculates the confidence level of the first size measurement result based on the normalized average edge gradient magnitude corresponding to the first size measurement result. In this embodiment, the normalized average edge gradient magnitude is an index used to quantify the edge sharpness, and its calculation formula is as follows: ; where is the normalized average edge gradient magnitude, N is the total number of edge pixels, is the edge pixel 's gradient magnitude, is the maximum gradient magnitude in the image to be detected.

[0061] In this embodiment, when measuring the size of the target product, it is necessary to first perform edge detection on the image to be detected of the target product to determine each edge pixel in the image to be detected, that is, the pixel whose gradient magnitude is greater than the preset value, and then the normalized average edge gradient magnitude corresponding to the first size measurement result can be calculated through the above calculation formula.

[0062] This embodiment notices that when performing size measurement, the higher the gradient magnitude of each edge pixel extracted by the edge detection algorithm (that is, the clearer the edge), the more accurate the size measurement result. Therefore, a solution is proposed to calculate the normalized average edge gradient magnitude corresponding to the first size measurement result (that is, normalizing and then averaging the gradient magnitudes of each edge pixel extracted by the edge detection algorithm) to reflect the confidence level of the first size measurement result.

[0063] Specifically, this embodiment can directly use the normalized average edge gradient magnitude corresponding to the first size measurement result as the confidence level of the first size measurement result, or perform certain mathematical operations on the basis of the normalized average edge gradient magnitude corresponding to the first size measurement result to obtain the confidence level of the first size measurement result.

[0064] In addition, it is also possible to pre-calibrate the proportion of the error between the first size measurement result and the actual physical size parameter being less than a preset value under different normalized average edge gradient amplitudes, and use this proportion as the confidence level mapped by the normalized average edge gradient amplitude, and construct the corresponding mapping relationship, so as to directly determine the confidence level of the first size measurement result according to this mapping relationship in practical applications.

[0065] Step S412: Determine the first weight corresponding to the first size measurement result and the second weight corresponding to the second size measurement result according to the confidence level of the first size measurement result and the confidence level of the second size measurement result. It should be noted that the first weight refers to the weight proportion of the first size measurement result during weighted fusion, and the second weight refers to the weight proportion of the second size measurement result during weighted fusion.

[0066] In this embodiment, when the size measurement deep learning model outputs the predicted second size measurement result, it also outputs the confidence level of the second size measurement result.

[0067] In this embodiment, when determining the weights according to the confidence levels, for example, if the confidence level of the first size measurement result is 0.9 and the confidence level of the second size measurement result is 0.6, then the first weight of the first size measurement result can be 0.9 / (0.9 + 0.6) = 0.6. Correspondingly, the weight of the second size measurement result can be 0.6 / (0.9 + 0.6) = 0.4.

[0068] In addition, when the confidence level of the first size measurement result is greater than 0.9, it can be determined that the first weight of the first size measurement result is equal to the confidence level of the first size measurement result. And when the confidence level of the first size measurement result is less than 0.6 and the confidence level of the second size measurement result is greater than 0.8, it can be determined that the second weight of the second size measurement result is equal to the confidence level of the second size measurement result.

[0069] It is not difficult to understand that the sum of the first weight and the second weight is 1, and the value range of the confidence level is from 0 to 1.

[0070] The specific confidence level weighting rule can be flexibly set according to actual needs. The above examples are for reference only, and this embodiment does not make specific limitations in this regard.

[0071] Step S413: Perform weighted fusion on the first size measurement result and the second size measurement result according to the first weight and the second weight to obtain the third size measurement result.

[0072] In this embodiment, by mapping the normalized average edge gradient magnitude to confidence, the environmental adaptability defect of the traditional visual detection method is for the first time transformed into a quantifiable weight parameter, enabling the weighted fusion process to dynamically respond to image quality changes. For example, when edge blurring is caused by low illuminance, the first weight of the first size measurement result is automatically reduced to avoid error propagation.

[0073] After calculating the confidence of the first size measurement result in this embodiment, the weight ratio of the first size measurement result and the second size measurement result during weighted fusion can be dynamically adjusted according to the confidence of the first size measurement result and the confidence of the second size measurement result, so as to achieve complementary decision optimization, retaining both the accuracy of the traditional visual detection in regular features and making use of the strong representation ability of deep learning for non-linear features to make up for the detection deviation of the traditional visual detection in complex scenarios such as blurred edges and noise interference, and further reducing the size measurement error in variable industrial scenarios such as light fluctuations, target adhesion, and data scarcity.

[0074] Further, in a feasible implementation manner, step S413 may include steps A10 to A20: Step A10, obtaining the shooting environment information of the image to be detected, and adjusting the first weight and the second weight according to the shooting environment information; It should be noted that the shooting environment information refers to the set of external condition parameters that affect the visual detection accuracy during image acquisition, and may include but are not limited to illuminance, light source type, camera gain and exposure time, relative movement speed between the camera and the target product, etc.

[0075] Step A20, performing weighted fusion on the first size measurement result and the second size measurement result according to the adjusted first weight and the adjusted second weight to obtain a third size measurement result.

[0076] On the basis of dynamic weight adjustment, this implementation manner further introduces the shooting environment information as a weight correction factor to achieve a more refined weight allocation strategy. Specifically, step A10 dynamically corrects the initial weight obtained in step S412 by real-time sensing or calculating the interference factors (such as light fluctuations, motion blur) in the shooting environment; step A20 then completes the weighted fusion based on the corrected weight, enabling the third size measurement result to take into account both the algorithm confidence and environmental adaptability, thus breaking through the limitation of solely relying on the confidence of the size measurement result. For example, when it is known that the illuminance is lower than 50 Lux, even if the confidence of the first size measurement result is high, the first weight is still actively reduced to prevent potential light mutation risks. Another example is that when the camera gain is too high resulting in significant image noise, even if the confidence of the first size measurement result is high, the second weight is still actively increased to use the noise suppression ability of the deep learning model to compensate for the measurement deviation of the traditional visual detection.

[0077] Regarding dynamic weight fusion, the following takes Blob analysis as an example for detailed description.

[0078] Based on the above first embodiment, a feature extraction method according to the third embodiment of the present application is proposed.

[0079] In the third embodiment of the present application, the same or similar content as the above embodiment can be referred to the above introduction and will not be repeated hereinafter.

[0080] Please refer to Figure 3 , Figure 3 which is a schematic flowchart provided for the third embodiment of the feature extraction method of the present application.

[0081] In this embodiment, the first visual detection result includes the first Blob analysis result and the normalized average edge gradient magnitude corresponding to the first Blob analysis result, and the second visual detection result includes the second Blob analysis result and the confidence of the second Blob analysis result; Step S400 performs weighted fusion on the first visual detection result and the second visual detection result to obtain the third visual detection result of the target product, which may include steps S421 to S423: Step S421 calculates the confidence of the first Blob analysis result according to the normalized average edge gradient magnitude corresponding to the first Blob analysis result; In this embodiment, when performing Blob analysis on the target product, it is necessary to first perform connected component labeling on the image to be detected of the target product to obtain each connected component in the image to be detected, and then determine the contour information of each connected component by contour extraction, that is, determine the edge pixels of each connected component. Furthermore, the normalized average edge gradient magnitude corresponding to the first Blob analysis result can be calculated through the calculation formula of the normalized average edge gradient magnitude in the second embodiment.

[0082] Correspondingly, when performing Blob analysis, the higher the gradient magnitude of the edge pixels of each marked connected component, the more accurate the Blob analysis result. Therefore, the confidence of the first Blob analysis result can be reflected by the normalized average edge gradient magnitude corresponding to the first Blob analysis result.

[0083] The specific confidence calculation method can refer to the second embodiment, and this embodiment does not make repeated limitations.

[0084] Step S422 determines the third weight corresponding to the first Blob analysis result and the fourth weight corresponding to the second Blob analysis result according to the confidence of the first Blob analysis result and the confidence of the second Blob analysis result; It should be noted that the third weight refers to the weight ratio of the first Blob analysis result during weighted fusion, and the fourth weight refers to the weight ratio of the second Blob analysis result during weighted fusion.

[0085] In this embodiment, while the Blob analysis deep learning model outputs the predicted second Blob analysis result, it also outputs the confidence level of the second Blob analysis result.

[0086] The specific confidence level weighting rule can refer to the above-mentioned second embodiment, and this embodiment will not elaborate on it too much.

[0087] Step S423: Perform weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain the third Blob analysis result.

[0088] Similar to the second embodiment, in this embodiment, the weight ratios of the first Blob analysis result and the second Blob analysis result during weighted fusion can also be dynamically adjusted through the confidence levels of each Blob analysis result, so as to achieve complementary decision optimization, retaining both the accuracy of traditional visual detection in regular features and utilizing the strong representation ability of deep learning for non-linear features to make up for the detection deviation of traditional visual detection in complex scenarios such as blurred edges and noise interference, and further improving the accuracy of Blob analysis in variable industrial scenarios such as light fluctuations, target adhesion, and data scarcity.

[0089] It is not difficult to understand that in this embodiment, the third weight and the fourth weight can also be adjusted through the shooting environment information.

[0090] In a feasible implementation manner, the first visual detection result includes the first Blob analysis result and the change rate of the morphological closing area corresponding to the first Blob analysis result, and the second visual detection result includes the second Blob analysis result and the confidence level of the second Blob analysis result; Step S400 performs weighted fusion on the first visual detection result and the second visual detection result to obtain the third visual detection result of the target product, which may include steps S431 to S433: Step S431: Calculate the confidence level of the first Blob analysis result according to the change rate of the morphological closing area corresponding to the first Blob analysis result; In this implementation manner, the change rate of the morphological closing area refers to the change rate of the area of the connected domain after performing morphological closing operation on the connected domain extracted by the connected component labeling algorithm.

[0091] This embodiment notes that when performing blob analysis, the smaller the change rate of the area of the connected component before and after the morphological closing operation on the connected component, the more complete the structure of the connected component, and the more reliable the first blob analysis result obtained based on this. Therefore, after determining the connected components in the image to be detected through the connected component labeling algorithm, the area of the connected component before and after the closing operation is determined through the morphological closing operation, so as to calculate the change rate of the morphological closing area corresponding to the first blob analysis result, and then the confidence of the first blob analysis result is reflected by it.

[0092] The specific confidence calculation method is similar to that of the above embodiment. It is possible to pre-calibrate the proportion that the error between the first blob analysis result and the actual connected component attribute information is less than the preset value under different change rates of the morphological closing area, so as to use this proportion as the confidence mapped by the change rate of the morphological closing area and construct the corresponding mapping relationship, so as to directly determine the confidence of the first blob analysis result according to this mapping relationship in actual applications. It is also possible to pre-design a calculation formula between the change rate of the morphological closing area and the confidence, so as to use this calculation formula to determine the confidence of the first blob analysis result.

[0093] Step S432: Determine the third weight corresponding to the first blob analysis result and the fourth weight corresponding to the second blob analysis result according to the confidence of the first blob analysis result and the confidence of the second blob analysis result; Step S433: Perform weighted fusion on the first blob analysis result and the second blob analysis result according to the third weight and the fourth weight to obtain the third blob analysis result.

[0094] This embodiment evaluates the confidence of the first blob analysis result from another dimension through the change rate of the morphological closing area, so as to realize the fusion of the blob analysis results with confidence weighting and obtain the third blob analysis result.

[0095] It is not difficult to understand that this embodiment can be combined with the above second embodiment, so as to comprehensively determine the confidence of the first blob analysis result through the change rate of the morphological closing area corresponding to the first blob analysis result and the normalized average edge gradient amplitude, and then realize a more accurate confidence weighting fusion, further improving the accuracy of the third blob analysis result.

[0096] In a feasible embodiment, the first visual detection result includes the first blob analysis result and the convex hull filling rate corresponding to the first blob analysis result, and the second visual detection result includes the second blob analysis result and the confidence of the second blob analysis result; Step S400 performs weighted fusion on the first visual detection result and the second visual detection result to obtain the third visual detection result of the target product, which may include steps S441 to S443: In step S441, according to the convex hull filling rate corresponding to the first Blob analysis result, the confidence level of the first Blob analysis result is calculated; As known to those skilled in the art, a convex hull refers to the smallest convex polygon that contains all pixel points of the target connected domain.

[0097] In this embodiment, each connected domain corresponds to a convex hull. The convex hull filling rate refers to the ratio of the area of the connected domain to the area of the convex hull corresponding to the connected domain. The convex hull filling rate corresponding to the first Blob analysis result refers to the total area of all connected domains involved in the first Blob analysis result, and the total area of the convex hulls corresponding to each connected domain.

[0098] Exemplarily, when the first Blob analysis result involves connected domains A, B, and C, the area of connected domain A is 10, the area of the corresponding convex hull a is 12, the area of connected domain B is 5, the area of the corresponding convex hull b is 6, the area of connected domain C is 1, and the area of the corresponding convex hull c is 2. Then the total area of connected domains A, B, and C is 10 + 5 + 1 = 16, and the total area of the convex hulls corresponding to each connected domain is 12 + 6 + 2 = 20. The convex hull filling rate corresponding to the first Blob analysis result is 16 / 20 = 80%.

[0099] This embodiment notes that when performing Blob analysis, the larger the convex hull filling rate corresponding to the first Blob analysis result, the more complete the connected domain structure, and the more reliable the first Blob analysis result based on this. Therefore, it is proposed that after determining the connected domains in the image to be detected through the connected domain labeling algorithm, the convex hulls of each connected domain can be determined, the convex hull filling rate corresponding to the first Blob analysis result can be calculated, and then the confidence level of the first Blob analysis result can be reflected by it.

[0100] The specific confidence level calculation method is similar to the above embodiment. Different ratios of the error between the first Blob analysis result and the actual connected domain attribute information being less than the preset value under different convex hull filling rates can be pre-calibrated, and this ratio can be used as the confidence level mapped by the convex hull filling rate, and the corresponding mapping relationship can be constructed to facilitate directly determining the confidence level of the first Blob analysis result according to this mapping relationship in practical applications. Or a calculation formula between the convex hull filling rate and the confidence level can be pre-designed, so as to apply this calculation formula to determine the confidence level of the first Blob analysis result.

[0101] Step S442: Determine the third weight corresponding to the first Blob analysis result and the fourth weight corresponding to the second Blob analysis result according to the confidence of the first Blob analysis result and the confidence of the second Blob analysis result. Step S443: Perform weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain the third Blob analysis result.

[0102] In this embodiment, the confidence of the first Blob analysis result is evaluated from another dimension through the convex hull filling rate, so as to realize the fusion of the confidence-weighted Blob analysis results and obtain the third Blob analysis result.

[0103] It is not difficult to understand that this embodiment can be combined with the above embodiments and implementation manners, so as to comprehensively determine the confidence of the first Blob analysis result through the morphological closing area change rate, the normalized average edge gradient amplitude, and the convex hull filling rate corresponding to the first Blob analysis result, and then realize a more accurate confidence-weighted fusion, further improving the accuracy of the third Blob analysis result.

[0104] In a feasible implementation manner, the first visual detection result includes the first Blob analysis result, as well as the normalized average edge gradient amplitude, the morphological closing area change rate, and the convex hull filling rate corresponding to the first Blob analysis result, and the second visual detection result includes the second Blob analysis result and the confidence of the second Blob analysis result. Step S400 performs weighted fusion on the first visual detection result and the second visual detection result to obtain the third visual detection result of the target product, which may include steps S451 to S453: Step S451: Calculate the confidence of the first Blob analysis result according to the normalized average edge gradient amplitude, the morphological closing area change rate, and the convex hull filling rate corresponding to the first Blob analysis result. Step S452: Determine the third weight corresponding to the first Blob analysis result and the fourth weight corresponding to the second Blob analysis result according to the confidence of the first Blob analysis result and the confidence of the second Blob analysis result. Step S453: Perform weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain the third Blob analysis result.

[0105] In this embodiment, in step S451, the confidence levels mapped respectively can be determined according to the mapping relationships between various parameters (i.e., the normalized average edge gradient magnitude, the change rate of morphological closing area, and the convex hull filling rate corresponding to the first Blob analysis result) and the confidence level, and then the confidence level of the final first Blob analysis result can be determined by taking the average or weighted average.

[0106] In addition, a mathematical model can be established between the normalized average edge gradient magnitude, the change rate of morphological closing area, and the convex hull filling rate corresponding to the first Blob analysis result, and the confidence level of the first Blob analysis result, so that the confidence level of the first Blob analysis result can be calculated through this mathematical model.

[0107] By introducing a confidence level evaluation mechanism based on multiple characteristic indexes (the normalized average edge gradient magnitude, the change rate of morphological closing area, and the convex hull filling rate), this embodiment provides a more accurate confidence level evaluation, and combines a dynamic weight adjustment strategy to ensure that the results with high confidence levels are preferentially adopted during the fusion process, thereby effectively coping with the challenges in various complex scenarios, realizing the intelligent fusion of the output results of traditional image processing algorithms and deep learning models, giving full play to the high efficiency of traditional image processing algorithms under specific conditions, and making full use of the powerful representation ability of deep learning models, realizing the complementary advantages of both, and greatly improving the overall efficiency of the visual detection system.

[0108] Based on the above embodiments, a feature extraction method according to the fourth embodiment of the present application is proposed.

[0109] In the fourth embodiment of the present application, the same or similar content as that in the above embodiments can be referred to the above introduction and will not be repeated hereinafter.

[0110] In this embodiment, the third visual detection result includes a third dimension measurement result and a third Blob analysis result; After step S400 performs weighted fusion on the first visual detection result and the second visual detection result to obtain the third visual detection result of the target product, the visual detection method may further include step S500: Step S500, according to the third dimension measurement result and the third Blob analysis result, perform defect type detection on the target product to obtain the defect type detection result of the target product.

[0111] Those skilled in the art know that defect type detection refers to visual detection for detecting the defect types of products.

[0112] After obtaining the fused third visual detection result in this embodiment, based on the third dimension measurement result and the third Blob analysis result with higher accuracy in the third visual detection result, more accurate defect type detection is performed on the target product, so as to output a more accurate defect type detection result, improving the intelligent level and quality control ability of the industrial vision detection system.

[0113] To facilitate the understanding of the technical concept of the above embodiments of the present application, a specific embodiment is provided: In this specific embodiment, the industrial vision detection system is integrated with a hybrid detection engine. The hybrid detection engine includes a traditional vision module, a deep learning module, and a result fusion module. Among them, the traditional vision module is used for traditional vision detection, the deep learning module is used for vision detection based on deep learning, and the result fusion module is used to perform confidence weighted fusion on the results of the two vision detections. The traditional vision module is integrated with a dimension measurement function implemented by sub-pixel edge detection, a Blob analysis function implemented by an improved algorithm for connected component labeling, and a QR (Quick Response) decoding function implemented by a morphological optimization positioning algorithm. The deep learning module uses a lightweight network architecture to implement dimension measurement function, Blob analysis function, defect type detection function, and character recognition function.

[0114] In this specific embodiment, after the industrial vision detection system detects a visual detection request based on the target product, it acquires the image to be detected of the target product, and then through the traditional vision module in the hybrid detection engine, based on a preset image processing algorithm, feature extraction is performed on the image to be detected to obtain the target features of the image to be detected, and visual detection is performed on the target product based on the target features to obtain the first visual detection result of the target product. At the same time, through the deep learning module in the hybrid detection engine, the image to be detected is input into a pre-trained visual detection deep learning model to obtain the second visual detection result of the target product output by the visual detection deep learning model. Finally, through the result fusion module in the hybrid detection engine, weighted fusion is performed on the first visual detection result and the second visual detection result to obtain the third visual detection result of the target product.

[0115] The industrial vision inspection system adopts a communication protocol configuration method that supports hot plugging, and is integrated with a protocol template library, a field mapping configurator, and a heartbeat detection module. The protocol template library contains protocol templates for multiple communication protocols, including TCP / IP (Transmission Control Protocol / Internet Protocol), such as the protocol template for communication, the protocol template for serial communication, and the protocol template for Modbus communication. The field mapping configurator allows users to define the correspondence between the data packet structure and the device register address through an XML (eXtensible Markup Language) configuration file. The heartbeat detection module supports a configurable keep-alive mechanism.

[0116] The industrial vision inspection system also includes a no-code debugging mechanism, which is equipped with a parameter visualization editor. Users can input the communication protocol or the parameters required for vision inspection through the visualization interface, and an XML configuration file will be automatically generated. When the system configures the communication protocol or conducts vision inspection, it can directly load the corresponding XML configuration file, enabling flexible debugging and configuration of parameters without the operator having code editing capabilities or modifying the underlying code.

[0117] The industrial vision inspection system also supports users to dynamically adjust the region of interest (ROI) of the image to be inspected through the visualization interface, so as to only perform vision inspection on the ROI set by the user, and superimpose the final vision inspection result on the ROI for the user to preview.

[0118] The industrial vision inspection system is also integrated with a protocol simulator. When a new device is connected, the protocol simulator can pre-simulate whether the configured communication protocol parameters are correct, and the protocol simulator supports abnormal injection testing.

[0119] Such as Figure 4 , in this specific embodiment, when a user needs to perform industrial vision inspection on a certain product, they only need to complete camera configuration, model configuration, project management, and parameter setting in sequence to achieve the complete business process setting for industrial vision inspection of the product. Then, the industrial vision system will automatically execute the vision inspection task on the product according to this business process setting and output the corresponding vision inspection result.

[0120] It should be noted that the above embodiments / implementation manners are only used to assist in understanding the present application, and do not constitute a limitation on the vision inspection method of the present application. Based on this technical concept, more forms of simple transformations are within the protection scope of the present application.

[0121] In addition, please refer to Figure 5 , Figure 5It is a schematic diagram of the device structure of the hardware operating environment of the electronic device involved in the vision detection method in the embodiments of the present application.

[0122] The present application also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps of the vision detection method in the above embodiments.

[0123] Next, refer to Figure 5 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic device may include, but is not limited to, terminal devices such as mobile phones, laptop computers, tablet computers, desktop computers, etc., or any electronic device capable of implementing the above functions. Figure 5 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0124] As Figure 5 shown, the electronic device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in the read-only memory 1002 or a program loaded from the storage device 1003 into the random access memory 1004. In the random access memory 1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. The input / output interface 1006 is also connected to the bus. Generally, the following systems may be connected to the input / output interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. Instead, more or fewer systems may be implemented or had.

[0125] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by a processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are executed.

[0126] The electronic device provided by the present application adopts the visual detection method in the above-mentioned embodiments, and can solve the technical problem of relatively low accuracy of visual detection results in industrial visual detection. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as those of the visual detection method provided by the above-mentioned embodiments, and other technical features in the electronic device are the same as those disclosed in the method of the above-mentioned embodiments, and will not be elaborated here.

[0127] It should be understood that the various parts disclosed in the present application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0128] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the above-mentioned claims.

[0129] In addition, the present application also provides a computer-readable storage medium, which has computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the steps of the visual detection method in the above-mentioned embodiments.

[0130] The computer-readable storage medium provided by the present application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memories, read-only memories (erasable programmable read-only memories, optical fibers, portable compact disk read-only memories, optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0131] The above computer-readable storage medium can be included in an electronic device or the electronic device; or it can exist independently without being assembled into the electronic device or the electronic device.

[0132] The above computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device is caused to: after detecting a visual detection request based on a target product, obtain a to-be-detected image of the target product; perform feature extraction on the to-be-detected image based on a preset image processing algorithm to obtain target features of the to-be-detected image, and perform visual detection on the target product based on the target features to obtain a first visual detection result of the target product, where the image processing algorithm includes an edge detection algorithm and / or a connected component labeling algorithm, and the first visual detection result includes a first dimension measurement result and / or a first Blob analysis result; input the to-be-detected image into a pre-trained visual detection deep learning model to obtain a second visual detection result of the target product output by the visual detection deep learning model, where the visual detection deep learning model includes a dimension measurement deep learning model and / or a Blob analysis deep learning model, and the second visual detection result includes a second dimension measurement result and / or a second Blob analysis result; perform weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product, where the third visual detection result includes a third dimension measurement result and / or a third Blob analysis result.

[0133] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0135] The modules involved in the embodiments described in this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the unit itself in some cases.

[0136] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., computer programs) for performing the steps of the above-mentioned visual detection method, and can solve the technical problem of relatively low accuracy of visual detection results in industrial visual detection. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the visual detection method provided in the above embodiments, and will not be elaborated here.

[0137] In addition, an embodiment of the present application further provides a computer program product, including a computer program, which when executed by a processor implements the steps of the visual detection method in the above embodiment.

[0138] The computer program product provided by the present application can solve the technical problem of low accuracy of visual detection results in industrial visual detection. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as those of the visual detection method provided by the above embodiment, and will not be elaborated here.

[0139] The above are only partial embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A visual detection method, characterized in that, The method includes: After detecting a visual detection request based on a target product, obtaining a to-be-detected image of the target product; Performing feature extraction on the to-be-detected image based on a preset image processing algorithm to obtain target features of the to-be-detected image, and performing visual detection on the target product based on the target features to obtain a first visual detection result of the target product, wherein the image processing algorithm includes an edge detection algorithm and / or a connected component labeling algorithm, and the first visual detection result includes a first dimension measurement result and / or a first Blob analysis result; Inputting the to-be-detected image into a pre-trained visual detection deep learning model to obtain a second visual detection result of the target product output by the visual detection deep learning model, wherein the visual detection deep learning model includes a dimension measurement deep learning model and / or a Blob analysis deep learning model, and the second visual detection result includes a second dimension measurement result and / or a second Blob analysis result; Performing weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product, wherein the third visual detection result includes a third dimension measurement result and / or a third Blob analysis result.

2. The visual inspection method according to claim 1, wherein The first visual detection result includes a first dimension measurement result and a normalized average edge gradient magnitude corresponding to the first dimension measurement result, and the second visual detection result includes a second dimension measurement result and a confidence level of the second dimension measurement result; The step of performing weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product includes: Calculating a confidence level of the first dimension measurement result according to the normalized average edge gradient magnitude corresponding to the first dimension measurement result; Determining a first weight corresponding to the first dimension measurement result and a second weight corresponding to the second dimension measurement result according to the confidence level of the first dimension measurement result and the confidence level of the second dimension measurement result; Performing weighted fusion on the first dimension measurement result and the second dimension measurement result according to the first weight and the second weight to obtain a third dimension measurement result.

3. The visual detection method according to claim 2, wherein The step of performing weighted fusion on the first dimension measurement result and the second dimension measurement result according to the first weight and the second weight to obtain a third dimension measurement result includes: Obtaining shooting environment information of the to-be-detected image, and adjusting the first weight and the second weight according to the shooting environment information; Performing weighted fusion on the first dimension measurement result and the second dimension measurement result according to the adjusted first weight and the adjusted second weight to obtain a third dimension measurement result.

4. The visual detection method according to claim 1, characterized in that The first visual detection result includes a first Blob analysis result and a normalized average edge gradient magnitude corresponding to the first Blob analysis result, and the second visual detection result includes a second Blob analysis result and a confidence level of the second Blob analysis result; The step of performing weighted fusion on the first visual detection result and the second visual detection result to obtain the third visual detection result of the target product includes: Calculating the confidence level of the first Blob analysis result according to the normalized average edge gradient amplitude corresponding to the first Blob analysis result; Determining the third weight corresponding to the first Blob analysis result and the fourth weight corresponding to the second Blob analysis result according to the confidence level of the first Blob analysis result and the confidence level of the second Blob analysis result; Performing weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain a third Blob analysis result.

5. The visual detection method according to claim 1, wherein The first visual detection result includes a first Blob analysis result and the change rate of the morphological closing area corresponding to the first Blob analysis result, and the second visual detection result includes a second Blob analysis result and the confidence level of the second Blob analysis result; The step of performing weighted fusion on the first visual detection result and the second visual detection result to obtain the third visual detection result of the target product includes: Calculating the confidence level of the first Blob analysis result according to the change rate of the morphological closing area corresponding to the first Blob analysis result; Determining the third weight corresponding to the first Blob analysis result and the fourth weight corresponding to the second Blob analysis result according to the confidence level of the first Blob analysis result and the confidence level of the second Blob analysis result; Performing weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain a third Blob analysis result.

6. The visual detection method according to claim 1, characterized in that, The first visual detection result includes a first Blob analysis result and the convex hull filling rate corresponding to the first Blob analysis result, and the second visual detection result includes a second Blob analysis result and the confidence level of the second Blob analysis result; The step of performing weighted fusion on the first visual detection result and the second visual detection result to obtain the third visual detection result of the target product includes: Calculating the confidence level of the first Blob analysis result according to the convex hull filling rate corresponding to the first Blob analysis result; Determining the third weight corresponding to the first Blob analysis result and the fourth weight corresponding to the second Blob analysis result according to the confidence level of the first Blob analysis result and the confidence level of the second Blob analysis result; Performing weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain a third Blob analysis result.

7. The visual inspection method according to claim 1, characterized in that The first visual detection result includes a first Blob analysis result, as well as the normalized average edge gradient magnitude, the morphological closing area change rate, and the convex hull filling rate corresponding to the first Blob analysis result. The second visual detection result includes a second Blob analysis result and the confidence level of the second Blob analysis result; The step of performing weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product includes: Calculating the confidence level of the first Blob analysis result based on the normalized average edge gradient magnitude, the morphological closing area change rate, and the convex hull filling rate corresponding to the first Blob analysis result; Determining a third weight corresponding to the first Blob analysis result and a fourth weight corresponding to the second Blob analysis result based on the confidence level of the first Blob analysis result and the confidence level of the second Blob analysis result; Performing weighted fusion on the first Blob analysis result and the second Blob analysis result according to the third weight and the fourth weight to obtain a third Blob analysis result.

8. The visual inspection method according to any one of claims 1 to 7, characterized in that The third visual detection result includes a third dimension measurement result and a third Blob analysis result; After the step of performing weighted fusion on the first visual detection result and the second visual detection result to obtain a third visual detection result of the target product, the method further includes: Performing defect type detection on the target product according to the third dimension measurement result and the third Blob analysis result to obtain a defect type detection result of the target product.

9. An electronic device, characterized in that, including: A memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps of the visual detection method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps of the visual detection method according to any one of claims 1 to 8 are implemented.

Images