A visual inspection method and related system for complex assembly structural parts

Abstract

The present application provides a visual inspection method and a related system for complex assembly structural parts, the method comprising: when a target detection type is a preset detection type, determining a target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm; performing multi-scale decomposition on a target area according to the target multi-scale decomposition algorithm and the first algorithm control parameter to obtain a low-frequency component part and n high-frequency component parts; determining a second feature extraction algorithm corresponding to the preset detection type and a second algorithm control parameter of the second feature extraction algorithm; performing feature extraction on m high-frequency component parts according to the second feature extraction algorithm, the second algorithm control parameter and m feature type labels to obtain m second feature sets; and determining a target detection result according to the m second feature sets and a target detection verification standard.

Description

Technical Field

[0001] The present application relates to the fields of intelligent manufacturing technology and robotics technology, and specifically to a visual inspection method and related system for complex assembly structural parts. Background Art

[0002] With the rapid development and progress of science and technology, aerospace technology has also developed rapidly. In the field of aerospace, the requirements for the final appearance inspection of complex assembly structural parts are very high, including: appearance size inspection, weight inspection, center of mass inspection, paint surface integrity inspection, missing inspection, assembly gap inspection, spray mark conformity inspection, bump and damage inspection, product number correctness inspection, redundant object inspection, etc. Usually, manual inspection combined with experience is required, resulting in low inspection efficiency. Therefore, the problem of how to improve the inspection efficiency of complex assembly structural parts needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present application provide a visual inspection method and a related system for complex assembly structural parts, which can improve the inspection efficiency of complex assembly structural parts.

[0004] In a first aspect, an embodiment of the present application provides a visual inspection method for complex assembly structural parts, which is applied to an industrial robot, wherein the industrial robot includes a camera, and the method includes:

[0005] Acquire a first image of a target device through the camera;

[0006] Performing feature extraction on the first image according to a first feature extraction algorithm to obtain a first feature set;

[0007] When there are features satisfying a preset condition in the first feature set, a second feature set satisfying the preset condition is screened out from the first feature set, an area corresponding to the second feature set is determined to obtain a target area, and an assembly structural part corresponding to the target area is identified as a complex assembly structural part;

[0008] Determine a target assembly structural component name corresponding to the second feature set;

[0009] Obtaining a target detection type and a target detection verification standard corresponding to the target assembly structural part name;

[0010] When the target detection type is a preset detection type, determining a target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm;

[0011] Perform multi-scale decomposition on the target area according to the target multi-scale decomposition algorithm and the first algorithm control parameter to obtain a low-frequency component part and n high-frequency component parts, each high-frequency component part corresponds to a high-frequency component label, and n is a positive integer;

[0012] Determine a second feature extraction algorithm corresponding to the preset detection type, a second algorithm control parameter of the second feature extraction algorithm, and a target extraction range, wherein the target extraction range includes m high-frequency component labels and m feature type labels, each high-frequency component label corresponds to a feature type label, and m is a positive integer less than or equal to n;

[0013] Obtaining corresponding m high-frequency component parts from the n high-frequency component parts according to the m high-frequency component labels;

[0014] Perform feature extraction on the m high-frequency component parts according to the second feature extraction algorithm, the second algorithm control parameters and the m feature type labels to obtain m second feature sets;

[0015] The target detection result is determined according to the m second feature sets and the target detection verification standard.

[0016] In a second aspect, an embodiment of the present application provides a visual inspection system for complex assembly structural parts, which is applied to an industrial robot, wherein the industrial robot includes a camera, and the system includes: an acquisition unit, an extraction unit, a determination unit, and a decomposition unit, wherein:

[0017] The acquisition unit is used to acquire a first image of a target device through the camera;

[0018] The extraction unit is used to extract features from the first image according to a first feature extraction algorithm to obtain a first feature set;

[0019] The determining unit is used to, when there are features satisfying the preset conditions in the first feature set, filter out a second feature set satisfying the preset conditions from the first feature set, determine an area corresponding to the second feature set, obtain a target area, identify the assembly structure corresponding to the target area as a complex assembly structure; and determine a name of the target assembly structure corresponding to the second feature set;

[0020] The acquisition unit is further used to acquire the target detection type and target detection verification standard corresponding to the name of the target assembly structure part;

[0021] The determining unit is further configured to determine, when the target detection type is a preset detection type, a target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm;

[0022] The decomposition unit is used to perform multi-scale decomposition on the target area according to the target multi-scale decomposition algorithm and the first algorithm control parameter to obtain a low-frequency component part and n high-frequency component parts, each high-frequency component part corresponds to a high-frequency component label, and n is a positive integer;

[0023] The determination unit is further used to determine a second feature extraction algorithm corresponding to the preset detection type, a second algorithm control parameter of the second feature extraction algorithm, and a target extraction range, wherein the target extraction range includes m high-frequency component labels and m feature type labels, each high-frequency component label corresponds to a feature type label, and m is a positive integer less than or equal to n;

[0024] The acquisition unit is further used to acquire corresponding m high-frequency component parts from the n high-frequency component parts according to the m high-frequency component labels;

[0025] The extraction unit is further used to perform feature extraction on the m high-frequency component parts according to the second feature extraction algorithm, the second algorithm control parameters and the m feature type labels to obtain m second feature sets;

[0026] The determination unit is further used to determine the target detection result according to the m second feature sets and the target detection verification standard.

[0027] In a third aspect, an embodiment of the present application provides an industrial robot, comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps in the first aspect of the embodiment of the present application.

[0028] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in the first aspect of the embodiment of the present application.

[0029] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiment of the present application. The computer program product may be a software installation package.

[0030] The implementation of the embodiments of the present application has the following beneficial effects:

[0031] It can be seen that the visual inspection method and related system for complex assembly structural parts described in the embodiments of the present application are applied to an industrial robot. The industrial robot includes a camera, which obtains a first image of a target device through the camera, extracts features from the first image according to a first feature extraction algorithm, and obtains a first feature set. When there are features that meet the preset conditions in the first feature set, a second feature set that meets the preset conditions is screened out from the first feature set, and an area corresponding to the second feature set is determined to obtain a target area. The assembly structural part corresponding to the target area is identified as a complex assembly structural part, and the name of the target assembly structural part corresponding to the second feature set is determined. The target detection type and target detection verification standard corresponding to the name of the target assembly structural part are obtained. When the target detection type is a preset detection type, a target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm are determined. The target area is multi-scale decomposed according to the target multi-scale decomposition algorithm and the first algorithm control parameter to obtain a low-frequency component part and n high-frequency component parts, each high-frequency component part corresponds to a high-frequency component label, and n is a positive integer. The second feature extraction corresponding to the preset detection type is determined. The invention discloses an algorithm, a second algorithm control parameter of a second feature extraction algorithm and a target extraction range, wherein the target extraction range includes m high-frequency component labels and m feature type labels, each high-frequency component label corresponds to a feature type label, and m is a positive integer less than or equal to n. According to the m high-frequency component labels, m high-frequency component parts in n high-frequency component parts are feature extracted according to the second feature extraction algorithm, the second algorithm control parameter and the m feature type labels to obtain m second feature sets. According to the m second feature sets and the target detection verification standard, the target detection result is determined. First, the camera can be used to identify complex assembly structural parts. Second, based on the corresponding detection type of the complex assembly structural parts, the corresponding multi-scale decomposition algorithm and the algorithm control parameter can be determined. Then, the depth detail image (high-frequency component part) corresponding to the detection type corresponding to the complex assembly structural parts is extracted. Then, based on the corresponding detection type of the complex assembly structural parts, the required features in the depth detail image are determined. These features are matched with the detection verification standard corresponding to the detection type of the complex assembly structural parts to obtain the detection result. In this way, the detection efficiency of the complex assembly structural parts can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1It is a flow chart of a visual inspection method for complex assembly structural parts provided in an embodiment of the present application;

[0034] Figure 2 is a schematic diagram of the structure of an industrial robot provided in an embodiment of the present application;

[0035] Figure 3 It is a block diagram of the functional units of a visual inspection system for complex assembly structural parts provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but in a possible example also includes steps or units that are not listed, or in a possible example also includes other steps or units inherent to these processes, methods, products or devices.

[0037] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] In the embodiments of the present application, the industrial robot can be understood as an industrial robot with visual inspection function, an automobile processing robot, an intelligent machine tool, a humanoid robot, etc., which is not limited here.

[0040] See also Figure 1 , Figure 1 The flowchart of a visual inspection method for complex assembly structural parts provided by an embodiment of the present application is shown in the figure. As shown in the figure, the method is applied to an industrial robot. The industrial robot includes a camera. The visual inspection method for complex assembly structural parts includes:

[0041] 101. Acquire a first image of a target device through the camera.

[0042] In the embodiment of the present application, the camera may include at least one of the following: an infrared camera, a visible light camera, etc., which are not limited here. The camera may also include one or more cameras.

[0043] The target device may include at least one of the following: aviation equipment, aerospace equipment, navigation equipment, amphibious equipment, automobiles, etc., which are not limited here.

[0044] In a specific implementation, the first image of the target device may be acquired through a camera.

[0045] 102. Perform feature extraction on the first image according to a first feature extraction algorithm to obtain a first feature set.

[0046] The first feature extraction algorithm may be preset or defaulted by the system. The first feature extraction algorithm may include at least one of the following: feature extraction algorithm, contour extraction algorithm, feature vector extraction algorithm, color extraction algorithm, coordinate extraction algorithm, etc., which are not limited here.

[0047] In a specific implementation, feature extraction may be performed on the first image according to a first feature extraction algorithm to obtain a first feature set. The first feature set may include one or more types of features, or the first feature set may include one or more features.

[0048] 103. When there are features satisfying a preset condition in the first feature set, a second feature set satisfying the preset condition is screened out from the first feature set, an area corresponding to the second feature set is determined, a target area is obtained, and the assembly structural part corresponding to the target area is identified as a complex assembly structural part.

[0049] The preset conditions may be preset or set by system default. Different complex assembly structural parts may correspond to different features, and thus corresponding preset conditions may be set based on the features of different complex assembly structural parts.

[0050] The target area may be one or more areas, and each area may correspond to a complex assembly structure.

[0051] In a specific implementation, when there are features that meet the preset conditions in the first feature set, it means that there are complex assembly structural parts. Then, a second feature set that meets the preset conditions is selected from the first feature set, and the area corresponding to the second feature set is determined to obtain a target area, and the assembly structural parts corresponding to the target area are identified as complex assembly structural parts. In this way, complex assembly structural parts can be identified based on the features possessed by the complex assembly structural parts.

[0052] 104. Determine a name of a target assembly structural component corresponding to the second feature set.

[0053] In the embodiment of the present application, since different complex assembly structural parts have different features, the target assembly structural part name of the corresponding complex assembly structural part can be identified based on the second feature set.

[0054] 105. Obtain a target detection type and a target detection verification standard corresponding to the target assembly structural component name.

[0055] Among them, the detection types may include at least one of the following: appearance size detection, weight detection, center of mass detection, paint integrity detection, missing assembly detection, assembly gap detection, spray mark compliance detection, bump and damage detection, product number correctness detection, excess material detection, etc., without limitation here.

[0056] In a specific implementation, different complex assembly structural parts require different contents to be inspected, i.e., different inspection types. For example, a mapping relationship between preset assembly structural part names and inspection types can be pre-stored, and then the target inspection type corresponding to the target assembly structural part name can be determined based on the mapping relationship. In addition, since different inspection types correspond to different inspection verification standards, the inspection verification standard can be related not only to the inspection type, but also to at least one of the following factors: environmental factors, the material of the target assembly structural part, the function of the target assembly structural part, the purpose of the target assembly structural part, etc., which are not limited here. Environmental factors can include at least one of the following: ambient temperature, ambient humidity, atmospheric pressure, ambient light brightness, wind speed, weather, magnetic field interference intensity, etc., which are not limited here.

[0057] 106. When the target detection type is a preset detection type, determine a target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm.

[0058] Among them, the preset detection type can be pre-set or system default. Different preset detection types correspond to different detection requirements. In specific implementation, for the field of high-precision detection, the requirements are different, and the depth of the feature is different. The depth of the feature can be reflected in the high-frequency component of the image. The high-frequency component parts of different depths have different characteristics. Often these deep-level features better reflect the needs of high-precision detection. Combined with the characteristics of the high-frequency component parts of the depth, it can meet the detection needs of complex assembly structural parts to a certain extent.

[0059] The target multi-scale decomposition algorithm may include at least one of the following: contourlet transform algorithm, non-subsampled contourlet transform algorithm, wavelet transform algorithm, ridgelet transform algorithm, shearlet transform algorithm, etc., which are not limited here. The first algorithm control parameter is used to control the multi-scale decomposition effect of the target multi-scale decomposition algorithm. For example, the first algorithm control parameter may include at least one of the following: the number of high-frequency component parts, the scale of the first algorithm control parameter, etc., which are not limited here.

[0060] In a specific implementation, when the target detection type is a preset detection type, it means that the detection is performed in combination with the characteristics of the high-frequency component of the depth. Specifically, the mapping relationship between the preset detection type and the multi-scale decomposition algorithm can be pre-stored, and then the target multi-scale decomposition algorithm corresponding to the target detection type can be determined based on the mapping relationship. The algorithm control parameters corresponding to the preset detection type and the target multi-scale decomposition algorithm can also be pre-stored, and then the first algorithm control parameters corresponding to the target multi-scale decomposition algorithm can be determined based on the mapping relationship. In this way, the multi-scale decomposition algorithm corresponding to the detection type and the corresponding algorithm control parameters can be obtained, which can ensure the detection accuracy of complex assembly structural parts.

[0061] Optionally, the above step 106, determining the target multi-scale decomposition algorithm corresponding to the preset detection type and the first algorithm control parameter corresponding to the target multi-scale decomposition algorithm, may include the following steps:

[0062] Determine a multiscale decomposition algorithm set corresponding to the preset detection type, wherein the multiscale decomposition algorithm set includes a plurality of multiscale decomposition algorithms; each multiscale decomposition algorithm corresponds to a reference image quality evaluation value;

[0063] Performing image quality evaluation on the target area to obtain a first image quality evaluation value;

[0064] Determine a target reference image quality evaluation value having the smallest difference with the first image quality evaluation value among the reference image quality evaluation values ​​corresponding to the multiple multi-scale decomposition algorithms;

[0065] Determining the target multi-scale decomposition algorithm according to the target reference image quality evaluation value;

[0066] Acquire a first initial algorithm control parameter of the target multi-scale decomposition algorithm;

[0067] Determining a first deviation between the target reference image quality evaluation value and the first image quality evaluation value;

[0068] determining a first optimization parameter corresponding to the first deviation;

[0069] The first initial algorithm control parameter is optimized according to the first optimization parameter to obtain the first algorithm control parameter.

[0070] The multiscale decomposition algorithm set includes multiple multiscale decomposition algorithms. Each multiscale decomposition algorithm corresponds to a reference image quality evaluation value. Different reference image quality evaluation values ​​are adapted to different multiscale decomposition algorithms. In this way, based on the detection type, the appropriate multiscale decomposition algorithm is adapted, so that the subsequent detection is more in line with the actual situation.

[0071] In a specific implementation, the mapping relationship between the detection type and the multi-scale decomposition algorithm set can be pre-stored, and then the multi-scale decomposition algorithm set corresponding to the preset detection type can be determined based on the mapping relationship. The mapping relationship between the preset detection type and the image quality evaluation index can also be pre-stored, and then the target image quality evaluation index corresponding to the preset detection type can be determined based on the mapping relationship.

[0072] Among them, the image quality evaluation index may include at least one of the following: clarity, average gradient, information entropy, edge preservation, average grayscale, signal-to-noise ratio, etc., which are not limited here. The target image quality evaluation index may include one or more image quality evaluation indicators, and then, the image quality of the target area may be evaluated to obtain a first image quality evaluation value, so that the appropriate image quality evaluation index can be adapted based on the detection type, and then, the image quality evaluation standard is unified, which is more conducive to adapting the appropriate multi-scale decomposition algorithm based on different image qualities.

[0073] Furthermore, a target reference image quality evaluation value having the smallest difference with the first image quality evaluation value among the reference image quality evaluation values ​​corresponding to multiple multi-scale decomposition algorithms can be determined, and then a reference image quality evaluation value closest to the first image quality evaluation value can be selected.

[0074] In a specific implementation, the target multi-scale decomposition algorithm can be determined according to the target reference image quality evaluation value, that is, the multi-scale decomposition algorithm corresponding to the target reference image quality evaluation value can be used as the target multi-scale decomposition algorithm. In addition, the first initial algorithm control parameter of the target multi-scale decomposition algorithm can also be obtained, and the first initial algorithm control parameter is used to control the multi-scale decomposition effect. For example, the first initial algorithm control parameter can include at least one of the following parameters: the number of high-frequency components, the scale of high-frequency components, the resolution of high-frequency components, the multi-scale decomposition speed, etc., which are not limited here.

[0075] Furthermore, the first deviation between the target reference image quality evaluation value and the first image quality evaluation value can be determined, that is, the first deviation = (target reference image quality evaluation value-first image quality evaluation value) / first image quality evaluation value. The first deviation not only reflects the size of the deviation, but also reflects the direction of the deviation. Since the target multi-scale decomposition algorithm is most suitable for the target reference image quality evaluation value, based on the first deviation, the optimized algorithm control parameters can be more suitable for the image quality at this time, thereby helping to improve the accuracy of subsequent detection.

[0076] Specifically, a mapping relationship between a preset deviation and an optimization parameter can be pre-stored, and the value range of the optimization parameter can be -0.1~0.1. Then, the first optimization parameter corresponding to the first deviation can be determined based on the mapping relationship, and then the first initial algorithm control parameter is optimized according to the first optimization parameter to obtain the first algorithm control parameter, that is, the first algorithm control parameter = (1+first optimization parameter) * first initial algorithm control parameter. In this way, since the target multi-scale decomposition algorithm is most suitable for the target reference image quality evaluation value, the optimized algorithm control parameter based on the first deviation can be more suitable for the image quality at this time, thereby helping to improve the accuracy of subsequent detection.

[0077] 107. Perform multi-scale decomposition on the target area according to the target multi-scale decomposition algorithm and the first algorithm control parameter to obtain a low-frequency component part and n high-frequency component parts, each high-frequency component part corresponds to a high-frequency component label, and n is a positive integer.

[0078] The high-frequency component label is used to indicate the high-frequency component number, that is, the number of the high-frequency component part.

[0079] In an embodiment of the present application, the target area can be multi-scale decomposed according to the target multi-scale decomposition algorithm and the first algorithm control parameters to obtain a low-frequency component part and n high-frequency component parts, each high-frequency component part corresponds to a high-frequency component label, and n is a positive integer. In this way, the required high-frequency components can be separated, and then the characteristics of the high-frequency component parts can be used to detect complex assembly structural parts of the corresponding dimensions of the preset detection type.

[0080] 108. Determine a second feature extraction algorithm corresponding to the preset detection type, a second algorithm control parameter of the second feature extraction algorithm, and a target extraction range, wherein the target extraction range includes m high-frequency component labels and m feature type labels, each high-frequency component label corresponds to a feature type label, and m is a positive integer less than or equal to n.

[0081] The second feature extraction algorithm may include at least one of the following: feature extraction algorithm, contour extraction algorithm, feature vector extraction algorithm, color extraction algorithm, coordinate extraction algorithm, etc., which are not limited here. For example, the feature type label may include at least one of the following: feature point, feature vector, feature contour, etc., which are not limited here.

[0082] The second algorithm control parameter of the second feature extraction algorithm is used to control the feature extraction effect, and different feature extraction algorithms correspond to different algorithm control parameters.

[0083] Among them, the feature type label is used to indicate the feature type.

[0084] In a specific implementation, different detection types require different feature requirements. The mapping relationship between the preset detection type and the feature extraction algorithm can be pre-stored, and then the second feature extraction algorithm corresponding to the preset detection type can be determined based on the mapping relationship. The mapping relationship between the preset detection type and the algorithm control parameters of the second feature extraction algorithm can also be pre-stored, and then the second algorithm control parameters of the second feature extraction algorithm corresponding to the preset detection type can be determined based on the mapping relationship. The mapping relationship between the preset detection type and the extraction range can also be pre-stored, and then the target extraction range corresponding to the preset detection type can be determined based on the mapping relationship. The target extraction range includes m high-frequency component labels and m feature type labels, each high-frequency component label corresponds to a feature type label, and m is a positive integer less than or equal to n. In this way, the corresponding second feature extraction algorithm, the second algorithm control parameters of the second feature extraction algorithm and the target extraction range can be determined based on the specific detection requirements, so as to ensure the detection accuracy of complex assembly structural parts.

[0085] 109. Obtain corresponding m high-frequency component parts from the n high-frequency component parts according to the m high-frequency component labels.

[0086] In a specific implementation, corresponding m high-frequency component parts can be obtained from n high-frequency component parts according to m high-frequency component labels, thereby helping to select the high-frequency component part corresponding to the preset detection type, thereby ensuring the detection accuracy of complex assembly structural parts.

[0087] 110. Perform feature extraction on the m high-frequency component parts according to the second feature extraction algorithm, the second algorithm control parameters and the m feature type labels to obtain m second feature sets.

[0088] In a specific implementation, feature extraction may be performed on m high-frequency component parts according to the second feature extraction algorithm, the second algorithm control parameters and the m feature type labels to obtain m second feature sets.

[0089] 111. Determine a target detection result according to the m second feature sets and the target detection verification standard.

[0090] Among them, the m second feature sets reflect the actual situation of the target area, and the target detection verification standard reflects the verification standard of the preset detection type. The final detection result can be obtained by comparing the two.

[0091] In a specific implementation, the target detection result can be determined according to the m second feature sets and the target detection verification standard, so that the detection efficiency of complex assembly structural parts can be improved.

[0092] Optionally, the following steps may also be included:

[0093] Determining a first image quality evaluation threshold and a second image quality evaluation threshold corresponding to the target detection type; the second image quality evaluation threshold is less than the first image quality evaluation threshold;

[0094] When the first image quality evaluation value is greater than or equal to the first image quality evaluation threshold, performing the step of determining a target multiscale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multiscale decomposition algorithm;

[0095] When the first image quality evaluation value is greater than or equal to the second image quality evaluation threshold and less than the first image quality evaluation threshold, performing image enhancement processing on the target area to obtain the target area after image enhancement processing;

[0096] When the first image quality evaluation value is less than the second image quality evaluation threshold, the target area is re-photographed by the camera.

[0097] The first image quality evaluation threshold and the second image quality evaluation threshold may be preset or set by system default.

[0098] In a specific implementation, different threshold pairs can be set based on different detection types, and the threshold pairs include two thresholds. Specifically, the mapping relationship between the preset detection type and the threshold pair can be pre-stored, and then, the first image quality evaluation threshold and the second image quality evaluation threshold corresponding to the target detection type can be determined based on the mapping relationship, and the second image quality evaluation threshold is smaller than the first image quality evaluation threshold.

[0099] Among them, when the first image quality evaluation value is greater than or equal to the first image quality evaluation threshold, it means that the image quality is very good, then the step of determining the target multi-scale decomposition algorithm corresponding to the preset detection type and the first algorithm control parameter corresponding to the target multi-scale decomposition algorithm can be executed, thereby ensuring the accuracy of subsequent detection.

[0100] Among them, when the first image quality evaluation value is greater than or equal to the second image quality evaluation threshold and less than the first image quality evaluation threshold, it means that the image quality is general, but it needs to be enhanced again. Therefore, image enhancement processing can be performed on the target area to obtain the target area after image enhancement processing, and then the step of determining the target multi-scale decomposition algorithm corresponding to the preset detection type and the first algorithm control parameter corresponding to the target multi-scale decomposition algorithm is performed. This image enhancement processing is a global image enhancement processing. After the global image enhancement processing, the image quality can be improved as a whole, and then the accuracy of the later detection can be guaranteed.

[0101] Among them, when the first image quality evaluation value is less than the second image quality evaluation threshold, it means that the image quality is not good and needs to be re-photographed. That is, the target area can be re-photographed through the camera, and then detection is performed based on the image related to the re-photographed target area, thereby ensuring the accuracy of subsequent detection.

[0102] Optionally, performing image enhancement processing on the target area to obtain the target area after image enhancement processing may include the following steps:

[0103] Determine a target low-frequency component part and a target high-frequency component part of the target area;

[0104] Determining a first energy value of the target low-frequency component portion and a second energy value of the target area;

[0105] determining an energy ratio between the first energy value and the second energy value;

[0106] When the energy ratio is within a preset range, a global image enhancement process is performed on the target region to obtain the target region after the image enhancement process.

[0107] The target high-frequency component part may be understood as all high-frequency component parts of the image corresponding to the target area.

[0108] The preset range may be preset or set by system default.

[0109] The global image enhancement process may include at least one of the following: histogram equalization, Retinex algorithm, contrast stretching, etc., which are not limited here.

[0110] In a specific implementation, the target low-frequency component part and the target high-frequency component part of the target area can be determined. The first energy value of the target low-frequency component part and the second energy value of the target area can also be determined, and then the energy ratio between the first energy value and the second energy value is determined, energy ratio = first energy value / second energy value, and then when the energy ratio is within a preset range, the target area is subjected to global image enhancement processing to obtain the target area after image enhancement processing. After global image enhancement processing, the overall image quality can be improved, and then, the accuracy of later detection can be guaranteed.

[0111] Optionally, the preset range includes an upper threshold and a lower threshold. When the energy ratio is greater than or equal to the upper threshold, the following steps may also be included:

[0112] determining a second degree of deviation between the energy ratio and the upper threshold;

[0113] determining a first image enhancement algorithm corresponding to the second deviation;

[0114] Obtaining a first image enhancement algorithm control parameter corresponding to the first image enhancement algorithm;

[0115] Determine a first signal-to-noise ratio of the target low-frequency component portion and a second signal-to-noise ratio of the target high-frequency component portion;

[0116] determining a first relative deviation between the first signal-to-noise ratio and the second signal-to-noise ratio;

[0117] determining a first feedback adjustment parameter corresponding to the first relative deviation;

[0118] Feedback-adjusting the first image enhancement algorithm control parameter according to the first feedback adjustment parameter to obtain a target first image enhancement algorithm control parameter;

[0119] Performing image enhancement processing on the target high-frequency component part according to the first image enhancement algorithm and the target first image enhancement algorithm control parameter to obtain the target high-frequency component part after image enhancement;

[0120] The target area after image enhancement processing is determined according to the target low-frequency component part and the target high-frequency component part after image enhancement.

[0121] In a specific implementation, the low-frequency component represents the main part of the image, and the high-frequency component represents the detailed features of the image. When the energy ratio is greater than or equal to the upper threshold, it means that the main body of the image obscures the detailed features of the image. Then, the second deviation between the energy ratio and the upper threshold can be determined, that is, the second deviation = (energy ratio-upper threshold) / upper threshold. The mapping relationship between the preset deviation and the image enhancement algorithm can also be pre-stored. Then, the first image enhancement algorithm corresponding to the second deviation can be determined based on the mapping relationship. The first image enhancement algorithm control parameters corresponding to the first image enhancement algorithm can also be obtained. Different deviations can correspond to different image enhancement algorithms. Different image enhancement algorithms can correspond to different image enhancement algorithm control parameters. The image enhancement algorithm control parameters are used to control the image enhancement effect.

[0122] Furthermore, a first signal-to-noise ratio of the target low-frequency component part and a second signal-to-noise ratio of the target high-frequency component part can be determined, and then a first relative deviation between the first signal-to-noise ratio and the second signal-to-noise ratio can be determined, the first relative deviation = (first signal-to-noise ratio - second signal-to-noise ratio) / (first signal-to-noise ratio + second signal-to-noise ratio), the relative deviation can make the feedback adjustment softer to a certain extent, and the mapping relationship between the preset relative deviation and the feedback adjustment parameter can be pre-stored, and then, based on the mapping relationship, a first feedback adjustment parameter corresponding to the first relative deviation can be determined, and then the first image enhancement algorithm control parameter can be feedback-adjusted according to the first feedback adjustment parameter to obtain the target first image enhancement algorithm control parameter, that is, the target first image enhancement algorithm control parameter = (1 + first feedback adjustment parameter) * first image enhancement algorithm control parameter, and then according to the first image enhancement algorithm and the target first image enhancement algorithm, the feedback adjustment parameter can be adjusted. The algorithm control parameters perform image enhancement processing on the high-frequency component part of the target to obtain the target high-frequency component part after image enhancement, and reconstruct it according to the target low-frequency component part and the target high-frequency component part after image enhancement to obtain the target area after image enhancement processing. In the case that the main body of the image obscures the detailed features of the image, the second deviation reflects to a certain extent the degree to which the main body of the image obscures the detailed features of the image. Then, the high-frequency component part can be enhanced in a targeted manner based on the degree to which the main body of the image obscures the detailed features of the image, that is, the image enhancement algorithm corresponding to the deviation is determined, and in order to prevent over-enhancement (over-enhancement) or under-enhancement (insufficient enhancement), the difference between the signal-to-noise ratios of the high and low frequency components is used to dynamically feedback adjust the algorithm control parameters of the image enhancement algorithm, so that the final enhancement effect can appropriately highlight the detailed features, which is helpful to improve the accuracy of subsequent detection.

[0123] Optionally, when the energy ratio is less than or equal to the lower limit threshold, the following steps may also be included:

[0124] determining a third degree of deviation between the energy ratio and the lower threshold;

[0125] Determine a second image enhancement algorithm corresponding to the third deviation and a second image enhancement algorithm control parameter corresponding to the second image enhancement algorithm;

[0126] Determining a third image enhancement algorithm corresponding to the third degree of deviation and a third image enhancement algorithm control parameter corresponding to the third image enhancement algorithm;

[0127] Performing image enhancement processing on the target low-frequency component part according to the second image enhancement algorithm and the control parameters of the second image enhancement algorithm to obtain the target low-frequency and high-frequency component part after image enhancement;

[0128] Performing image enhancement processing on the target low-frequency component part according to the third image enhancement algorithm and the control parameters of the third image enhancement algorithm to obtain the target high-frequency component part after image enhancement;

[0129] The target area after image enhancement processing is determined according to the target low-frequency component part after image enhancement and the target high-frequency component part after image enhancement.

[0130] In a specific implementation, when the energy ratio is less than or equal to the lower threshold, it means that the main body of the image is poor and the image details are abnormal. Therefore, the third deviation between the energy ratio and the lower threshold can be determined, and the third deviation = (lower threshold-energy ratio) / lower threshold. It is also possible to pre-store a first mapping relationship between a preset deviation and an image enhancement algorithm, and then determine a second image enhancement algorithm corresponding to the third deviation based on the first mapping relationship, and it is also possible to pre-store a mapping relationship between a preset deviation and an image enhancement algorithm control parameter of the second image enhancement algorithm, and determine a second image enhancement algorithm control parameter corresponding to the second image enhancement algorithm based on the mapping relationship.

[0131] Correspondingly, a second mapping relationship between a preset deviation degree and an image enhancement algorithm can be pre-stored, and a third image enhancement algorithm corresponding to the third deviation degree can be determined based on the second mapping relationship. Also, a mapping relationship between a preset deviation degree and an image enhancement algorithm control parameter of the third image enhancement algorithm can be pre-stored, and the third image enhancement algorithm control parameter corresponding to the third image enhancement algorithm can be determined based on the mapping relationship.

[0132] Next, the target low-frequency component part can be image enhanced according to the second image enhancement algorithm and the second image enhancement algorithm control parameters to obtain the target low-frequency and high-frequency component part after image enhancement, and then the target low-frequency component part can be image enhanced according to the third image enhancement algorithm and the third image enhancement algorithm control parameters to obtain the target high-frequency component part after image enhancement, and the target area after image enhancement is determined according to the enhanced target low-frequency component part and the enhanced target high-frequency component part. When the energy ratio is less than or equal to the lower threshold, the image subject is poor and the image details are abnormal. Therefore, both high and low frequencies need to be enhanced. In addition, the third deviation reflects the degree of poor subject and the degree of abnormality of high-frequency details. Therefore, the image enhancement algorithm corresponding to the deviation and the corresponding image enhancement algorithm control parameters can be adapted respectively, so that the subject can be made full and the details can be highlighted, thereby helping to ensure detection accuracy.

[0133] It can be seen that the visual inspection method for complex assembly structural parts described in the embodiment of the present application is applied to an industrial robot, which includes a camera. A first image of a target device is obtained through the camera, and features are extracted from the first image according to a first feature extraction algorithm to obtain a first feature set. When there are features that meet the preset conditions in the first feature set, a second feature set that meets the preset conditions is screened out from the first feature set, and an area corresponding to the second feature set is determined to obtain a target area. The assembly structural part corresponding to the target area is identified as a complex assembly structural part, and the name of the target assembly structural part corresponding to the second feature set is determined. The target detection type and target detection verification standard corresponding to the name of the target assembly structural part are obtained. When the target detection type is a preset detection type, a target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm are determined. The target area is multi-scale decomposed according to the target multi-scale decomposition algorithm and the first algorithm control parameter to obtain a low-frequency component part and n high-frequency component parts, each high-frequency component part corresponds to a high-frequency component label, and n is a positive integer. The second feature extraction algorithm corresponding to the preset detection type is determined. , the second algorithm control parameters of the second feature extraction algorithm and the target extraction range, the target extraction range includes m high-frequency component labels and m feature type labels, each high-frequency component label corresponds to a feature type label, m is a positive integer less than or equal to n, according to the m high-frequency component labels, from the m high-frequency component parts of the n high-frequency component parts, according to the second feature extraction algorithm, the second algorithm control parameters and the m feature type labels, feature extraction is performed on the m high-frequency component parts to obtain m second feature sets, and the target detection result is determined according to the m second feature sets and the target detection verification standard. First, the camera can be used to identify complex assembly structural parts. Second, based on the corresponding detection type of the complex assembly structural parts, the corresponding multi-scale decomposition algorithm and algorithm control parameters can be determined. Then, the depth detail image (high-frequency component part) corresponding to the detection type corresponding to the complex assembly structural parts is extracted, and then based on the corresponding detection type of the complex assembly structural parts, the required features in the depth detail image are determined, and these features are matched with the detection verification standard corresponding to the detection type of the complex assembly structural parts to obtain the detection result. In this way, the detection efficiency of the complex assembly structural parts can be improved.

[0134] In accordance with the above embodiment, please refer to Figure 2 , Figure 2 : is a schematic diagram of the structure of an industrial robot provided in an embodiment of the present application. As shown in the figure, the industrial robot includes a processor, a memory, a communication interface and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor. In the embodiment of the present application, the industrial robot includes a camera, and the program includes instructions for executing the following steps:

[0135] Acquire a first image of a target device through the camera;

[0136] Performing feature extraction on the first image according to a first feature extraction algorithm to obtain a first feature set;

[0137] When there are features satisfying a preset condition in the first feature set, a second feature set satisfying the preset condition is screened out from the first feature set, an area corresponding to the second feature set is determined to obtain a target area, and an assembly structural part corresponding to the target area is identified as a complex assembly structural part;

[0138] Determine a target assembly structural component name corresponding to the second feature set;

[0139] Obtaining a target detection type and a target detection verification standard corresponding to the target assembly structural part name;

[0140] When the target detection type is a preset detection type, determining a target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm;

[0141] Perform multi-scale decomposition on the target area according to the target multi-scale decomposition algorithm and the first algorithm control parameter to obtain a low-frequency component part and n high-frequency component parts, each high-frequency component part corresponds to a high-frequency component label, and n is a positive integer;

[0142] Determine a second feature extraction algorithm corresponding to the preset detection type, a second algorithm control parameter of the second feature extraction algorithm, and a target extraction range, wherein the target extraction range includes m high-frequency component labels and m feature type labels, each high-frequency component label corresponds to a feature type label, and m is a positive integer less than or equal to n;

[0143] Obtaining corresponding m high-frequency component parts from the n high-frequency component parts according to the m high-frequency component labels;

[0144] Perform feature extraction on the m high-frequency component parts according to the second feature extraction algorithm, the second algorithm control parameters and the m feature type labels to obtain m second feature sets;

[0145] The target detection result is determined according to the m second feature sets and the target detection verification standard.

[0146] Optionally, in terms of determining the target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm, the program includes instructions for executing the following steps:

[0147] Determine a multiscale decomposition algorithm set and a target image quality evaluation index corresponding to the preset detection type, wherein the multiscale decomposition algorithm set includes a plurality of multiscale decomposition algorithms; each multiscale decomposition algorithm corresponds to a reference image quality evaluation value;

[0148] Performing image quality evaluation on the target area according to the target image quality evaluation index to obtain a first image quality evaluation value;

[0149] Determine a target reference image quality evaluation value having the smallest difference with the first image quality evaluation value among the reference image quality evaluation values ​​corresponding to the multiple multi-scale decomposition algorithms;

[0150] Determining the target multi-scale decomposition algorithm according to the target reference image quality evaluation value;

[0151] Acquire a first initial algorithm control parameter of the target multi-scale decomposition algorithm;

[0152] Determining a first deviation between the target reference image quality evaluation value and the first image quality evaluation value;

[0153] determining a first optimization parameter corresponding to the first deviation;

[0154] The first initial algorithm control parameter is optimized according to the first optimization parameter to obtain the first algorithm control parameter.

[0155] Optionally, the program further includes instructions for executing the following steps:

[0156] Determine a first image quality evaluation threshold and a second image quality evaluation threshold corresponding to the target detection type; the second image quality evaluation threshold is less than the first image quality evaluation threshold;

[0157] When the first image quality evaluation value is greater than or equal to the first image quality evaluation threshold, performing the step of determining a target multiscale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multiscale decomposition algorithm;

[0158] When the first image quality evaluation value is greater than or equal to the second image quality evaluation threshold and less than the first image quality evaluation threshold, performing image enhancement processing on the target area to obtain the target area after image enhancement processing;

[0159] When the first image quality evaluation value is less than the second image quality evaluation threshold, the target area is re-photographed by the camera.

[0160] Optionally, in the aspect of performing image enhancement processing on the target area to obtain the target area after image enhancement processing, the above program includes instructions for executing the following steps:

[0161] Determine a target low-frequency component part and a target high-frequency component part of the target area;

[0162] Determining a first energy value of the target low-frequency component portion and a second energy value of the target area;

[0163] determining an energy ratio between the first energy value and the second energy value;

[0164] When the energy ratio is within a preset range, a global image enhancement process is performed on the target region to obtain the target region after the image enhancement process.

[0165] Optionally, the preset range includes an upper threshold and a lower threshold, and when the energy ratio is greater than or equal to the upper threshold, the program further includes instructions for executing the following steps:

[0166] determining a second degree of deviation between the energy ratio and the upper threshold;

[0167] determining a first image enhancement algorithm corresponding to the second deviation;

[0168] Obtaining a first image enhancement algorithm control parameter corresponding to the first image enhancement algorithm;

[0169] Determine a first signal-to-noise ratio of the target low-frequency component portion and a second signal-to-noise ratio of the target high-frequency component portion;

[0170] determining a first relative deviation between the first signal-to-noise ratio and the second signal-to-noise ratio;

[0171] determining a first feedback adjustment parameter corresponding to the first relative deviation;

[0172] Feedback-adjusting the first image enhancement algorithm control parameter according to the first feedback adjustment parameter to obtain a target first image enhancement algorithm control parameter;

[0173] Performing image enhancement processing on the target high-frequency component part according to the first image enhancement algorithm and the target first image enhancement algorithm control parameter to obtain the target high-frequency component part after image enhancement;

[0174] The target area after image enhancement processing is determined according to the target low-frequency component part and the target high-frequency component part after image enhancement.

[0175] Optionally, when the energy ratio is less than or equal to the lower threshold, the program includes instructions for performing the following steps:

[0176] determining a third degree of deviation between the energy ratio and the lower threshold;

[0177] Determine a second image enhancement algorithm corresponding to the third deviation and a second image enhancement algorithm control parameter corresponding to the second image enhancement algorithm;

[0178] Determining a third image enhancement algorithm corresponding to the third degree of deviation and a third image enhancement algorithm control parameter corresponding to the third image enhancement algorithm;

[0179] Performing image enhancement processing on the target low-frequency component part according to the second image enhancement algorithm and the control parameters of the second image enhancement algorithm to obtain the target low-frequency and high-frequency component part after image enhancement;

[0180] Performing image enhancement processing on the target low-frequency component part according to the third image enhancement algorithm and the control parameters of the third image enhancement algorithm to obtain the target high-frequency component part after image enhancement;

[0181] The target area after image enhancement processing is determined according to the target low-frequency component part after image enhancement and the target high-frequency component part after image enhancement.

[0182] It can be seen that the industrial robot for complex assembly structural parts described in the embodiment of the present application includes a camera, which acquires a first image of a target device through the camera, extracts features from the first image according to a first feature extraction algorithm, and obtains a first feature set. When there are features that meet the preset conditions in the first feature set, a second feature set that meets the preset conditions is screened out from the first feature set, and an area corresponding to the second feature set is determined to obtain a target area. The assembly structural parts corresponding to the target area are identified as complex assembly structural parts, and the names of the target assembly structural parts corresponding to the second feature set are determined. The target detection type and the target detection verification standard corresponding to the name of the target assembly structural parts are obtained. When the target detection type is a preset detection type, the target multi-scale decomposition algorithm corresponding to the preset detection type and the first algorithm control parameters corresponding to the target multi-scale decomposition algorithm are determined. The target area is multi-scale decomposed according to the target multi-scale decomposition algorithm and the first algorithm control parameters to obtain a low-frequency component part and n high-frequency component parts, each high-frequency component part corresponds to a high-frequency component label, and n is a positive integer. The second feature extraction algorithm and the second feature corresponding to the preset detection type are determined. The second algorithm control parameters of the extraction algorithm and the target extraction range, the target extraction range includes m high-frequency component labels and m feature type labels, each high-frequency component label corresponds to a feature type label, m is a positive integer less than or equal to n, according to the m high-frequency component labels, from the m high-frequency component parts of the n high-frequency component parts, according to the second feature extraction algorithm, the second algorithm control parameters and the m feature type labels, feature extraction is performed on the m high-frequency component parts to obtain m second feature sets, and the target detection result is determined according to the m second feature sets and the target detection verification standard. First, the camera can be used to identify complex assembly structural parts. Second, based on the corresponding detection type of the complex assembly structural parts, the corresponding multi-scale decomposition algorithm and algorithm control parameters can be determined, and then the depth detail image (high-frequency component part) corresponding to the detection type corresponding to the complex assembly structural parts is extracted, and then based on the corresponding detection type of the complex assembly structural parts, the required features in the depth detail image are determined, and these features are matched with the detection verification standard corresponding to the detection type of the complex assembly structural parts to obtain the detection result. In this way, the detection efficiency of the complex assembly structural parts can be improved.

[0183] Figure 3 300 is a functional unit composition block diagram of a visual inspection system 300 for complex assembly structural parts involved in an embodiment of the present application. The visual inspection system 300 for complex assembly structural parts is applied to an industrial robot, and the industrial robot includes a camera. The visual inspection system 300 for complex assembly structural parts includes: an acquisition unit 301, an extraction unit 302, a determination unit 303, and a decomposition unit 304, wherein:

[0184] The acquisition unit 301 is used to acquire a first image of a target device through the camera;

[0185] The extraction unit 302 is used to extract features from the first image according to a first feature extraction algorithm to obtain a first feature set;

[0186] The determining unit 303 is used to select a second feature set that meets the preset condition from the first feature set when there is a feature that meets the preset condition in the first feature set, determine the area corresponding to the second feature set, obtain the target area, identify the assembly structure corresponding to the target area as a complex assembly structure; and determine the name of the target assembly structure corresponding to the second feature set;

[0187] The acquisition unit 301 is further used to acquire the target detection type and target detection verification standard corresponding to the target assembly structure part name;

[0188] The determining unit 303 is further configured to determine, when the target detection type is a preset detection type, a target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm;

[0189] The decomposition unit 304 is used to perform multi-scale decomposition on the target area according to the target multi-scale decomposition algorithm and the first algorithm control parameter to obtain a low-frequency component part and n high-frequency component parts, each high-frequency component part corresponds to a high-frequency component label, and n is a positive integer;

[0190] The determination unit 303 is further used to determine a second feature extraction algorithm corresponding to the preset detection type, a second algorithm control parameter of the second feature extraction algorithm, and a target extraction range, wherein the target extraction range includes m high-frequency component labels and m feature type labels, each high-frequency component label corresponds to a feature type label, and m is a positive integer less than or equal to n;

[0191] The acquisition unit 301 is further configured to acquire corresponding m high-frequency component parts from the n high-frequency component parts according to the m high-frequency component labels;

[0192] The extraction unit 302 is further configured to perform feature extraction on the m high-frequency component parts according to the second feature extraction algorithm, the second algorithm control parameters and the m feature type labels to obtain m second feature sets;

[0193] The determining unit 303 is further configured to determine a target detection result according to the m second feature sets and the target detection verification standard.

[0194] Optionally, in determining the target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm, the determining unit 303 is specifically configured to:

[0195] Determine a multiscale decomposition algorithm set and a target image quality evaluation index corresponding to the preset detection type, wherein the multiscale decomposition algorithm set includes a plurality of multiscale decomposition algorithms; each multiscale decomposition algorithm corresponds to a reference image quality evaluation value;

[0196] Performing image quality evaluation on the target area according to the target image quality evaluation index to obtain a first image quality evaluation value;

[0197] Determine a target reference image quality evaluation value having the smallest difference with the first image quality evaluation value among the reference image quality evaluation values ​​corresponding to the multiple multi-scale decomposition algorithms;

[0198] Determining the target multi-scale decomposition algorithm according to the target reference image quality evaluation value;

[0199] Acquire a first initial algorithm control parameter of the target multi-scale decomposition algorithm;

[0200] Determining a first deviation between the target reference image quality evaluation value and the first image quality evaluation value;

[0201] determining a first optimization parameter corresponding to the first deviation;

[0202] The first initial algorithm control parameter is optimized according to the first optimization parameter to obtain the first algorithm control parameter.

[0203] Optionally, the visual inspection system 300 for complex assembly structural parts is also specifically used for:

[0204] Determine a first image quality evaluation threshold and a second image quality evaluation threshold corresponding to the target detection type; the second image quality evaluation threshold is less than the first image quality evaluation threshold;

[0205] When the first image quality evaluation value is greater than or equal to the first image quality evaluation threshold, performing the step of determining a target multiscale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multiscale decomposition algorithm;

[0206] When the first image quality evaluation value is greater than or equal to the second image quality evaluation threshold and less than the first image quality evaluation threshold, performing image enhancement processing on the target area to obtain the target area after image enhancement processing;

[0207] When the first image quality evaluation value is less than the second image quality evaluation threshold, the target area is re-photographed by the camera.

[0208] Optionally, in the aspect of performing image enhancement processing on the target area to obtain the target area after image enhancement processing, the visual inspection system 300 for complex assembly structural parts is further specifically used for:

[0209] Determine a target low-frequency component part and a target high-frequency component part of the target area;

[0210] Determining a first energy value of the target low-frequency component portion and a second energy value of the target area;

[0211] determining an energy ratio between the first energy value and the second energy value;

[0212] When the energy ratio is within a preset range, a global image enhancement process is performed on the target region to obtain the target region after the image enhancement process.

[0213] Optionally, the preset range includes an upper threshold and a lower threshold. When the energy ratio is greater than or equal to the upper threshold, the visual inspection system 300 for complex assembly structural parts is further specifically used for:

[0214] determining a second degree of deviation between the energy ratio and the upper threshold;

[0215] determining a first image enhancement algorithm corresponding to the second deviation;

[0216] Obtaining a first image enhancement algorithm control parameter corresponding to the first image enhancement algorithm;

[0217] Determine a first signal-to-noise ratio of the target low-frequency component portion and a second signal-to-noise ratio of the target high-frequency component portion;

[0218] determining a first relative deviation between the first signal-to-noise ratio and the second signal-to-noise ratio;

[0219] determining a first feedback adjustment parameter corresponding to the first relative deviation;

[0220] Feedback-adjusting the first image enhancement algorithm control parameter according to the first feedback adjustment parameter to obtain a target first image enhancement algorithm control parameter;

[0221] Performing image enhancement processing on the target high-frequency component part according to the first image enhancement algorithm and the target first image enhancement algorithm control parameter to obtain the target high-frequency component part after image enhancement;

[0222] The target area after image enhancement processing is determined according to the target low-frequency component part and the target high-frequency component part after image enhancement.

[0223] Optionally, when the energy ratio is less than or equal to the lower threshold, the visual inspection system 300 for complex assembly structural parts is further specifically used for:

[0224] determining a third degree of deviation between the energy ratio and the lower threshold;

[0225] Determine a second image enhancement algorithm corresponding to the third deviation and a second image enhancement algorithm control parameter corresponding to the second image enhancement algorithm;

[0226] Determining a third image enhancement algorithm corresponding to the third degree of deviation and a third image enhancement algorithm control parameter corresponding to the third image enhancement algorithm;

[0227] Performing image enhancement processing on the target low-frequency component part according to the second image enhancement algorithm and the control parameters of the second image enhancement algorithm to obtain the target low-frequency and high-frequency component part after image enhancement;

[0228] Performing image enhancement processing on the target low-frequency component part according to the third image enhancement algorithm and the control parameters of the third image enhancement algorithm to obtain the target high-frequency component part after image enhancement;

[0229] The target area after image enhancement processing is determined according to the target low-frequency component part after image enhancement and the target high-frequency component part after image enhancement.

[0230] It can be seen that the visual inspection system for complex assembly structural parts described in the embodiment of the present application is applied to an industrial robot, which includes a camera. A first image of a target device is obtained through the camera, and features are extracted from the first image according to a first feature extraction algorithm to obtain a first feature set. When there are features that meet the preset conditions in the first feature set, a second feature set that meets the preset conditions is screened out from the first feature set, and an area corresponding to the second feature set is determined to obtain a target area. The assembly structural part corresponding to the target area is identified as a complex assembly structural part, and the name of the target assembly structural part corresponding to the second feature set is determined. The target detection type and target detection verification standard corresponding to the name of the target assembly structural part are obtained. When the target detection type is a preset detection type, a target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm are determined. The target area is multi-scale decomposed according to the target multi-scale decomposition algorithm and the first algorithm control parameter to obtain a low-frequency component part and n high-frequency component parts, each high-frequency component part corresponds to a high-frequency component label, and n is a positive integer. The second feature extraction algorithm corresponding to the preset detection type is determined. , the second algorithm control parameters of the second feature extraction algorithm and the target extraction range, the target extraction range includes m high-frequency component labels and m feature type labels, each high-frequency component label corresponds to a feature type label, m is a positive integer less than or equal to n, according to the m high-frequency component labels, from the m high-frequency component parts of the n high-frequency component parts, according to the second feature extraction algorithm, the second algorithm control parameters and the m feature type labels, feature extraction is performed on the m high-frequency component parts to obtain m second feature sets, and the target detection result is determined according to the m second feature sets and the target detection verification standard. First, the camera can be used to identify complex assembly structural parts. Second, based on the corresponding detection type of the complex assembly structural parts, the corresponding multi-scale decomposition algorithm and algorithm control parameters can be determined. Then, the depth detail image (high-frequency component part) corresponding to the detection type corresponding to the complex assembly structural parts is extracted, and then based on the corresponding detection type of the complex assembly structural parts, the required features in the depth detail image are determined, and these features are matched with the detection verification standard corresponding to the detection type of the complex assembly structural parts to obtain the detection result. In this way, the detection efficiency of the complex assembly structural parts can be improved.

[0231] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps of any method recorded in the above method embodiments, and the above computer includes an industrial robot.

[0232] The present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps of any method described in the method embodiment. The computer program product may be a software installation package, and the computer includes an industrial robot.

[0233] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0234] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0235] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0236] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0237] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0238] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the above-mentioned methods in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0239] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable memory, which may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0240] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A visual inspection method for complex assembly structural parts, characterized in that: Applied to an industrial robot, the industrial robot includes a camera, and the method includes: Acquire a first image of a target device through the camera; Performing feature extraction on the first image according to a first feature extraction algorithm to obtain a first feature set; When there are features satisfying a preset condition in the first feature set, a second feature set satisfying the preset condition is screened out from the first feature set, an area corresponding to the second feature set is determined to obtain a target area, and an assembly structural part corresponding to the target area is identified as a complex assembly structural part; Determine a target assembly structural component name corresponding to the second feature set; Obtaining a target detection type and a target detection verification standard corresponding to the target assembly structural part name; When the target detection type is a preset detection type, determining a target multiscale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multiscale decomposition algorithm, specifically comprising: determining a multiscale decomposition algorithm set and a target image quality evaluation index corresponding to the preset detection type, the multiscale decomposition algorithm set comprising a plurality of multiscale decomposition algorithms, each multiscale decomposition algorithm corresponding to a reference image quality evaluation value, performing image quality evaluation on the target area according to the target image quality evaluation index to obtain a first image quality evaluation value, determining a target reference image quality evaluation value having the smallest difference with the first image quality evaluation value among the reference image quality evaluation values ​​corresponding to the plurality of multiscale decomposition algorithms, determining the target multiscale decomposition algorithm according to the target reference image quality evaluation value, acquiring a first initial algorithm control parameter of the target multiscale decomposition algorithm, determining a first deviation between the target reference image quality evaluation value and the first image quality evaluation value, determining a first optimization parameter corresponding to the first deviation, optimizing the first initial algorithm control parameter according to the first optimization parameter, and obtaining the first algorithm control parameter; Perform multi-scale decomposition on the target area according to the target multi-scale decomposition algorithm and the first algorithm control parameter to obtain a low-frequency component part and n high-frequency component parts, each high-frequency component part corresponds to a high-frequency component label, and n is a positive integer; Determine a second feature extraction algorithm corresponding to the preset detection type, a second algorithm control parameter of the second feature extraction algorithm, and a target extraction range, wherein the target extraction range includes m high-frequency component labels and m feature type labels, each high-frequency component label corresponds to a feature type label, and m is a positive integer less than or equal to n; Obtaining corresponding m high-frequency component parts from the n high-frequency component parts according to the m high-frequency component labels; Perform feature extraction on the m high-frequency component parts according to the second feature extraction algorithm, the second algorithm control parameters and the m feature type labels to obtain m second feature sets; The target detection result is determined according to the m second feature sets and the target detection verification standard.

2. The method according to claim 1, characterized in that The method further comprises: Determine a first image quality evaluation threshold and a second image quality evaluation threshold corresponding to the target detection type; the second image quality evaluation threshold is less than the first image quality evaluation threshold; When the first image quality evaluation value is greater than or equal to the first image quality evaluation threshold, performing the step of determining a target multiscale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multiscale decomposition algorithm; When the first image quality evaluation value is greater than or equal to the second image quality evaluation threshold and less than the first image quality evaluation threshold, performing image enhancement processing on the target area to obtain the target area after image enhancement processing; When the first image quality evaluation value is less than the second image quality evaluation threshold, the target area is re-photographed by the camera.

3. The method according to claim 2, characterized in that The performing image enhancement processing on the target area to obtain the target area after image enhancement processing includes: Determine a target low-frequency component part and a target high-frequency component part of the target area; Determining a first energy value of the target low-frequency component portion and a second energy value of the target area; determining an energy ratio between the first energy value and the second energy value; When the energy ratio is within a preset range, a global image enhancement process is performed on the target region to obtain the target region after the image enhancement process.

4. The method according to claim 3, characterized in that The preset range includes an upper threshold and a lower threshold. When the energy ratio is greater than or equal to the upper threshold, the method further includes: determining a second degree of deviation between the energy ratio and the upper threshold; determining a first image enhancement algorithm corresponding to the second deviation; Obtaining a first image enhancement algorithm control parameter corresponding to the first image enhancement algorithm; Determine a first signal-to-noise ratio of the target low-frequency component portion and a second signal-to-noise ratio of the target high-frequency component portion; determining a first relative deviation between the first signal-to-noise ratio and the second signal-to-noise ratio; determining a first feedback adjustment parameter corresponding to the first relative deviation; Feedback-adjusting the first image enhancement algorithm control parameter according to the first feedback adjustment parameter to obtain a target first image enhancement algorithm control parameter; Performing image enhancement processing on the target high-frequency component part according to the first image enhancement algorithm and the target first image enhancement algorithm control parameter to obtain the target high-frequency component part after image enhancement; The target area after image enhancement processing is determined according to the target low-frequency component part and the target high-frequency component part after image enhancement.

5. A visual inspection system for complex assembly structural parts, characterized in that: Applied to an industrial robot, the industrial robot includes a camera, the system is used to implement the method according to claim 1, the system includes: an acquisition unit, an extraction unit, a determination unit, and a decomposition unit, wherein: The acquisition unit is used to acquire a first image of a target device through the camera; The extraction unit is used to extract features from the first image according to a first feature extraction algorithm to obtain a first feature set; The determining unit is used to, when there are features satisfying the preset conditions in the first feature set, filter out a second feature set satisfying the preset conditions from the first feature set, determine an area corresponding to the second feature set, obtain a target area, identify the assembly structure corresponding to the target area as a complex assembly structure; and determine a name of the target assembly structure corresponding to the second feature set; The acquisition unit is further used to acquire the target detection type and target detection verification standard corresponding to the name of the target assembly structure part; The determining unit is further configured to determine, when the target detection type is a preset detection type, a target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm; The decomposition unit is used to perform multi-scale decomposition on the target area according to the target multi-scale decomposition algorithm and the first algorithm control parameter to obtain a low-frequency component part and n high-frequency component parts, each high-frequency component part corresponds to a high-frequency component label, and n is a positive integer; The determination unit is further used to determine a second feature extraction algorithm corresponding to the preset detection type, a second algorithm control parameter of the second feature extraction algorithm, and a target extraction range, wherein the target extraction range includes m high-frequency component labels and m feature type labels, each high-frequency component label corresponds to a feature type label, and m is a positive integer less than or equal to n; The acquisition unit is further used to acquire corresponding m high-frequency component parts from the n high-frequency component parts according to the m high-frequency component labels; The extraction unit is further used to perform feature extraction on the m high-frequency component parts according to the second feature extraction algorithm, the second algorithm control parameters and the m feature type labels to obtain m second feature sets; The determination unit is further used to determine the target detection result according to the m second feature sets and the target detection verification standard.

6. The system according to claim 5, characterized in that In terms of determining the target multi-scale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multi-scale decomposition algorithm, the determining unit is specifically used to: Determine a multiscale decomposition algorithm set and a target image quality evaluation index corresponding to the preset detection type, wherein the multiscale decomposition algorithm set includes a plurality of multiscale decomposition algorithms; each multiscale decomposition algorithm corresponds to a reference image quality evaluation value; Performing image quality evaluation on the target area according to the target image quality evaluation index to obtain a first image quality evaluation value; Determine a target reference image quality evaluation value having the smallest difference with the first image quality evaluation value among the reference image quality evaluation values ​​corresponding to the multiple multi-scale decomposition algorithms; Determining the target multi-scale decomposition algorithm according to the target reference image quality evaluation value; Acquire a first initial algorithm control parameter of the target multi-scale decomposition algorithm; Determining a first deviation between the target reference image quality evaluation value and the first image quality evaluation value; determining a first optimization parameter corresponding to the first deviation; The first initial algorithm control parameter is optimized according to the first optimization parameter to obtain the first algorithm control parameter.

7. The system according to claim 6, characterized in that The system is also specifically used for: Determine a first image quality evaluation threshold and a second image quality evaluation threshold corresponding to the target detection type; the second image quality evaluation threshold is less than the first image quality evaluation threshold; When the first image quality evaluation value is greater than or equal to the first image quality evaluation threshold, performing the step of determining a target multiscale decomposition algorithm corresponding to the preset detection type and a first algorithm control parameter corresponding to the target multiscale decomposition algorithm; When the first image quality evaluation value is greater than or equal to the second image quality evaluation threshold and less than the first image quality evaluation threshold, performing image enhancement processing on the target area to obtain the target area after image enhancement processing; When the first image quality evaluation value is less than the second image quality evaluation threshold, the target area is re-photographed by the camera.

8. The system according to claim 7, characterized in that In the aspect of performing image enhancement processing on the target area to obtain the target area after image enhancement processing, the system is further specifically used for: Determine a target low-frequency component part and a target high-frequency component part of the target area; Determining a first energy value of the target low-frequency component portion and a second energy value of the target area; determining an energy ratio between the first energy value and the second energy value; When the energy ratio is within a preset range, a global image enhancement process is performed on the target region to obtain the target region after the image enhancement process.

9. The system according to claim 8, characterized in that The preset range includes an upper threshold and a lower threshold. When the energy ratio is greater than or equal to the upper threshold, the system is further specifically configured to: determining a second degree of deviation between the energy ratio and the upper threshold; determining a first image enhancement algorithm corresponding to the second deviation; Obtaining a first image enhancement algorithm control parameter corresponding to the first image enhancement algorithm; Determine a first signal-to-noise ratio of the target low-frequency component portion and a second signal-to-noise ratio of the target high-frequency component portion; determining a first relative deviation between the first signal-to-noise ratio and the second signal-to-noise ratio; determining a first feedback adjustment parameter corresponding to the first relative deviation; Feedback-adjusting the first image enhancement algorithm control parameter according to the first feedback adjustment parameter to obtain a target first image enhancement algorithm control parameter; Performing image enhancement processing on the target high-frequency component part according to the first image enhancement algorithm and the target first image enhancement algorithm control parameter to obtain the target high-frequency component part after image enhancement; The target area after image enhancement processing is determined according to the target low-frequency component part and the target high-frequency component part after image enhancement.

Images