Industrial product intelligent detection method and device

Abstract

The invention discloses an industrial product intelligent detection method and device. The detection method comprises the following steps: 1) teaching and obtaining path points [p1, p2,..., pn] and shooting parameters [C1, C2,..., Cn]; 2) generating a defect detection path; adjusting and optimizing the number, the position and the sequence of the path points [p1, p2,..., pn] and the corresponding shooting parameters [C1, C2,..., Cn] to obtain defect detection path points [p * 1, p * 2,..., p * m] and detection shooting parameters [C * 1, C * 2,..., C * m]; and 3) obtaining a detection image set of the product according to the defect detection path points [p * 1, p * 2,..., p * m] and the detection shooting parameters [C * 1, C * 2,..., C * m], and carrying out image detection processing on the detection image set by adopting a universal defect detection algorithm to obtain a surface defect detection result of the product. Corresponding detection paths are generated for different products, adaptability is good, comprehensive defect detection of the products is achieved, and the application range is wide.

Description

Technical Field

[0001] The present invention relates to the technical field of product detection, and in particular to an intelligent detection method and device for industrial products. Background Art

[0002] Machine vision technology is widely used in the industrial field for appearance and quality inspection, covering multiple product categories. Traditional visual inspection technology adopts an assembly line inspection method, with multiple inspection stations set up on the inspection production line, and different cameras and industrial light sources arranged at the stations. By changing the position of the camera and light source, the appearance of industrial products is traversed and inspected. This inspection method has obvious limitations, and the defect detection range is limited, which is limited to more obvious and local defect types, and it is difficult to be competent for comprehensive detection of product defects. In addition, when the current product changes, the inspection equipment on the product line may need to be readjusted and remade, and the algorithm must also be redeveloped. The previous product line inspection equipment is eliminated, and the utilization rate is too low, which directly increases the production cost of the product. Summary of the invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, it is desired to provide an intelligent detection method and device for industrial products, which can generate corresponding detection paths for different products, have good adaptability, realize comprehensive defect detection of products, and have a wide range of applications.

[0004] The present invention provides an industrial product intelligent detection method, comprising the following steps:

[0005] 1) Teach and obtain path points [p1, p2, ..., p n ] and the corresponding shooting parameters [C1, C2, ..., C n ], shooting parameters are the exposure parameters set by the industrial camera;

[0006] 2) Generate defect detection paths, adjust and optimize path points [p1, p2, ..., p n ] and the corresponding shooting parameters [C1, C2, ..., C n ], and obtain the defect detection path points [p*1, p*2, ..., p* m ] and the corresponding detection shooting parameters [C*1, C*2, ..., C* m ];

[0007] 3) According to the defect detection path point [p*1, p*2, ..., p* m ], detect shooting parameters [C*1, C*2, ..., C* m] Obtain a detection image set of the product, use a universal defect detection algorithm to perform image detection processing on the detection image set, and obtain the surface defect detection result of the product.

[0008] Furthermore, in the step 1), a number of defective products of a product are selected, and the number of defective products includes all defects of the product; the defective products are moved to each target point position one by one, and are photographed and images are obtained respectively; the image is checked, and if the defect is clearly visible, the target point position is recorded as the path point p i , record the corresponding shooting parameters C i ; until all defect positions are covered, the path points [p1, p2, ..., p n ] and the corresponding shooting parameters [C1, C2, ..., C n ], i=1...n.

[0009] Furthermore, in step 2), the optimization of the path points includes the following steps:

[0010] 211) Calculate each path point p i The detection field V i :

[0011] V i =f(p i ,C i );

[0012] Where i = 1...n, f is the path point p i and shooting parameters C i and detection field V i The function of the relationship between the detection field V i It is the size and range of the spatial measurement area under a single measurement field of view;

[0013] 212) The optimization goal is to maximize the detection coverage and minimize the path length, that is, to ensure that all defect areas are covered by at least one path point p i The detection field V i Minimize the total distance of the detection path when covering;

[0014] The optimization objective function F is:

[0015]

[0016] Among them, S pi is the path point p i The area of ​​defective region covered, S D is the total defect area;

[0017] S pi / SD is the path point p i Detection coverage;

[0018] λ is the optimization weight coefficient of the shortest path, N(p i ) is the function for calculating the path length.

[0019] Furthermore, in step 2), the optimization of the shooting parameters includes the following steps:

[0020] 221) For the same defect detection path point p* i Apply different shooting parameters to take pictures to obtain q images, and synthesize the q images into an HDR image by weighted averaging method or radiance map estimation method;

[0021]

[0022] Where j = 1...q, I HDR (x,y) is the pixel value at position (x,y) in the HDR image;

[0023] w(L j (x,y)) is the weight function;

[0024] L j (x,y) is the brightness value of position (x,y) in the jth image;

[0025] t j is the exposure time of the jth image;

[0026] 222) performing histogram analysis and threshold segmentation on images with different exposure values;

[0027]

[0028] Among them, H j (L) is the grayscale histogram of the jth image;

[0029] I j (x, y) is the grayscale value of position (x, y) in the jth image;

[0030] L is the gray level;

[0031] When I j When (x,y)=L,δ(I j (x,y)-L) function value is 1, otherwise it is 0;

[0032] 223) Calculate the matching degree;

[0033]

[0034] Among them, E jis the matching degree between the jth image and the HDR image,

[0035] H HDR (L) is the grayscale histogram of the HDR image;

[0036] min(H HDR (L),H j (L)) is to calculate the overlap of the two histograms at each gray level, which is used to measure the similarity between the jth image and the HDR image;

[0037] 224) Draw the matching degree and grayscale curve of q images, fit the curve and find the peak to find the defect detection path point p* i The optimal exposure value at the current position is taken as the shooting parameter C* i .

[0038] Furthermore, in step 3), the general defect detection algorithm includes the following steps:

[0039] 31) In the defect detection task, the path point p* i And the corresponding shooting parameter C* i It is represented as a one-dimensional sequence vector At; the input one-dimensional sequence vector At is dimensionally expanded through a single linear layer to obtain an expanded one-dimensional vector A*t, whose dimension d is the same as the dimension d of the image feature vector f Consistency;

[0040] 32) Obtain five consecutive images from the detection image set as image subsets, use the improved CSPDarknet network to extract features for each image in the image subset, and generate feature maps of three different scales, including small-scale feature maps, medium-scale feature maps, and large-scale feature maps; splice the five frames of feature maps of each scale in the channel dimension to form a high-dimensional feature map F S , whose dimension is five times that of the original single-frame feature map;

[0041] 33) Compare the expanded one-dimensional vector A*t with the corresponding high-dimensional feature map F S Fusion is performed to form the fused feature representation F t ; F t Input to the encoder module of the Transformer network, the encoder module represents the fused feature F based on the self-attention mechanism t Processing to generate high-level feature representation E t , feature representation E t The dimension and feature representation F t The dimension is consistent with d f ;

[0042] 34) Represent the feature Et The decoder module is input to the Transformer, and the decoder module decodes the feature representation into the detection result based on the sequence-to-sequence mapping; the output of the decoder module is a set of vectors representing the defect location and defect type, which are used to identify and classify the defects in the detection image.

[0043] In addition, the present invention also provides a detection device for the above-mentioned intelligent detection method of industrial products, including a detection platform and a multi-degree-of-freedom motion mechanism and a visual camera module arranged on the detection platform, and the multi-degree-of-freedom motion mechanism is provided with a clamp for clamping the product to be detected.

[0044] Furthermore, the multi-degree-of-freedom motion mechanism is a multi-degree-of-freedom motion platform or a multi-axis manipulator.

[0045] Furthermore, the visual camera module includes an industrial camera, a halogen lamp light source and an LED parallel light source; the visual camera module is electrically connected to a programmable logic controller.

[0046] Compared with the prior art, the beneficial effects of the present invention are:

[0047] The detection method of the present invention obtains the path points [p1, p2, ..., p n ], and the corresponding shooting parameters [C1, C2, ..., C n ]; then optimize the initial detection path to obtain the final defect detection path points [p*1, p*2, ..., p* m ] and the corresponding detection shooting parameters [C*1, C*2, ..., C* m ]; then, according to the defect detection path and the detection shooting parameters, a detection image set of the product is obtained, and a universal defect detection algorithm is used to perform image detection processing on the detection image set to obtain the surface defect detection result of the product. The detection method of the present invention generates corresponding defect detection paths for different products, and images of various defects of the product can be obtained according to the defect detection path. It has good adaptability, realizes comprehensive defect detection of products, and has a wide range of applications.

[0048] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0050] Figure 1 is a flow chart of the detection method;

[0051] Figure 2 It is a structural schematic diagram of the detection device;

[0052] Figure 3 Schematic diagram of the network model of the general defect detection algorithm.

[0053] Numbers in the figure: 1. Testing platform; 2. Multi-degree-of-freedom motion mechanism; 3. Visual camera module; 4. Fixture;

[0054] 31. Industrial camera; 32. Halogen lamp light source; 33. LED parallel light source. DETAILED DESCRIPTION

[0055] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant inventions, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0056] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0057] Example 1

[0058] Please refer to Figure 1 , an embodiment of the present invention provides an industrial product intelligent detection method, comprising the following steps:

[0059] 1) Select several defective products of a product, which include all defects of the product; move the defective products to the target points one by one through teaching, take pictures of them and obtain images; check the images, if the defects are clearly visible, record the target point position as the path point p i , record the corresponding shooting parameters C i ; until all defect positions are covered, the path points [p1, p2, ..., p n ] and the corresponding shooting parameters [C1, C2, ..., C n ], i=1...n; shooting parameters are exposure parameters set by industrial cameras;

[0060] 2) Generate defect detection path; adjust and optimize path points [p1, p2, ..., p n ] and the corresponding shooting parameters [C1, C2, ..., C n ], and obtain the final defect detection path points [p*1, p*2, ..., p*m ] and the corresponding detection shooting parameters [C*1, C*2, ..., C* m ];

[0061] The optimization of path points includes the following steps:

[0062] 211) Calculate each path point p i The detection field V i :

[0063] V i =f(p i ,C i );

[0064] Where i = 1...n, f is the path point p i and shooting parameters C i and detection field V i The function of the relationship between the detection field V i It is the size and range of the spatial measurement area under a single measurement field of view;

[0065] 212) The optimization goal is to maximize the detection coverage and minimize the path length, that is, to ensure that all defect areas are covered by at least one path point p i The detection field V i Minimize the total distance of the detection path when covering;

[0066] The optimization objective function F is:

[0067]

[0068] Among them, S pi is the path point p i The area of ​​defective region covered, S D is the total defect area;

[0069] S pi / S D is the path point p i Detection coverage;

[0070] λ is the optimization weight coefficient of the shortest path, N(p i ) is a function for calculating path length;

[0071] The optimization of shooting parameters includes the following steps:

[0072] 221) For the same defect detection path point p* i Apply different shooting parameters to take pictures to obtain q images, and synthesize the q images into an HDR image by weighted averaging method or radiance map estimation method;

[0073]

[0074] Where j = 1...q, I HDR (x,y) is the pixel value at position (x,y) in the HDR image;

[0075] w(L j (x,y)) is the weight function;

[0076] L j (x,y) is the brightness value of position (x,y) in the jth image;

[0077] t j is the exposure time of the jth image;

[0078] 222) performing histogram analysis and threshold segmentation on images with different exposure values;

[0079]

[0080] Among them, H j (L) is the grayscale histogram of the jth image;

[0081] I j (x, y) is the grayscale value of position (x, y) in the jth image;

[0082] L is the gray level;

[0083] When I j When (x,y)=L,δ(I j (x,y)-L) function value is 1, otherwise it is 0;

[0084] 223) Calculate the matching degree;

[0085]

[0086] Among them, E j is the matching degree between the jth image and the HDR image,

[0087] H HDR (L) is the grayscale histogram of the HDR image;

[0088] min(H HDR (L),H j (L)) is to calculate the overlap of the two histograms at each gray level, which is used to measure the similarity between the jth image and the HDR image;

[0089] 224) Draw the matching degree and grayscale curve of q images, fit the curve and find the peak to find the defect detection path point p* i The optimal exposure value at the current position is taken as the shooting parameter C* i;

[0090] 3) According to the defect detection path point [p*1, p*2, ..., p* m ], detect shooting parameters [C*1, C*2, ..., C* m ] Obtain a detection image set of a product, use a universal defect detection algorithm to perform image detection processing on the detection image set, and obtain a surface defect detection result of the product;

[0091] like Figure 3 As shown, the general defect detection algorithm includes the following steps:

[0092] 31) In the defect detection task, the path point p* i And the corresponding shooting parameter C* i It is represented as a one-dimensional sequence vector At; the input one-dimensional sequence vector At is dimensionally expanded through a single linear layer to obtain an expanded one-dimensional vector A*t, whose dimension d is the same as the dimension d of the image feature vector f Consistency;

[0093] 32) Obtain five consecutive images from the detection image set as image subsets, use the improved CSPDarknet network to extract features for each image in the image subset, and generate feature maps of three different scales, including small-scale feature maps, medium-scale feature maps, and large-scale feature maps; splice the five frames of feature maps of each scale in the channel dimension to form a high-dimensional feature map F S , whose dimension is five times that of the original single-frame feature map;

[0094] 33) Compare the expanded one-dimensional vector A*t with the corresponding high-dimensional feature map F S Fusion is performed to form the fused feature representation F t ; F t Input to the encoder module of the Transformer network, the encoder module represents the fused feature F based on the self-attention mechanism t Processing to generate high-level feature representation E t , feature representation E t The dimension and feature representation F t The dimension is consistent with d f ;

[0095] 34) Represent the feature E t The decoder module is input to the Transformer, and the decoder module decodes the feature representation into the detection result based on the sequence-to-sequence mapping; the output of the decoder module is a set of vectors representing the defect location and defect type, which are used to identify and classify the defects in the detection image.

[0096] In this embodiment, the surface defect detection results of the product include the location of the defect, the classification information of the defect, and the confidence of the defect. Specifically, the location of the defect is obtained by regression method, the classification information of the defect is obtained by classifier, and the confidence of the defect indicates the credibility of the network to the detected defect. Through the self-attention mechanism and multi-layer perception mechanism in the model, efficient integration and accurate extraction of features are achieved, thereby improving the accuracy and robustness of complex detection of surface defects of industrial products.

[0097] The detection method of the present invention obtains the path points [p1, p2, ..., p n ], and the corresponding shooting parameters [C1, C2, ..., C n ]; then optimize the initial detection path to obtain the defect detection path points [p*1, p*2, ..., p* m ] and the corresponding detection shooting parameters [C*1, C*2, ..., C* m ]; then, a detection image set of the product is obtained according to the defect detection path and the detection shooting parameters, and a universal defect detection algorithm is used to perform image detection processing on the detection image set to obtain the surface defect detection result of the product.

[0098] The detection method of the present invention generates corresponding defect detection paths for different products. Images of various defects of the product can be obtained according to the defect detection paths. It has good adaptability, realizes comprehensive defect detection of products, and has a wide range of applications.

[0099] Example 2

[0100] Please refer to Figure 2 An embodiment of the present invention further provides an intelligent detection device for industrial products, comprising a detection platform 1 and a multi-degree-of-freedom motion mechanism 2 and a visual camera module 3 arranged on the detection platform 1; the multi-degree-of-freedom motion mechanism 2 is a multi-degree-of-freedom motion platform or a multi-axis manipulator; the visual camera module 3 comprises an industrial camera 31, a halogen lamp light source 32 and an LED parallel light source 33; the visual camera module 3 is electrically connected to a programmable logic controller; and a fixture 4 for clamping the product to be detected is arranged on the multi-degree-of-freedom motion mechanism 2.

[0101] In this embodiment, a multi-degree-of-freedom motion mechanism 2, such as a multi-axis manipulator or other similar multi-degree-of-freedom motion platform, is used to mount the workpiece to be inspected. The motion mechanism has sufficient degrees of freedom to achieve omnidirectional clamping and multi-angle movement, so as to fully inspect the workpiece at different angles and optical environments; the motion path and speed can also be accurately controlled by an automated control system to improve the inspection efficiency and accuracy.

[0102] During the inspection process, the multi-degree-of-freedom motion mechanism 2 moves the workpiece to be inspected to the starting point of the inspection mode through the fixture 4, and moves point by point according to the pre-taught trajectory and takes pictures. For different inspection modes, the light source is adjusted to ensure appropriate lighting conditions. Subsequently, the captured images are analyzed using the industrial product intelligent inspection network based on human behavior simulation, and whether the inspected product is a defective product is determined based on the pre-learned model to achieve comprehensive defect detection.

[0103] The present invention is based on deep learning technology and combines a large number of real workpiece defect images for training, which can accurately and efficiently identify various types of defects. The workpiece is photographed from multiple angles and combined with the known motion mechanism posture corresponding to the photographed workpiece, which realizes multi-dimensional, high-precision and high-adaptability detection of the same defect.

[0104] In the description of this specification, the description of the terms "one embodiment", "some embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0105] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An intelligent detection method for industrial products, characterized in that: The steps include: 1) Teach and obtain path points [p1, p2, ..., p n ] and the corresponding shooting parameters [C1, C2, ..., C n ], shooting parameters are the exposure parameters set by the industrial camera; 2) Generate defect detection paths, adjust and optimize path points [p1, p2, ..., p n ] and the corresponding shooting parameters [C1, C2, ..., C n ], and obtain the defect detection path points [p*1, p*2, ..., p* m ] and the corresponding detection shooting parameters [C*1, C*2, ..., C* m ]; 3) According to the defect detection path point [p*1, p*2, ..., p* m ], detect shooting parameters [C*1, C*2, ..., C* m ] Obtain a detection image set of the product, use a universal defect detection algorithm to perform image detection processing on the detection image set, and obtain the surface defect detection result of the product.

2. The industrial product intelligent detection method according to claim 1, characterized in that: In the step 1), a number of defective products of a product are selected, and the number of defective products includes all defects of the product; the defective products are moved to the positions of the target points one by one, and are photographed and images are obtained respectively; the images are checked, and if the defects are clearly visible, the position of the target point is recorded as the path point p i , record the corresponding shooting parameters C i ; until all defect positions are covered, the path points [p1, p2, ..., p n ] and the corresponding shooting parameters [C1, C2, ..., C n ], i=1...n.

3. The industrial product intelligent detection method according to claim 1, characterized in that: In step 2), the optimization of the path points includes the following steps: 211) Calculate each path point p i The detection field V i : V i =f(p i ,C i ); Where i = 1...n, f is the path point p i and shooting parameters C i and detection field V i The function of the relationship between the detection field V i It is the size and range of the spatial measurement area under a single measurement field of view; 212) The optimization goal is to maximize the detection coverage and minimize the path length, that is, to ensure that all defect areas are covered by at least one path point p i The detection field V i Minimize the total distance of the detection path when covering; The optimization objective function F is: Among them, S pi is the path point p i The area of ​​defective region covered, S D is the total defect area; S pi / S D is the path point p i Detection coverage; λ is the optimization weight coefficient of the shortest path, N(p i ) is the function for calculating the path length.

4. The industrial product intelligent detection method according to claim 1, characterized in that: In step 2), the optimization of shooting parameters includes the following steps: 221) For the same defect detection path point p* i Apply different shooting parameters to take pictures to obtain q images, and synthesize the q images into an HDR image by weighted averaging method or radiance map estimation method; Where j = 1...q, I HDR (x,y) is the pixel value at position (x,y) in the HDR image; w(L j (x,y)) is the weight function; L j (x,y) is the brightness value of position (x,y) in the jth image; t j is the exposure time of the jth image; 222) performing histogram analysis and threshold segmentation on images with different exposure values; Among them, H j (L) is the grayscale histogram of the jth image; I j (x, y) is the grayscale value of position (x, y) in the jth image; L is the gray level; When I j When (x,y)=L,δ(I j (x,y)-L) function value is 1, otherwise it is 0; 223) Calculate the matching degree; Among them, E j is the matching degree between the jth image and the HDR image, H HDR (L) is the grayscale histogram of the HDR image; min(H HDR (L),H j (L)) is to calculate the overlap of the two histograms at each gray level, which is used to measure the similarity between the jth image and the HDR image; 224) Draw the matching degree and grayscale curve of q images, fit the curve and find the peak to find the defect detection path point p* i The optimal exposure value at the current position is taken as the shooting parameter C* i .

5. The industrial product intelligent detection method according to claim 1, characterized in that: In step 3), the general defect detection algorithm includes the following steps: 31) In the defect detection task, the path point p* i And the corresponding shooting parameter C* i It is represented as a one-dimensional sequence vector At; the input one-dimensional sequence vector At is dimensionally expanded through a single linear layer to obtain an expanded one-dimensional vector A*t, whose dimension d is the same as the dimension d of the image feature vector f Consistency; 32) Obtain five consecutive images from the detection image set as image subsets, use the improved CSPDarknet network to extract features for each image in the image subset, and generate feature maps of three different scales, including small-scale feature maps, medium-scale feature maps, and large-scale feature maps; splice the five frames of feature maps of each scale in the channel dimension to form a high-dimensional feature map F S , whose dimension is five times that of the original single-frame feature map; 33) Compare the expanded one-dimensional vector A*t with the corresponding high-dimensional feature map F S Fusion is performed to form the fused feature representation F t ; F t Input to the encoder module of the Transformer network, the encoder module represents the fused feature F based on the self-attention mechanism t Processing to generate high-level feature representation E t , feature representation E t The dimension and feature representation F t The dimension is consistent with d f ; 34) Represent the feature E t The decoder module is input to the Transformer, and the decoder module decodes the feature representation into the detection result based on the sequence-to-sequence mapping; the output of the decoder module is a set of vectors representing the defect location and defect type, which are used to identify and classify the defects in the detection image.

6. A detection device for the industrial product intelligent detection method according to any one of claims 1 to 5, characterized in that: The invention comprises a testing platform, a multi-degree-of-freedom motion mechanism and a visual camera module arranged on the testing platform, wherein the multi-degree-of-freedom motion mechanism is provided with a clamp for clamping the tested product.

7. The detection device according to claim 6, characterized in that: The multi-degree-of-freedom motion mechanism is a multi-degree-of-freedom motion platform or a multi-axis manipulator.

8. The detection device according to claim 6, characterized in that: The visual camera module comprises an industrial camera, a halogen lamp light source and an LED parallel light source; the visual camera module is electrically connected to a programmable logic controller.

Images