Autonomous Multi-Angle Joint Detection Method for Non-Lambertian Appearance

Abstract

The present invention discloses an autonomous multi-angle joint detection method for non-Lambertian appearances, which relates to the field of automatic detection technology. In a fixed detection scenario, the appearance of a single product is detected, and a multi-view planning strategy is obtained through a data-driven artificial intelligence method. Through the learned view planning strategy and the sample set used for learning, the Top-k approximate neighbor samples of the new product image are queried, and the corresponding view planning strategy is obtained. The weights of the k retrieval view planning strategies are adjusted through the samples of the new product, and the multi-view planning strategy of any new product image is estimated by means of weighted summation. A deep reinforcement learning framework is adopted to continuously try and error to obtain the view planning strategy. A new few-shot learning method is used to achieve knowledge transfer: LSH is used to retrieve similar samples, and the planning strategy is corrected by re-weighting. Gradient compensation traces are used to cope with sparse rewards: the temporal features of the state are combined with the gradient compensation traces to accelerate the learning process.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic detection, and more specifically, to an autonomous multi-angle joint detection method for non-Lambertian appearances. Background Art

[0002] The appearance quality of a product is an important part of the product quality, which directly affects the consumer's senses and reduces its commercial value and brand reputation. Specifically, product appearance inspection includes stains, scratches, shallow pits, shallow tumors, edge defects, pattern defects, etc. Automated visual inspection (AVI) systems have the advantages of high detection accuracy, fast detection speed, and low cost, and have been widely applied. However, the products to be inspected are complex and variable, and it is difficult to design a general visual inspection system. Therefore, how to determine the image acquisition perspective and achieve knowledge transfer under different scenarios is the key to constructing a general AVI system. Summary of the Invention

[0003] The purpose of the present invention is to provide an autonomous multi-angle joint detection method for non-Lambertian appearances, which plans an acquisition perspective sequence for a product through deep reinforcement learning, and then realizes product quality detection; knowledge transfer between different products and different scenarios is achieved through transfer learning and noise-resistant learning, which helps to promote the automation process of product appearance detection.

[0004] The above technical purpose of the present invention is achieved through the following technical solutions: an autonomous multi-angle joint detection method for non-Lambertian appearances, including the following steps:

[0005] S1: Detect the appearance of a single product in a fixed detection scenario, and obtain a multi-perspective planning strategy through a data-driven artificial intelligence method;

[0006] S2: Query the Top-k approximate neighbor samples of the new product image through the learned perspective planning strategy and the sample set used for learning, and obtain the corresponding perspective planning strategy;

[0007] S3: Adjust the weights of the k retrieval perspective planning strategies through the samples of the new product, and estimate the multi-perspective planning strategy of any new product image in a weighted sum manner.

[0008] The present invention is further configured such that in step S1, the detection of the appearance of a single product is performed by using a method for detecting slight defects on a non-Lambertian surface based on deep reinforcement learning; automatic vision detection modeling is adopted, and a camera is used to capture images of the product to be inspected at any angle; subsequently, the captured images of the product to be inspected are preprocessed and input into a deep neural network model with convolutional layers; the offset angle by which the manipulator needs to rotate the product to be inspected is output; and then the manipulator is controlled to rotate the product and capture new images until it is determined whether there are appearance defects on the product.

[0009] The present invention is further configured such that the automatic vision detection is enhanced to a Markov decision process model:

[0010] M = <δ, A, R, P, γ>

[0011] Wherein, δ is the state space, that is, the sequence of images of the product that have been captured; A is the decision space, that is, the angle of rotation of the manipulator; R is the reward function, that is, the numerical goodness or badness of the current action, R: δ×A→R; P is the environmental dynamics, that is, the mapping from the current state to a new state after being changed by the action, P: δ×A→δ; γ∈[0, 1] is the attenuation coefficient, that is, the influence of the current action on future rewards gradually decreases with the time interval.

[0012] The present invention is further configured such that the deep neural network model adopts a combination of multiple convolutional-pooling layers and dense layers, the number of nodes in the output layer is |A|, and what is output is the value of each action; let the network model be f, its input is the state and the action, and the output is the action value, and the learning of the value function is regarded as a regression problem. Its objective function is the mean square error:

[0013]

[0014] s t represents the state at time t, s t+1 represents the state at the next moment at time t, a t represents the action at time t, r t represents the reward, θ t represents the current network parameters.

[0015] The present invention is further configured such that in steps S2 and S3, a method for cross-scenario policy correction of deep reinforcement learning based on transfer learning is adopted. When the appearance of the product changes and images of the new product are captured, first, similar product detection is realized through a hash group, and then the k product images retrieved are input into the existing model to obtain the corresponding planning strategy, and then the weights are adjusted through learning to estimate the policy to be corrected, so as to complete knowledge transfer.

[0016] The present invention is further configured to: use locality - sensitive hashing to assist in retrieval. Locality - sensitive hashing encourages collisions, such that through mapping, two similar points in the original space have similar hash values, and vice versa; the input of locality - sensitive hashing is the dense vector obtained by embedding the new product image X' into the original model through the convolutional - pooling layer; the output is a discrete hash value, and then k original product images x similar to the new product image's hash value are retrieved; the convolutional layer is CNN:R m →R n The locality - sensitive hashing is LSH:R n →П d ; The formal description of retrieving the top - k existing samples with high similarity through locality - sensitive hashing is:

[0017] I(X',K)=TopK x∈D (Sim(LSH(CNN(x)),LSH(CNN(X')))),

[0018] where Sim:Π d ×Π d →[0,1] is a similarity function.

[0019] The present invention is further configured to: learn and adjust weights. The weights are composed of k nodes, and a neural network layer with softmax normalization is used. A three - layer neural network is adopted, and no activation function is used to calculate the weights. The input is k similar perspective planning strategies, represented by I(x’,k)∈R |A|×k and the output is the weight y∈R k . If the weight learning network is g, then the objective function is:

[0020]

[0021] The present invention is further configured to: during the rotation of the manipulator, to avoid sampling the product at the same angle multiple times, the image sequence of the currently inspected product that has been collected is used as the state encoding. Its encoding method combines the product's historical images according to the maximum number of rotations as the input of the deep neural network or uses a neural network model such as a long - short - term memory network that can be used for time - series problems to encode the historical image sequence;

[0022] Its action encoding uses discrete encoding, that is, with a small angular offset as the interval, [0°, 360°] is evenly divided, plus two decision items of "qualified" and "defective"; its reward function uses sparse rewards. A reward of +1 is given only when the product is correctly detected as "qualified" or "defective"; when the product quality discrimination is incorrect, a reward of -1 is given; other action rewards are zero.

[0023] The present invention is further configured to: use a recurrent neural network (RNN) to encode the temporal features of the state, and its calculation formula is:

[0024] h t = sigmoid(m t w h + b h )

[0025] m t = tanh(x t w m1 + m t-1 w m2 + b m )

[0026] where x t and h t are the input and output in the recurrent neural network structure respectively, m0 is randomly assigned by initialization, w m1 , w m2 and w h are three matrices (dense layers), updated by backpropagation, sigmoid and tanh are activation functions used to normalize the output space.

[0027] The present invention is further configured to solve the problem of sparse rewards by using gradient compensation traces, and its formula is:

[0028]

[0029]

[0030] where θ t is the weight of each layer of the network at time t, e t is the key cumulative amount in the eligibility trace, α t is the learning rate, γ t is the discount factor; the temporal difference error is denoted as:

[0031] δ t = r t + γmax a f(s t+1 , a)-f(s t , a t )

[0032]

[0033] In summary, the present invention has the following beneficial effects: 1. In the work based on multiple perspectives, the perspective planning is designed for a specific product appearance and detection scenario, and the image capture angle is fixed. This application adopts dynamic multi-perspective planning and realizes the automatic planning of detection perspectives through a data-driven artificial intelligence method. 2. Since the detection performance will be significantly reduced after key factors affecting visual information collection, such as the shape of the product to be detected and the position of the light source in the detection scenario, change, this application makes it quickly adapt to the new product to be detected according to the existing knowledge through small-sample correction. 3. For the problem of sparse rewards, the gradient compensation trace is used for solution, and the learning process is accelerated by combining the temporal characteristics of the state and the gradient compensation trace to improve the learning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 are shallow pits and scratches on a non-Lambertian surface;

[0035] Figure 2 is the operation block diagram of a typical automatic visual detection system;

[0036] Figure 3 is the technical roadmap of the non-Lambertian surface slight defect detection based on deep reinforcement learning in the embodiment of the present invention;

[0037] Figure 4 is the technical roadmap of the cross-scene policy correction of deep reinforcement learning based on transfer learning in the embodiment of the present invention;

[0038] Figure 5 is the technical roadmap of the reward sparse environment learning acceleration method based on gradient compensation trace in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] The following further describes the present invention in detail Figures 1-5 in conjunction with the accompanying drawings.

[0040] Embodiment: An autonomous multi-angle joint detection method for non-Lambertian appearance, as Figures 1-5 shown, includes the following steps:

[0041] S1: Detect the appearance of a single product in a fixed detection scenario, and obtain a multi-perspective planning strategy through a data-driven artificial intelligence method;

[0042] S2: Query the Top-k approximate neighbor samples of the new product image through the learned perspective planning strategy and the sample set used for learning, and obtain the corresponding perspective planning strategy;

[0043] S3: Adjust the weights of the k retrieved perspective planning strategies through the samples of the new product, and estimate the multi-perspective planning strategy of any new product image by means of weighted sum.

[0044] To address the detection of the non-Lambertian appearance of smooth coatings, in the detection scenario with a fixed light source, the angles between the product to be detected, the light source, and the camera are adjusted by a robotic arm. On the one hand, make full use of the differences between the main image and the smooth surface such as shallow pits during reflection. On the other hand, strive to avoid the situation where the high-intensity reflection area covers fine scratches. At the same time, reduce the false detection problem caused by light source reflection by adjusting the relative angles. Reinforcement learning is a data-driven artificial intelligence method that can effectively handle sequential decision-making problems. At the same time, considering the problem of difficult image feature extraction, deep reinforcement learning is selected to control the angle sequence of the robotic arm rotation. The technical route is as Figure 3 shown.

[0045] The method for detecting minor defects on the non-Lambertian surface based on deep reinforcement learning is used to detect the appearance of a single product; automatic vision detection modeling is adopted, and the camera is used to capture images of the product to be detected at any angle; subsequently, the captured images of the product to be detected are preprocessed and input into a deep neural network model with convolutional layers; the offset angle by which the robotic arm needs to rotate the product to be detected is output; and then the robotic arm is controlled to rotate the product and capture new images until it is determined whether there are appearance defects on the product.

[0046] Before constructing the surface detection learning system, it is necessary to model automatic vision detection as a Markov decision process that can be processed by reinforcement learning:

[0047] M = <δ, A, R, P, γ>

[0048] where δ is the state space, that is, the sequence of images captured by the product; A is the decision space, that is, the angle of rotation of the robotic arm; R is the reward function, that is, the numerical goodness or badness of the current action, R: δ × A → R; P is the environmental dynamics, that is, the mapping from the current state to the new state after the action is changed, P: δ × A → δ; γ ∈ [0, 1] is the attenuation coefficient, that is, the influence of the current action on future rewards gradually decreases over time intervals.

[0049] During the rotation process of the robotic arm, to avoid sampling the product at the same angle multiple times, the sequence of images captured by the current product to be detected is used as the state encoding. The encoding method combines the historical images of the product as the input of the deep neural network according to the maximum number of rotations or uses neural network models such as long short-term memory networks that can be used for time series problems to achieve the encoding of the historical image sequence;

[0050] Its action encoding adopts discrete encoding, that is, [0°, 360°] is evenly divided at small offset angles, plus two decision items of "qualified" and "defective"; its reward function adopts sparse reward, and a reward of +1 is given only when the product is correctly detected as "qualified" or "defective"; when the product quality is misjudged, a reward of -1 is given; other action rewards are zero. To ensure the efficiency of the automatic visual inspection system and reduce the number of rotations of the manipulator during the inspection of a single product, the attenuation coefficient γ = 0.99 is proposed.

[0051] The deep neural network model adopts a combination of multiple convolutional-pooling layers and dense layers. The number of nodes in the output layer is |A|, and the value of each action is output; let the network model be f, its input is the state and action, and the output is the action value. The learning of the value function is regarded as a regression problem. Its objective function is the mean square error:

[0052]

[0053] s t represents the state at time t, s t+1 represents the state at the next moment at time t, a t represents the action at time t, r t represents the reward, θ t represents the current network parameters.

[0054] To address problems such as changes in light sources, imaging devices, and the appearance of products to be inspected in product inspection scenarios, and thus improve the transferability of the automatic visual inspection system, knowledge is quickly transferred through transfer learning. The technical route is as Figure 4 shown. When the appearance of the product changes, after capturing the image of the new product, first, similar product detection is achieved through the hash group, and then the k product images retrieved are input into the existing model to obtain the corresponding planning strategy, and then the weights are adjusted through learning for estimating the strategy to be corrected, thus completing knowledge transfer.

[0055] To quickly and accurately obtain historical images similar to the new product image, Locality-Sensitive Hashing (LSH) is used to assist in retrieval. Locality-Sensitive Hashing encourages collisions, and through mapping, two similar points in the original space have similar hash values, and vice versa. Therefore, it can be used to achieve large-scale information storage and retrieval. The input of Locality-Sensitive Hashing is the dense vector after embedding the new product image X' into the original model through the convolutional-pooling layer; the output is the discrete hash value, and then k original product images x similar to the new product image are retrieved according to the hash value of the new product image; the convolutional layer is CNN: R m →R n The Locality-Sensitive Hashing is LSH: R n →П d; Retrieving the top-k existing samples with high similarity through locality-sensitive hashing is formally described as:

[0056] I(X',K) = TopK x∈D (Sim(LSH(CNN(x)),LSH(CNN(X')))),

[0057] where Sim: Π d ×Π d →[0,1] is a similarity function.

[0058] Learning to adjust the weights, the weights are composed of k nodes, a neural network layer with softmax normalization, a three-layer neural network is used, and no activation function is used to calculate the weights. The input is k similar perspective planning strategies, represented by I(x’,k) ∈ R |A|×k and the output is the weight y ∈ R k . If the weight learning network is g, then the objective function is:

[0059]

[0060] To address the problem of slow learning caused by long detection sequences and scarce rewards at the initial stage of learning due to continuous rotation angles, gradient compensation traces are used to improve the backpropagation speed of rewards during learning. The technical route is as Figure 5 shown. First, the image sequence collected after the product rotates is encoded by a recurrent neural network. Subsequently, the state with temporal features is used as the input of a regression task and connected to the subsequent neural network serving as a value function. Finally, when updating the network weights, gradient compensation traces are used to replace the update gradient of the original optimizer to accelerate the update of the value function.

[0061] To reduce the situation where certain angles may be repeatedly visited only based on the current visual image, a recurrent neural network (RNN) is used to encode the temporal features of the state. That is, the image information observed in the past will exist in the encoding after the current image mapping in a certain form. Thus, repeated detections at the same angle are reduced. Compared with gated recurrent units and long short-term memory networks, the computational complexity of RNN is lower. In the case of small data volume, the output performance of RNN and its variant networks is similar. Its calculation formula is:

[0062] h t = sigmoid(m t w h +b h )

[0063] m t = tanh(x t w m1 +mt-1 w m2 +b m )

[0064] Among them, x t and h t are respectively the input and output in the recurrent neural network structure. m0 is randomly assigned by initialization. w m1 , w m2 and w h are three matrices (dense layers), which are updated by backpropagation. Sigmoid and tanh are activation functions used to normalize the output space. Its training value function network is the same as the method for constructing a deep network model.

[0065] The eligibility trace can be simply described as the accumulation of historical gradients. It changes the update rule so that the value of the historical state (action) can also be updated due to the current reward. The gradient compensation trace is a type of eligibility trace proposed in this application. Compared with the existing eligibility traces, the gradient compensation trace will not cause the influence of the reward on the state to decay or even disappear over time. Therefore, the gradient compensation trace is in line with the specific situation of this project and can be used to solve the problem of sparse rewards. To solve the problem of sparse rewards using the gradient compensation trace, its formula is:

[0066]

[0067]

[0068] Among them, θ t is the weight of each layer of the network at time t, e t is the key cumulative quantity in the eligibility trace, α t is the learning rate, γ t is the discount factor; the temporal difference error is denoted as:

[0069] δ t = r t + γ max a f(s t+1 , a)-f(s t , a t )

[0070]

[0071] This application uses a new visual detection method for adaptive perspective planning: adopting a deep reinforcement learning framework to continuously try and error to obtain the perspective planning strategy. It uses a new few-shot learning method to achieve knowledge transfer: adopting LSH to retrieve similar samples and correcting the planning strategy by reweighting. It uses the gradient compensation trace to deal with sparse rewards: combining the temporal features of the state with the gradient compensation trace to accelerate the learning process.

[0072] This specific embodiment is only an interpretation of the present invention and is not a limitation thereof. After reading this specification, those skilled in the art may make modifications to this embodiment without creative contributions as needed, but as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. An autonomous multi-angle joint detection method for non-Lambertian appearance, characterized in that: Including the following steps: S1: Detect the appearance of a single product in a fixed detection scenario, and obtain a multi-view planning strategy through a data-driven artificial intelligence method; S2: Query the Top-k approximate neighbor samples of the new product image through the learned view planning strategy and the sample set used for learning, and obtain the corresponding view planning strategy; S3: Adjust the weights of the k retrieval view planning strategies through the samples of the new product, and estimate the multi-view planning strategy of any new product image in a weighted sum manner; In step S1, the detection of the appearance of a single product is carried out by using a non-Lambertian surface slight defect detection method based on deep reinforcement learning; automatic vision detection modeling is adopted, and a camera is used to capture the product image to be detected at any angle; subsequently, the captured product image to be detected is preprocessed and input into a deep neural network model with a convolutional layer; the offset angle for the manipulator to rotate the product to be detected is output; furthermore, the manipulator is controlled to rotate the product and capture a new image until it is determined whether there are appearance defects on the product; Steps S2 and S3 adopt a cross-scenario strategy correction method of deep reinforcement learning based on transfer learning. When the product appearance changes and the image of the new product is captured, first, similar product detection is realized through a hash group, and then the k product images retrieved are input into the existing model to obtain the corresponding planning strategy, and then the weights are adjusted through learning for estimating the strategy to be corrected, so as to complete knowledge transfer.

2. The autonomous multi-angle joint detection method for non-Lambertian appearance according to claim 1, wherein: Reinforcing Automatic Visual Inspection into a Markov Decision Process Model: ; Among them, is the state space, that is, the sequence of images captured by the product; A is the decision space, that is, the angle of rotation of the manipulator; R is the reward function, that is, the numerical goodness or badness of the current action, ; P is the environmental dynamics, that is, the mapping from the current state to the new state after being changed by the action, ; is the attenuation coefficient, that is, the influence of the current action on future rewards gradually decreases with the time interval.

3. The autonomous multi-angle joint detection method for non-Lambertian appearance according to claim 1, characterized in that: The deep neural network model adopts a combination of multiple convolutional-pooling layers and dense layers. The number of nodes in the output layer is |A|, and the value of each action is output. Let the network model be , whose input is the state and action, and the output is the action value. The learning of the value function is regarded as a regression problem. Its objective function is the mean squared error: ； Represents the state at time t, Represents the state at the next moment of time t, Represents the action at time t, Represents the reward, Represents the current network parameters.

4. The autonomous multi-angle joint detection method for non-Lambertian appearance according to claim 3, characterized in that: Use locality-sensitive hashing to assist in retrieval. Locality-sensitive hashing encourages collisions, and through mapping, two similar points in the original space have similar hash values. The input of locality-sensitive hashing is the dense vector after the new product image is embedded through the convolutional-pooling layer in the original model. The output is a discrete hash value, and then k original product images x similar to the new product image are retrieved according to the hash value of the new product image. The convolutional layer is ; The locality-sensitive hashing is ; The formal description of retrieving the top-k existing samples with high similarity through locality-sensitive hashing is as follows: ； Among them, is the similarity function.

5. The autonomous multi-angle joint detection method for non-Lambertian appearance according to claim 3, characterized in that: In order to avoid sampling the product at the same angle multiple times during the rotation process of the manipulator, the image sequence of the current product to be detected is used as the state encoding, and its encoding method combines the historical images of the product according to the maximum rotation times as a neural network model that can be used for time series problems to realize the encoding of the historical image sequence; Its action encoding adopts discrete encoding, that is, it is evenly divided at intervals of small offset angles [0°, 360°], plus two decision items of "qualified" and "defective"; its reward function adopts sparse reward, and a reward of +1 will be given only when the product is correctly detected as "qualified" or "defective"; when the product quality discrimination is incorrect, a reward of -1 is given; other action rewards are zero.

6. The autonomous multi-angle joint detection method for non-Lambertian appearance according to claim 5, characterized in that: The state encoding adopts a recurrent neural network RNN to encode the time series characteristics of the state, and its calculation formula is: ； ； Among them, and are the input and output in the recurrent neural network structure respectively. is randomly assigned by initialization. , and are the three matrices of the dense layer, updated by backpropagation. Sigmoid and tanh are activation functions used to normalize the output space.

7. The autonomous multi-angle joint detection method for non-Lambertian appearance according to claim 6, characterized in that: The problem of sparse reward is solved by using gradient compensation traces.

Images