A quality monitoring method for industrial assembly lines based on image recognition

Abstract

The present invention discloses an industrial assembly line quality monitoring method based on image recognition, comprising the following steps: S1, installing a laser line scan camera and an industrial optical camera on the monitored assembly line; S2, simultaneously collecting a large amount of contour data A acquired by the laser line scan camera and image data B acquired by the industrial optical camera; S3, taking the maximum value of each frame of data in A as the volume measurement value of the spherical product, recorded as C; S4, according to the principle of time synchronization, splitting C and B into two parts respectively, C into C1 and C2, and B into B1 and B2; S5, using B1 and C1 as training sample data for training, and obtaining a model M, wherein the input of the model M is B1 and the output is C1. By constructing a neural network model, the image data acquired in real time by the industrial optical camera is replaced by the model M to complete the task of the laser line scan camera monitoring the upper limit of the volume of the spherical product on the assembly line, thereby realizing the replacement of the laser line scan camera with the industrial optical camera.

Description

Technical Field

[0001] The invention belongs to the technical field of industrial assembly line quality monitoring, and in particular relates to an industrial assembly line quality monitoring method based on image recognition. Background Art

[0002] At present, the technology of ordinary optical cameras is quite mature, with increasingly simple structures and relatively low costs, so the demand is huge; on the other hand, laser line scan cameras have complex structures and high costs, so the demand is relatively small. In the current industrial assembly line quality monitoring process, when monitoring the upper limit of the volume of spherical products on the assembly line, most of the solutions using laser line scan cameras to obtain contour data are affected by the aforementioned reasons, and this solution is expensive. Therefore, there is a need for a method of obtaining the volume of spherical products on an assembly line by using an industrial optical camera instead of a laser line scan camera, so as to help save the cost of laser line scan cameras in the above scenario. The method described in the present invention is intended to solve this problem. Summary of the invention

[0003] In view of the shortcomings of the prior art, the purpose of the present invention is to provide an industrial assembly line quality monitoring method based on image recognition, which solves the problems raised in the background technology.

[0004] The present invention provides the following technical solutions:

[0005] An industrial assembly line quality monitoring method based on image recognition comprises the following steps:

[0006] S1, installing a laser line scan camera and an industrial optical camera on the monitored production line, wherein the laser line scan camera and the industrial optical camera are used to collect images of the same monitoring point area;

[0007] S2, simultaneously collects a large amount of contour data A obtained by laser line scan cameras and image data B obtained by industrial optical cameras;

[0008] S3, take the maximum value of each frame of data in A as the volume measurement value of the spherical product, recorded as C;

[0009] S4, according to the principle of time synchronization, split C and B into two parts respectively, C into C1 and C2, and B into B1 and B2;

[0010] S5, training is performed using B1 and C1 as training sample data to obtain a model M, wherein the input of the model M is B1 and the output is C1.

[0011] Preferably, the method further comprises the step: S6, using B2 and C2 as test sample data to test the model M trained in step S5, to verify whether the deviation of the input B2 through the model M and the output C2 is less than a threshold value;

[0012] S7, remove the laser line scan camera, input the image data obtained by the optical camera in real time into the model M, and output the measurement value D of the spherical products on the monitored assembly line. If the deviation of the measurement value D is greater than the threshold, it is considered that the volume of the spherical products on the assembly line exceeds the limit.

[0013] Preferably, in step S2, the laser line scan camera and the industrial optical camera respectively transmit the captured images to the single chip microcomputer, and the single chip microcomputer respectively analyzes and processes the images.

[0014] Preferably, the process of image analysis and processing of the laser line scan camera by the single chip microcomputer includes the following steps: A, collecting a reference image of the monitoring point area when no spherical product passes by;

[0015] B. When the spherical product passes by, the image of the monitoring point area is collected in real time, and after the image of each frame is segmented and binarized with the best threshold, all the contour data of the spherical product obtained in each frame are sequentially stored in the corresponding array P1;

[0016] C. All the data in each array P1 are sorted from large to small using the quick sort method to obtain the sorted corresponding array P2;

[0017] D, select the first data Ck in each array P2 and store it in data set C.

[0018] Preferably, the single chip microcomputer obtains a data set Bk for each frame after performing necessary processing such as grayscale, rotation, and cropping on the image taken by the industrial optical camera, and multiple Bk constitute a data set B.

[0019] Preferably, the model M is a neural network model, and a training process of the neural network model includes the following steps: a. During forward propagation, multiple data values ​​of the data set Bk are sequentially input from multiple nodes of the input layer, processed layer by layer by the hidden layer, and the trained network connection weights are obtained, and the value Ch output by network calculation is obtained from the output end of the output layer;

[0020] b. Determine whether the difference between Ch and Ck corresponding to the same timestamp is less than the threshold. If so, it meets expectations and the training ends. Otherwise, the error back propagation process begins.

[0021] Preferably, a training process of the neural network model also includes the following steps: Step c, when entering the error back propagation process, the error between Ch and Ck corresponding to the same timestamp is back propagated from the network output layer to the input layer, and during the back propagation process, the connection weights of the neurons in each layer are modified.

[0022] Preferably, the laser line scan camera is installed in the monitoring point area of ​​the assembly line and measures vertically downward. Its detection laser beam is perpendicular to the movement direction of the assembly line track. The industrial optical camera and the laser line scan camera are installed in a similar position and take pictures vertically downward to obtain photo samples. The frame rate of the laser line scan camera is 20f / s, the frame rate of the industrial optical camera = 20f / s, and the resolution = 800x600.

[0023] Preferably, in step S4, data set C is randomly split into two parts, C1 and C2, where C1 is 80% of the data volume of C and C2 is 20% of the data volume of C. Similarly, B is split into B1 and B2. It is necessary to ensure that the timestamps of the C1 and B1 data sets are consistent, and the timestamps of the C2 and B2 data sets are consistent.

[0024] When the single chip microcomputer processes the image taken by the industrial optical camera, in order to achieve the optimal threshold value so as to increase the accuracy of data acquisition in the image, the image size of the monitoring point area is x*y, the gray value level is i, and the number of pixels corresponding to each gray value level i is A i , the segmentation threshold of the background and foreground (the spherical product area and the non-spherical product area on the image) is T, the proportion of the pixels in the foreground area to the entire image is recorded as Pq, the proportion of the pixels in the background area to the entire image is recorded as Pb, the average grayscale is recorded as μq and μb respectively, then Pq+Pb=1; the average grayscale of the image is recorded as μ, then Pb and Pq respectively satisfy the following relationship:

[0025] Pq=1-Pb;

[0026] According to the above formula, μb, μq and μ respectively satisfy the following relationships:

[0027]

[0028] According to the above formula, we can get the formula of inter-class variance g(k):

[0029] g(k)=Pb(μb-μ) 2 +Pq(μq-μ) 2 ;

[0030] In the above formula, the larger the g(k), the greater the difference between the background and the foreground. Therefore, the best threshold is when g(k) takes the maximum value.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] (1) The present invention discloses an industrial assembly line quality monitoring method based on image recognition. The method constructs a neural network model, uses the image data of a laser line scan camera as input, and uses the image data of an industrial optical camera as output to train and learn the model M. Thus, the image data collected in real time by the industrial optical camera can be replaced by the model M to complete the task of the laser line scan camera to monitor the volume of spherical products on the assembly line, thereby replacing the laser line scan camera with the industrial optical camera and saving the cost of the laser line scan camera.

[0033] (2) The present invention provides an industrial assembly line quality monitoring method based on image recognition. All data in each array P1 are sorted from large to small using a quick sort method to obtain the sorted corresponding arrays P2. The quick sort method has obvious advantages in its time complexity compared to other sorting methods, and has the advantages of fast sorting, etc., which can improve the accuracy of dynamic detection of the assembly line.

[0034] (3) The present invention provides an industrial assembly line quality monitoring method based on image recognition, by i The relationship between Pb, Pq, μb, μq and μ is limited, so that the microcontroller can reach the optimal threshold when processing the image taken by the industrial optical camera, thereby increasing the accuracy of data acquisition in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 It is a flow chart of the method of the present invention.

[0037] Figure 2 It is a sample image of a frame of data in the data set A of the present invention.

[0038] Figure 3 It is a sample image of a frame of data in data set B of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Embodiment 1:

[0042] See also Figure 1-3 , an industrial assembly line quality monitoring method based on image recognition, comprising the following steps:

[0043] S1, installing a laser line scan camera and an industrial optical camera on the monitored production line, wherein the laser line scan camera and the industrial optical camera are used to collect images of the same monitoring point area;

[0044] S2, simultaneously collects a large amount of contour data A obtained by laser line scan cameras and image data B obtained by industrial optical cameras;

[0045] S3, take the maximum value of each frame of data in A as the volume measurement value of the spherical product, recorded as C;

[0046] S4, according to the principle of time synchronization, split C and B into two parts respectively, C into C1 and C2, and B into B1 and B2;

[0047] S5, training is performed using B1 and C1 as training sample data to obtain a model M, wherein the input of the model M is B1 and the output is C1.

[0048] S6, using B2 and C2 as test sample data, testing the model M trained in step S5, and verifying whether the deviation of the input B2 through the model M and the output C2 is less than the threshold;

[0049] S7, remove the laser line scan camera, input the image data obtained by the optical camera in real time into the model M, and output the measurement value D of the spherical products on the monitored assembly line. If the deviation of the measurement value D is greater than the threshold, it is considered that the volume of the spherical products on the assembly line exceeds the limit.

[0050] In step S2, the laser line scan camera and the industrial optical camera transmit the captured images to the single chip microcomputer respectively, and the single chip microcomputer analyzes and processes the images respectively.

[0051] In step S4, the data set C is randomly split into two parts, C1 and C2. Preferably, C1 is 80% of the data volume of C, and C2 is 20% of the data volume of C. Similarly, B is split into B1 and B2, with B1 being 80% of the data volume of B and B2 being 20% ​​of the data volume of B. It is necessary to ensure that the timestamps of the C1 and B1 data sets are consistent, and the timestamps of the C2 and B2 data sets are consistent.

[0052] The process of image analysis and processing of the laser line scan camera by the single chip microcomputer includes the following steps: A, collecting a reference image of the monitoring point area when no spherical product passes by;

[0053] B. When the spherical product passes by, the image of the monitoring point area is collected in real time, and after the image of each frame is segmented and binarized with the best threshold, all the contour data of the spherical product obtained in each frame are sequentially stored in the corresponding array P1;

[0054] C. All the data in each array P1 are sorted from large to small using the quick sort method to obtain the sorted corresponding array P2;

[0055] D, select the first data Ck in each array P2 and store it in data set C.

[0056] By using the quick sort method to sort all the data in each array P1 from large to small, the corresponding arrays P2 are obtained after sorting. Compared with other sorting methods (bubble sort, insertion sort, selection sort, merge sort), the quick sort method has obvious advantages in its time complexity. It has the advantages of fast sorting and time saving, and can improve the accuracy of dynamic detection of the pipeline.

[0057] The single chip microcomputer obtains a data set Bk for each frame after performing necessary processing such as grayscale, rotation, and cropping on the image taken by the industrial optical camera. Multiple Bk constitute a data set B.

[0058] The model M is a neural network model, and a training process of the neural network model includes the following steps: a. During forward propagation, multiple data values ​​of the data set Bk are sequentially input from multiple nodes of the input layer, processed layer by layer by the hidden layer, and the trained network connection weights are obtained, and the value Ch output by network calculation is obtained from the output end of the output layer;

[0059] b. Determine whether the difference between Ch and Ck corresponding to the same timestamp is less than the threshold. If so, it meets expectations and the training ends. Otherwise, the error back propagation process begins.

[0060] The training process of the neural network model also includes the following steps: Step c, when entering the error back propagation process, the error between Ch and Ck corresponding to the same timestamp is back propagated from the network output layer to the input layer, and during the back propagation process, the connection weights of the neurons in each layer are modified.

[0061] The laser line scan camera is installed in the monitoring point area of ​​the assembly line and measures vertically downward. Its detection laser beam is perpendicular to the movement direction of the assembly line track. The industrial optical camera and the laser line scan camera are installed in a similar position and take pictures vertically downward to obtain photo samples. The frame rate of the laser line scan camera is 20f / s, the frame rate of the industrial optical camera = 20f / s, and the resolution = 800x600.

[0062] Embodiment 2:

[0063] On the basis of the first embodiment, when the single chip microcomputer processes the image taken by the industrial optical camera, in order to achieve the optimal threshold value so as to increase the accuracy of data acquisition in the image, the image size of the monitoring point area is x*y, the gray value level is i, and the number of pixels corresponding to each gray value level i is A i , the segmentation threshold of the background and foreground (the spherical product area and the non-spherical product area on the image) is T, the proportion of the pixels in the foreground area to the entire image is recorded as Pq, the proportion of the pixels in the background area to the entire image is recorded as Pb, the average grayscale is recorded as μq and μb respectively, then Pq+Pb=1; the average grayscale of the image is recorded as μ, then Pb and Pq respectively satisfy the following relationship:

[0064] Pq=1-Pb;

[0065] According to the above formula, μb, μq and μ respectively satisfy the following relationships:

[0066]

[0067] According to the above formula, we can get the formula of inter-class variance g(k):

[0068] g(k)=Pb(μb-μ) 2 +Pq(μq-μ) 2 ;

[0069] In the above formula, the larger the g(k), the greater the difference between the background and the foreground. Therefore, the best threshold is when g(k) takes the maximum value.

[0070] The method obtained by the above technical solution is an industrial assembly line quality monitoring method based on image recognition. This method constructs a neural network model, takes the maximum value of each frame of the image data of the laser line scan camera as the input value, and takes the image data of the industrial optical camera as the output value to train and learn the model M, so that the image data collected in real time by the industrial optical camera can be replaced by the model M to complete the task of the laser line scan camera to monitor the volume of spherical products on the assembly line, thereby replacing the laser line scan camera with the industrial optical camera and saving the cost of the laser line scan camera.

[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various changes and modifications. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An industrial assembly line quality monitoring method based on image recognition, characterized in that: The following steps are involved: S1, installing a laser line scan camera and an industrial optical camera on the monitored production line, wherein the laser line scan camera and the industrial optical camera are used to collect images of the same monitoring point area; S2, simultaneously collects a large amount of contour data A obtained by laser line scan cameras and image data B obtained by industrial optical cameras; S3, take the maximum value of each frame of data in A as the volume measurement value of the spherical product, recorded as C; S4, according to the principle of time synchronization, split C and B into two parts respectively, C into C1 and C2, and B into B1 and B2; S5, training with B1 and C1 as training sample data to obtain a model M, wherein the input of the model M is B1 and the output is C1; S6, using B2 and C2 as test sample data, testing the model M trained in step S5, and verifying whether the deviation of the input B2 through the model M and the output C2 is less than the threshold; S7, remove the laser line scan camera, input the image data obtained by the optical camera in real time into the model M, and output the measurement value D of the spherical products on the monitored assembly line. If the deviation of the measurement value D is greater than the threshold, it is considered that the volume of the spherical products on the assembly line exceeds the limit.

2. According to claim 1, an industrial assembly line quality monitoring method based on image recognition is characterized in that: In step S2, the laser line scan camera and the industrial optical camera transmit the captured images to the single chip microcomputer respectively, and the single chip microcomputer analyzes and processes the images respectively.

3. According to claim 2, an industrial assembly line quality monitoring method based on image recognition is characterized in that: The process of image analysis and processing of the laser line scan camera by the single chip microcomputer includes the following steps: A, collecting a reference image of the monitoring point area when no spherical product passes by; B. When the spherical product passes by, the image of the monitoring point area is collected in real time, and after the image of each frame is segmented and binarized with the best threshold, all the contour data of the spherical product obtained in each frame are sequentially stored in the corresponding array P1; C. All the data in each array P1 are sorted from large to small using the quick sort method to obtain the sorted corresponding array P2; D, select the first data Ck in each array P2 and store it in data set C.

4. According to claim 3, an industrial assembly line quality monitoring method based on image recognition is characterized in that: The single chip microcomputer processes the image taken by the industrial optical camera by grayscale, rotation and cropping to obtain a data set Bk for each frame, and multiple Bk constitute a data set B.

5. The industrial assembly line quality monitoring method based on image recognition according to claim 4 is characterized in that: The model M is a neural network model, and a training process of the neural network model includes the following steps: a. During forward propagation, multiple data values ​​of the data set Bk are sequentially input from multiple nodes of the input layer, processed layer by layer by the hidden layer, and the trained network connection weights are obtained, and the value Ch output by network calculation is obtained from the output end of the output layer; b. Determine whether the difference between Ch and Ck corresponding to the same timestamp is less than the threshold. If so, it meets expectations and the training ends. Otherwise, the error back propagation process begins.

6. The industrial assembly line quality monitoring method based on image recognition according to claim 5 is characterized in that: The training process of the neural network model also includes the following steps: Step c, when entering the error back propagation process, the error between Ch and Ck corresponding to the same timestamp is back propagated from the network output layer to the input layer, and during the back propagation process, the connection weights of the neurons in each layer are modified.

Images