A method for detecting fast moving packaging boxes on a production line

By applying a fast mobile packaging box detection method based on photography imaging and neural network on the production line, the problem of manual monitoring of packaging box status in traditional manufacturing is solved, and efficient and accurate detection with unmanned monitoring is achieved, reducing costs and increasing costs Automation level.

CN115035292BActive Publication Date: 2024-11-29QBOSSTECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210420396.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-21
Publication Date
2024-11-29
Estimated Expiration
2042-04-21

AI Technical Summary

Technical Problem

The lack of intelligence and automation in traditional manufacturing production lines has led to the need for manual real-time monitoring of packaging boxes, increasing labor costs and limiting rapid industrial iteration.

Method used

Using the optical principle of photography imaging and the fast moving packaging box detection method based on neural network calculation, the box state is judged and quality monitoring is completed through the new fast moving target automatic cutting algorithm.

Benefits of technology

It realizes unmanned packaging box status judgment, improves the accuracy and efficiency of detection, reduces labor costs, and adapts to image comparison in different environments, improving the automation level of the production line.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115035292B_ABST
    Figure CN115035292B_ABST
Patent Text Reader

Abstract

The invention provides a method for detecting fast-moving packaging boxes on a production line, and belongs to the technical field of assembly line packaging box detection. The method comprises the following steps: acquiring an image of a packaging box on a production line, dividing the image into N squares of the same size; taking one group of squares as i, and judging whether i≤N holds; constructing a plane equation of the i-th square, and randomly and uniformly selecting four points in a pixel area of ​​the i-th square, counting the number of points satisfying the plane equation, and judging whether the images in the square belong to the same object; taking the part with the largest object area in the N squares, and confirming it as a box image; dividing the box image into M convolution areas, taking one of the convolution areas as i, judging whether i≤M holds, calculating g corresponding to the i-th convolution area, and performing a convolution operation on the i-th convolution area; calculating the similarity between the box before and after the convolution and the original box image, and comparing the two groups of similarities to judge whether the box is normal or abnormal.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of assembly line packaging box detection, and in particular, relates to a method for detecting fast-moving packaging boxes on a production line. Background Art

[0002] Mobile crate monitoring refers to the technology of automatically detecting and tracking crates moving quickly on the production line and judging whether the crates meet the production quality requirements. With the rapid development of my country's economy and the substantial increase in national income, traditional manufacturing has rapidly transformed to Industry 4.0, which has put forward higher requirements for intelligent, digital and automated production levels. As of July 2021, there were a total of 167,000 medium and large production lines in my country, of which less than 19,000 were equipped with automatic crate appearance inspection. Most production lines require manual inspection and have not achieved intelligence and digitization, which limits the rapid iteration of the industry.

[0003] During the actual production process, the production line needs to be equipped with special quality inspectors to monitor the status of the box in real time. For example, they need to determine whether the box is damaged by the baler, whether the box is closed normally, whether the product inside the box is exposed, etc. Most production lines are closed production lines, which makes it impossible for quality inspectors to switch between the left and right sides of the production line. Therefore, this type of production line needs to be equipped with two quality inspectors, which greatly increases labor costs. Summary of the invention

[0004] An embodiment of the present invention provides a method for detecting fast-moving packaging boxes on a production line. Based on the optical principles of photographic imaging and the calculation process characteristics and structural characteristics of the neural network used in the subsequent image analysis, a new automatic cutout algorithm for fast-moving targets is proposed, and the cutout result is compared with a normal picture of the box to determine the state of the box and complete quality monitoring.

[0005] In view of the above problems, the technical solution proposed by the present invention is:

[0006] A method for detecting fast-moving packaging boxes on a production line comprises the following steps:

[0007] S1, obtain the image of the packaging box on the production line and convert the RGB information of each pixel in the image into brightness information L;

[0008] S2, divide the image into N squares of the same size, each square has a side length of a, and extend the image;

[0009] S3, taking one of the groups of squares as i, judging whether i≤N holds, if so, jumping to step S4, if not, jumping to step S6;

[0010] S4, construct the plane equation of the i-th square, obtain the plane normal vector of square i, and randomly and uniformly select four points in the i-th square pixel area, count the number of points that satisfy the plane equation, and determine whether the images in the square belong to the same object;

[0011] S5, sequentially select the squares where i=i+1, and execute the judgment of steps S3-S4;

[0012] S6, taking the part with the largest object area among the N squares and confirming it as the box image;

[0013] S7, divide the box image into M convolution regions, take one of the convolution regions as i, and determine whether i≤M is true. If so, jump to step S8, otherwise jump to step S10;

[0014] S8, calculating g corresponding to the i-th convolution region, and performing a convolution operation on the i-th convolution region;

[0015] S9, sequentially taking the convolution regions where i=i+1, and executing the judgments of steps S6-S7;

[0016] S10, calculating the similarity thr0 between the box after convolution and the original box image, calculating the similarity thr0' between the box before convolution and the original box image, and comparing the two sets of similarities to determine whether the box is normal or abnormal.

[0017] As a preferred technical solution of the present invention, the method of converting RGB information into brightness information L in step S1 is:

[0018]

[0019] in,

[0020]

[0021] As a preferred technical solution of the present invention, in step S2, the side length of the square is a=8 pixels.

[0022] As a preferred technical solution of the present invention, the step S4 further includes:

[0023] S41, taking the three vertices of the i-th square pixel area as p1, p2, and p3;

[0024] S42, constructing a coordinate system with the upper left corner of the image as the origin, the direction from left to right as the x-axis direction, the direction from top to bottom as the y-axis direction, and the direction vertically outward as the z-axis direction, the two-dimensional coordinate of each pixel point can be expressed as (px, py), and the brightness value L calculated in step S1 is used as the third dimension of the pixel point, then the three-dimensional coordinate of each pixel point can be expressed as (px, py, pz);

[0025] S43, determine the coordinates of the three vertices of the i-th square as (px1, py1, pz1), (px2, py2, pz2), (px3, py3, pz3) in sequence, and construct the plane equation

[0026]

[0027] S44, obtain the plane normal vector (A, B, C) of the i-th square according to the plane equation.

[0028] As a preferred technical solution of the present invention, in step S4, if three or more of the four randomly uniformly selected points satisfy the plane equation in step S43, it means that the i-th square pixel area belongs to the same object, otherwise it means that the area contains the boundary of the object.

[0029] As a preferred technical solution of the present invention, in step S7, the size of the convolution kernel is 3×3.

[0030] As a preferred technical solution of the present invention, it is characterized in that in step S8, the convolution kernel function

[0031]

[0032] For any convolution region reg i , construct the region matrix T with the brightness of each pixel l,i ,

[0033]

[0034] Matrix T k and T l,i The size is 3×3;

[0035] σ=r(T k ×T l,i )

[0036] Where r is the rank of the matrix, and σ is used to construct the normal distribution probability density function.

[0037]

[0038] Among them, e is a natural constant, and the determinant d is calculated. i =|T k ×T l,i |, construct the equation to be solved Get g, then use T k Perform a convolution operation on the image.

[0039] As a preferred technical solution of the present invention, in step S10, the calculation method of similarity is as follows:

[0040]

[0041] where X and Y are the matrices of two images respectively,

[0042] The similarity between the box image after convolution and the original box image is:

[0043]

[0044] where X B0 is the matrix of the current image after performing a convolution operation on the image using T k and Y 原始图像 is the matrix of the original box image after performing a convolution operation,

[0045] The similarity between the box image before convolution and the original box image is:

[0046]

[0047] where X' B0 is the matrix of the current image, and Y' 原始图像 is the matrix of the original box image,

[0048] If thr0 ≥ thr0', it indicates that the box is normal; if thr0 < thr0', it indicates that the box is abnormal.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] (1) This method is based on the optical principle of photographic imaging, constructs a luminance function according to the luminance distribution of the image, and realizes box matting by judging whether the luminance equation is differentiable;

[0051] (2) An adaptive kernel function is used to compare the box images to determine whether the box meets the requirements. The traditional comparison method first sets a threshold. If the similarity between the box and the standard box is greater than the threshold, it indicates that the box is qualified; otherwise, it indicates that the box is unqualified. However, the threshold is different in different environments, resulting in the need to continuously adjust the threshold according to the actual environment. This method uses an adaptive method, and its threshold is not fixed, but changes continuously according to the imaging effect of the current image, improving the accuracy of image comparison, saving labor, and reducing production costs.

[0052] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present invention more obvious and understandable, the following specifically illustrates the specific embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of a method for detecting fast-moving packaging boxes on a production line disclosed in the present invention;

[0054] Figure 2 The present invention is a schematic diagram of the working method of detecting fast-moving packaging boxes on a production line. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Example

[0057] An embodiment of the present invention provides a method for detecting a fast-moving packaging box on a production line, comprising the following steps:

[0058] S1, obtain the image of the packaging box on the production line, and convert the RGB information of each pixel in the image into brightness information L. The conversion formula is:

[0059]

[0060] in,

[0061]

[0062] S2, divide the image into N squares of the same size, with the side length of each square being a. The recognition effect is best when a=8 pixels, and extend the image. Suppose the storage format of a box image is x×y×3, which means that the photo has a total of x×y pixels.

[0063]

[0064] Among them, mod is the remainder function, and Floor is the floor function. According to the above formula, when mod[x,a]≠0, there exists At this time, the left side of the image needs to be extended, and the number of extended pixels is ex.

[0065]

[0066] Similarly, when mod[y,a]≠0, there exists At this time, the bottom edge of the image needs to be extended by the number of pixels ey.

[0067]

[0068] Because the extended pixels have no box information, for the convenience of calculation, the extended ex and ey pixels are uniformly specified as R=0, B=0, G=0;

[0069] S3, taking one of the groups of squares as i, judging whether i≤N holds, if so, jumping to step S4, if not, jumping to step S6;

[0070] S4, construct the plane equation of the i-th square, obtain the plane normal vector of square i, and randomly and uniformly select four points in the i-th square pixel area, count the number of points that satisfy the plane equation, and determine whether the images in the square belong to the same object;

[0071] In S41, each square pixel area contains four vertices. Since three points can determine a surface, any three of the four vertices can calculate the same plane equation. Therefore, the three vertices of the i-th square pixel area are p1, p2, and p3 respectively.

[0072] S42, constructing a coordinate system with the upper left corner of the image as the origin, the direction from left to right as the x-axis direction, the direction from top to bottom as the y-axis direction, and the direction vertically outward as the z-axis direction, the two-dimensional coordinate of each pixel point can be expressed as (px, py), and the brightness value L calculated in step S1 is used as the third dimension of the pixel point, then the three-dimensional coordinate of each pixel point can be expressed as (px, py, pz);

[0073] S43, determine the coordinates of the three vertices of the i-th square as (px1, py1, pz1), (px2, py2, pz2), (px3, py3, pz3) in sequence, and construct the plane equation

[0074]

[0075] S44, obtaining the plane normal vector (A, B, C) of the i-th square according to the plane equation;

[0076] In addition, if three or more of the four randomly uniformly selected points satisfy the plane equation in step S43, it means that the i-th square pixel area belongs to the same object, otherwise it means that the area contains the boundary of the object;

[0077] S5, sequentially select the squares where i=i+1, and execute the judgment of steps S3-S4;

[0078] S6, taking the part with the largest object area among the N squares and confirming it as the box image;

[0079] S7, divide the box image into M convolution regions, the size of the convolution kernel is 3×3, take one of the convolution regions as i, and determine whether i≤M is true. If so, jump to step S8, otherwise jump to step S10;

[0080] It should be noted that during the convolution process, 3×3 regions will be extracted from the box image matrix in turn and multiplied with the convolution kernel. This part is a general convolution method, so it will not be described here.

[0081] S8, calculating g corresponding to the i-th convolution region, and performing a convolution operation on the i-th convolution region;

[0082] Among them, the convolution kernel function

[0083]

[0084] For any convolution region reg i , construct the region matrix T with the brightness of each pixel l,i ,

[0085]

[0086] Matrix T k and T l,i The size is 3×3;

[0087] σ=r(T k ×T l,i )

[0088] Where r is the rank of the matrix, and σ is used to construct the normal distribution probability density function.

[0089]

[0090] Among them, e is a natural constant, and the determinant d is calculated. i =|T k ×T l,i |, construct the equation to be solved Get g, then use T k Perform convolution operation on the image;

[0091] S9, sequentially taking the convolution regions where i=i+1, and executing the judgments of steps S6-S7;

[0092] S10, calculating the similarity thr0 between the box after convolution and the original box image, calculating the similarity thr0' between the box before convolution and the original box image, and comparing the two sets of similarities to determine whether the box is normal or abnormal;

[0093] Among them, the calculation method of similarity is as follows:

[0094]

[0095] Among them, X and Y are the matrices of two images respectively,

[0096] The similarity between the box image after convolution and the original box image is:

[0097]

[0098] where X B0 is the matrix after the current image is convolved using T k and Y 原始图像 is the matrix after the original box image is convolved,

[0099] The similarity between the box image before convolution and the original box image is:

[0100]

[0101] where X' B0 is the matrix of the current image, and Y' 原始图像 is the matrix of the original box image,

[0102] If thr0 ≥ thr0', it indicates that the box is normal; if thr0 < thr0', it indicates that the box is abnormal.

[0103] The above is only the preferred embodiment of the present invention and is not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for detecting fast-moving packaging boxes on a production line, characterized in that: The following steps are involved: S1, obtain the image of the packaging box on the production line and convert the RGB information of each pixel in the image into brightness information L; S2, divide the image into N squares of the same size, each square has a side length of a, and extend the image; S3, taking one of the groups of squares as i, judging whether i≤N holds, if so, jumping to step S4, if not, jumping to step S6; S4, construct the plane equation of the i-th square, obtain the plane normal vector of square i, and randomly and uniformly select four points in the i-th square pixel area, count the number of points that satisfy the plane equation, and determine whether the images in the square belong to the same object; The step S4 further comprises: S41, taking the three vertices of the i-th square pixel area as p1, p2, and p3; S42, constructing a coordinate system with the upper left corner of the image as the origin, the direction from left to right as the x-axis direction, the direction from top to bottom as the y-axis direction, and the direction vertically outward as the z-axis direction, the two-dimensional coordinate of each pixel point can be expressed as (px, py), and the brightness value L calculated in step S1 is used as the third dimension of the pixel point, then the three-dimensional coordinate of each pixel point can be expressed as (px, py, pz); S43, determine the coordinates of the three vertices of the i-th square as (px1, py1, pz1), (px2, py2, pz2), (px3, py3, pz3) in sequence, and construct the plane equation S44, obtaining the plane normal vector (A, B, C) of the i-th square according to the plane equation; S5, sequentially select the squares where i=i+1, and execute the judgment of steps S3-S4; S6, taking the part with the largest object area among the N squares and confirming it as the box image; S7, divide the box image into M convolution regions, take one of the convolution regions as i, and determine whether i≤M is true. If so, jump to step S8, otherwise jump to step S10; In step S8, the convolution kernel function For any convolution region reg i , construct the region matrix T with the brightness of each pixel l,i , Matrix T k and T l,i The size is 3×3; σ=r(T k ×T l,i ) Where r is the rank of the matrix, and σ is used to construct the normal distribution probability density function. Among them, e is a natural constant, and the determinant d is calculated. i =|T k ×T l,i |, construct the equation to be solved Get g, then use T k Perform convolution operation on the image; S8, calculating g corresponding to the i-th convolution region, and performing a convolution operation on the i-th convolution region; S9, sequentially taking the convolution regions where i=i+1, and executing the judgments of steps S6-S7; S10, calculating the similarity thr0 between the box after convolution and the original box image, calculating the similarity thr0' between the box before convolution and the original box image, and comparing the two sets of similarities to determine whether the box is normal or abnormal.

2. A method for detecting fast-moving packaging boxes on a production line according to claim 1, characterized in that: The method of converting RGB information into brightness information L in step S1 is: in, 3. A method for detecting fast-moving packaging boxes on a production line according to claim 1, characterized in that: In step S2, the side length of the square is a=8 pixels.

4. A method for detecting fast-moving packaging boxes on a production line according to claim 1, characterized in that: In step S4, if three or more of the four randomly uniformly selected points satisfy the plane equation in step S43, it means that the i-th square pixel area belongs to the same object, otherwise it means that the area contains the boundary of the object.

5. A method for detecting fast-moving packaging boxes on a production line according to claim 1, characterized in that: In step S7, the size of the convolution kernel is 3×3.

6. A method for detecting fast-moving packaging boxes on a production line according to claim 1, characterized in that: In step S10, the similarity is calculated as follows: Among them, X and Y are the matrices of the two images respectively. The similarity between the box image after convolution and the original box image is: Where X B0 Use T for the current image k The matrix after the convolution operation on the image, Y 原始图像 is the matrix after the convolution operation on the original box image. The similarity between the box image before convolution and the original box image is as follows: where X' B0 is the matrix of the current image, Y' 原始图像 is the matrix of the original box image, If thr0 ≥ thr0', it indicates that the box is normal; if thr0 < thr0', it indicates that the box is abnormal.

Citation Information

Patent Citations

  • Nondestructive container detection method based on Compton gamma light-nuclear resonance fluorescence

    CN104198515A

  • Container surface image and container number acquisition system and device

    CN110443795A