Method and device for quickly positioning, enhancing and identifying two-dimensional code

Abstract

The invention discloses a method and a device for quickly positioning, enhancing and identifying a two-dimensional code, and the method comprises the steps: carrying out the graying and binaryzation of an original image collected from a camera, and carrying out the compression of rows and columns through a run length coding method, and storing the preprocessed image information. Pixel segments conforming to features are extracted from horizontal and vertical scanning compression information according to black and white pixel density features of a two-dimensional code, pixel segment information is converted into feature maps in a newly-built horizontal and vertical image, and an intersection of the two feature maps is taken. And obtaining connected domains from the intersection, traversing the connected domains, screening and rejecting the connected domains which do not conform to conditions, obtaining coordinate point information of the connected domains which conform to the conditions, and carrying out frame selection in the original image to obtain a coarse positioning frame of the two-dimensional code. And cutting the two-dimensional code image according to the two-dimensional code coarse positioning frame, putting the cut two-dimensional code image into the trained improved MIMO-Unet model for reasoning to obtain an enhanced two-dimensional code image, simultaneously calling three decoding libraries to decode the two-dimensional code image in a multi-thread mode, and judging and returning an identification result.

Description

Technical Field

[0001] The present invention relates to machine vision and deep learning technology, and in particular to a method and device for positioning and identifying a two-dimensional code. Background Art

[0002] We have entered the era of Industry 4.0, and traditional manufacturing is transforming towards digitalization and intelligence. With the continuous transmission of large amounts of information, the development of QR code technology has been brought about. The higher information capacity and ease of use of QR codes make them a more common information encoding method. QR code technology is applied to various scenarios of industrial production. Accurately identifying the information contained in the QR code is of great significance to the efficient collection, classification, and storage of commodity data, which brings certain requirements for efficiency and accuracy to the detection and recognition technology of QR codes.

[0003] The traditional method of identifying QR codes is to use a handheld QR code scanner or a smart phone to identify them, which has the disadvantages of slow speed, low efficiency, false detection and missed detection, and cannot be recognized in batches. The QR code detection and recognition technology based on machine vision has the advantages of high automation and high speed. Since the convolutional neural network was proposed, the target detection method based on deep learning can extract multi-level feature information from the image, and has developed from a two-stage detection algorithm to a single-stage detection algorithm and an anchor-free detection algorithm, which greatly improves the accuracy of target detection. At present, the target detection technology based on deep convolutional neural networks has made rapid progress in the field of QR code detection, but the complex model training process and high-performance hardware support make it difficult to meet the real-time requirements of QR code detection in industrial scenarios. In complex industrial scenarios, due to the image noise generated by industrial cameras in specific environments, the motion blur generated by objects carrying QR codes during rapid movement, the small proportion of QR codes to be identified in the image, the blurring of the details between QR codes, and other problems will hinder the accurate recognition of QR codes. Traditional image enhancement algorithms are based on mathematical models and prior knowledge. Although they can achieve significant results on specific images, they usually need to assume that the blur type is known. The blur types in actual scenes are complex and changeable. Traditional algorithms often cannot accurately estimate the blur kernel or blur model and are difficult to adapt to all situations.

[0004] In view of the above problems, how to quickly detect and locate QR codes and enhance QR code recognition is an issue that needs to be solved urgently. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method and device for quickly locating and enhancing the recognition of a two-dimensional code.

[0006] The first aspect of the present invention relates to a method for quickly locating and enhancing recognition of a two-dimensional code, comprising the following steps:

[0007] (1) Data preprocessing: Preprocess the data images collected by the camera, including image grayscale conversion, image binarization, and image data compression.

[0008] (1-1) Image grayscale conversion: Convert the collected color image into a grayscale image. The main part of the QR code consists mainly of black pixel blocks and white pixel blocks. Grayscale conversion can simplify the image matrix and improve the calculation efficiency.

[0009] (1-2) Image binarization: Further simplify the grayscale image and convert the grayscale image into a binary image using the Otsu method to eliminate the interference caused by complex backgrounds and image noise to the greatest extent.

[0010] (1-3) Image data compression: Since industrial images are relatively large, using the idea of run-length encoding to compress the pixel information of the binary image greatly shortens the time and facilitates subsequent feature extraction. Traverse each pixel in the binary image row by row, find the intersection points of the black and white pixel block transformations, and store the coordinates of these points in a two-dimensional container 1. Traverse each pixel in the binary image column by column and perform the same operation, and compress the image information into a two-dimensional container 2. The run-length encoding for each row or column can be summarized as:

[0011] T r =[L 1 ,L 2 ,...,L k

[0012] where L i represents the continuous pixel length of the i-th segment, and the black and white continuous pixel segments are arranged alternately.

[0013] (2) Detection and positioning: The alternation of black and white pixels in the binary image of the QR code area is the most frequent in the entire binary image, and the lengths of the continuous black pixels and the continuous white pixels in the binary image of the QR code area are close. According to this feature, scan horizontally from the two-dimensional container 1 to judge the values that meet the conditions, and convert their values into coordinates and store them in the candidate coordinate container 1. Since the position of the QR code in the image may not be straight and the angle will deviate in most cases, but whether scanning the QR code area horizontally or vertically, the lengths of the continuous black or white pixels in the binary image of the QR code area are close. Using this feature, then scan vertically from the two-dimensional container 2, convert the values that meet the conditions into coordinates and store them in the coordinate container 2, and take the intersection of the values in the coordinate container 1 and the coordinate container 2 in the image. This intersection part is the main part of the QR code. Finally, according to the information of the largest outer rectangle of the connected domain that meets the conditions, determine the approximate position of the QR code in the image to obtain the rough positioning image of the QR code.

[0014] ​(2-1) Feature extraction: Traverse the two-dimensional container 1 row by row, and scan horizontally to find adjacent and similarly colored pixels with similar lengths in each row. Record their starting and ending subscripts and convert them into the form of coordinate points and store them in the candidate coordinate container 1. Then traverse the two-dimensional container 2 column by column, and perform the same operation vertically to obtain the coordinate container 2.

[0015] points=(r,L i )

[0016] p={points[0],points[1],...,points[H-1]}

[0017] Among them, points are the endpoint coordinates of consecutive pixel segments with similar lengths, and p represents the set of all points satisfying this condition in each row.

[0018] (2-2) Obtaining the intersection: Create two new pure black images with the same size as the original captured image, one distributed horizontally and the other vertically. According to the coordinates stored in the coordinate container, use white line segments to connect the endpoint coordinates one by one on the horizontal and vertical new images respectively to form feature maps. Perform a bitwise AND operation on the pixels of the horizontal and vertical feature maps to obtain the intersection image.

[0019] (2-3) Rough positioning: Extract different connected components in the intersection image, traverse each connected component, and obtain the corner point coordinates, length, and width information of the largest circumscribed rectangle of each connected component. Filter out some non-QR code areas according to the aspect ratio of the length and width, and filter out some non-QR code areas with too large or too small areas according to the length of consecutive identical pixel segments. Select the candidate QR code area from the original image according to the corner point information.

[0020] (3) Image enhancement: Use the improved MIMO-Unet enhancement model to perform deblurring on the candidate QR code image, eliminate various blurs generated by the QR code in the scene, and improve the image clarity.

[0021] (3-1) Model improvement: Add the CBAM attention mechanism to the AFF asymmetric feature fusion module of the MIMO-Unet network.

[0022] (3-2) Deblurring: Use the trained improved MIMO-Unet model to perform inference deblurring on the candidate QR code image to obtain the deblurred QR code image.

[0023] (4) QR code recognition: Perform multi-threaded decoding on the image-enhanced QR code image to obtain unified decoding information and return the recognition result.

[0024] (4-1) Multi-threaded decoding: There are various methods for QR code decoding. In the present invention, a multi-threaded operation mode of calling three decoding libraries, namely zbar, opencv, and zxing, is adopted to decode the same QR code image.

[0025] (4-2) Decoded information processing: Compare the decoded information of the three decoding methods. If the decoded information is the same, return the information content; if different, return decoding failure.

[0026] The second aspect of the present invention relates to a device for quickly positioning and enhancing the recognition of QR codes, including a memory and one or more processors. Executable code is stored in the memory, and when the one or more processors execute the executable code, it is used to implement the method for quickly positioning and enhancing the recognition of QR codes of the present invention.

[0027] The third aspect of the present invention relates to a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, it implements the method for quickly positioning and enhancing the recognition of QR codes of the present invention.

[0028] The beneficial effects of the present invention are mainly manifested in:

[0029] 1. Compress the picture information through run-length encoding, and use a positioning algorithm that extracts features based on the black and white pixel density in the QR code area, which can quickly locate the QR code in the image under complex industrial scenarios.

[0030] 2. Combine the channel and spatial attention mechanisms to improve the MIMO-UNet model, learn the multi-scale blurred image features, make the model pay more attention to important features, improve the performance of de-blurring, realize the image enhancement of the blurred QR code, and facilitate the decoding of the QR code. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is the overall flow chart of the method of the present invention.

[0032] Figure 2 It is an example diagram of run-length encoding compression of image information of the present invention.

[0033] Figure 3 a)- Figure 3 c) are the horizontal and vertical feature maps and the intersection image of the present invention, where Figure 3 a) is the horizontal feature map, Figure 3 b) is the vertical feature map, - Figure 3 c) is the intersection of the horizontal and vertical feature maps.

[0034] Figure 4 It is the de-blurring process diagram of the present invention.

[0035] Figure 5 It is the multi-threaded decoding method diagram of the present invention.

[0036] Figure 6 It is a schematic diagram of the device of the present invention. Specific Embodiments

[0037] The present invention will be further described below with reference to the accompanying drawings.

[0038] The present invention relates to a method for quickly positioning and enhancing the recognition of two-dimensional codes. In a specific embodiment, three two-dimensional code images collected in an industrial scenario are taken as examples for illustration, and the present invention is elaborated in a more detailed manner in combination with the accompanying drawings and embodiments. It should be clear that the description of the embodiments is only for explaining the present invention and not for limiting the present invention.

[0039] Embodiment 1

[0040] Below in combination with the attached Figures 1 to 4 figure, this embodiment provides a method for quickly positioning and enhancing the recognition of two-dimensional codes, including the following steps:

[0041] (1) Data preprocessing: Perform image preprocessing on the data image collected by the camera, including image grayscale conversion, image binarization, and image data compression.

[0042] (1-1) Image grayscale conversion: Convert the collected color image into a grayscale image. The main part of the two-dimensional code is mainly composed of black pixel blocks and white pixel blocks. Grayscale conversion can simplify the image matrix and improve the calculation efficiency.

[0043] (1-2) Image binarization: Further simplify the grayscale image, and use the maximum between-class variance method to convert the grayscale image into a binary image, maximizing the elimination of interference caused by complex backgrounds and image noise.

[0044] (1-3) Image data compression: Since industrial images are relatively large, using the idea of run-length encoding, compressing the pixel information of the binarized image greatly shortens the time and facilitates subsequent feature extraction. Traverse each pixel in the binary image row by row, find the intersection points of the black and white pixel block transformations, and store the coordinates of these points in a two-dimensional container 1. Traverse each pixel in the binary image column by column, perform the same operation, and compress the image information into a two-dimensional container 2. The run-length encoding of each row or column can be summarized as:

[0045] T r =[L 1 ,L 2 ,...,L k

[0046] where L i represents the continuous pixel length of the i-th segment, and the black and white continuous pixel segments are arranged alternately.

[0047] ​(2) Detection and positioning: In the binary image of the QR code area, the alternation of black and white pixels in the part it occupies in the entire binary image is the most frequent, and the lengths of the continuous black pixels and the continuous white pixels in the binary image of the QR code area are close. According to this feature, scan horizontally from the two-dimensional container 1 to judge the values that meet the conditions, and convert their values into coordinates and store them in the candidate coordinate container 1. Since the position of the QR code in the image may not be correct and the angle will deviate in most cases, but whether scanning the QR code area horizontally or vertically, the lengths of the continuous black or white pixels in the binary image of the QR code area are close. Using this feature, then scan vertically from the two-dimensional container 2, convert the values that meet the conditions into coordinates and store them in the coordinate container 2, and take the intersection of the values in the coordinate container 1 and the coordinate container 2 in the image. This intersection part is the main body part of the QR code. Finally, according to the information of the largest circumscribed rectangle of the connected domain that meets the conditions, determine the approximate position of the QR code in the image to obtain the rough positioning image of the QR code.

[0048] (2-1) Feature extraction: Traverse the two-dimensional container 1 row by row, scan horizontally to find adjacent and similarly colored continuous pixels with close lengths in each row, record their starting subscripts and ending subscripts, and convert them into the form of coordinate points and store them in the candidate coordinate container 1. Then traverse the two-dimensional container 2 column by column, scan vertically and perform the same above operations to obtain the coordinate container 2.

[0049] points=(r,L i )

[0050] p={points[0],points[1],...,points[H-1]}

[0051] where points are the endpoint coordinates of the continuous pixel segments with close lengths, and p represents the set of all points that meet this condition in each row.

[0052] (2-2) Obtaining the intersection: Create two new pure black images with the same size as the original captured image, one distributed horizontally and one vertically. According to the coordinates stored in the coordinate container, connect the endpoint coordinates one by one with white line segments on the horizontal and vertical new images respectively to form feature maps. Perform a bitwise AND operation on the pixels of the horizontal and vertical feature maps to obtain the intersection image.

[0053] As Figure 3 shown, where Figure 3 a) is the horizontal feature map, Figure 3 b) is the vertical feature map, Figure 3 c) is the intersection of the horizontal and vertical feature maps.

[0054] (2-3) Coarse positioning: Extract different connected components from the intersection image, traverse each connected component, and obtain the corner coordinates, length, and width information of the maximum circumscribed rectangle of each connected component. Filter out some non-two-dimensional code regions based on the aspect ratio of the length to the width, and filter out some non-two-dimensional code regions with too large or too small areas based on the length of consecutive identical pixel segments. Select the candidate two-dimensional code regions from the original image according to the corner information.

[0055] (3) Image enhancement: Use the improved MIMO-Unet enhancement model to perform deblurring operations on the candidate two-dimensional code image, eliminate various blurs generated by the two-dimensional code in the scene, and improve the image clarity.

[0056] (3-1) Model improvement: Add the CBAM attention mechanism to the AFF asymmetric feature fusion module of the MIMO-Unet network.

[0057] (3-2) Deblurring: Perform inference deblurring on the candidate two-dimensional code image using the trained improved MIMO-Unet model to obtain the deblurred two-dimensional code image.

[0058] (4) Identify the two-dimensional code: Perform multi-threaded decoding on the image-enhanced two-dimensional code image to obtain unified decoding information and return the recognition result.

[0059] (4-1) Multi-threaded decoding: There are various two-dimensional code decoding methods. The present invention adopts the method of multi-threaded operation of calling three decoding libraries, namely zbar, opencv, and zxing, to decode the same two-dimensional code image.

[0060] (4-2) Decoding information processing: Compare the decoding information of the three decoding methods. If the decoding information is the same, return the information content; if it is different, return decoding failure.

[0061] Embodiment 2

[0062] Refer to Figure 6 , this embodiment relates to a device for quickly positioning and enhancing the recognition of two-dimensional codes, including a memory and one or more processors. Executable code is stored in the memory. When the one or more processors execute the executable code, it is used to implement the method for quickly positioning and enhancing the recognition of two-dimensional codes in Embodiment 1.

[0063] Embodiment 3

[0064] This embodiment relates to a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, it implements the method for quickly positioning and enhancing the recognition of two-dimensional codes in Embodiment 1.

[0065] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention.

Claims

1. A method for quickly locating and enhancing the recognition of a QR code, comprising the following steps: (1) Data preprocessing: The data images collected by the camera are preprocessed, including image grayscale, image binarization and image data compression; (2) Detection and positioning: The binary image of the two-dimensional code area has the most frequent alternation of black and white pixels in the entire binary image, and the length of continuous black pixels and continuous white pixels in the binary image of the two-dimensional code area are close. Based on this feature, a horizontal scan is performed from two-dimensional container 1 to determine the value that meets the conditions, and its value is converted into coordinates and stored in candidate coordinate container 1. Since the position of the two-dimensional code in the image may not be positive, the angle will be offset in most cases, but whether the two-dimensional code area is scanned horizontally or vertically, the length of continuous black or white pixels in the binary image of the two-dimensional code area is close. Using this feature, a vertical scan is performed from two-dimensional container 2, and the values ​​that meet the conditions are converted into coordinates and stored in coordinate container 2. The values ​​in coordinate container 1 and coordinate container 2 are intersected in the image, and the intersection part is the main part of the two-dimensional code. Finally, the approximate position of the two-dimensional code in the image is determined based on the maximum external rectangle information of the connected domain that meets the conditions, and a rough positioning image of the two-dimensional code is obtained. (3) Image enhancement: Use the improved MIMO-Unet enhancement model to deblur the candidate QR code image, eliminate the various blurs caused by the QR code in the scene, and improve the image clarity; (4) Recognize the QR code: Perform multi-thread decoding on the enhanced QR code image to obtain unified decoding information and return the recognition result.

2. A method for rapid positioning and enhanced recognition of a two-dimensional code as claimed in claim 1, characterized in that: Step (1) specifically includes: (1-1) Image grayscale: Convert the collected color image into a grayscale image. The main part of the QR code is mainly composed of black pixel blocks and white pixel blocks. Grayscale processing can simplify the image matrix and improve calculation efficiency. (1-2) Image binarization: further simplify the grayscale image and use the maximum inter-class variance method to convert the grayscale image into a binary image to eliminate the interference caused by complex background and image noise to the greatest extent; (1-3) Image data compression: Since industrial images are relatively large, the idea of ​​run-length encoding is used to compress the pixel information of the binary image, which greatly shortens the time and facilitates subsequent feature extraction; each pixel in the binary image is traversed row by row, the intersection point of the black and white pixel block transformation is found, and the coordinates of the point are stored in the two-dimensional container 1; each pixel in the binary image is traversed column by column, and the same operation is performed to compress the image information into the two-dimensional container 2; the run-length encoding of each row or column can be summarized as: T r =[L1,L2,...,L k ] Among them, L i It represents the length of continuous pixels in the i-th segment, and the black and white continuous pixel segments are arranged alternately.

3. A method for rapid positioning and enhanced recognition of a two-dimensional code as claimed in claim 1, characterized in that: Step (2) specifically includes: (2-1) Extracting features: Traverse the two-dimensional container 1 row by row, scan horizontally to find consecutive pixels of the same color that are adjacent and of similar length in each row, record their starting and ending subscripts, convert them into coordinate points and store them in candidate coordinate container 1; then traverse the two-dimensional container 2 column by column, scan vertically and perform the same operation as above to obtain coordinate container 2; points=(r,L i ) p={points[0],points[1],...,points[H-1]} Where points are the endpoint coordinates of continuous pixel segments with similar lengths, and p represents the set of all points that meet this condition in each row; (2-2) Obtaining the intersection: Create two new pure black images, which are equal in size to the original captured images, one distributed horizontally and the other distributed vertically. According to the coordinates stored in the coordinate container, use white line segments to connect the endpoint coordinates one by one on the horizontal and vertical new images to form a feature map; perform a bitwise AND operation on the pixels of the horizontal and vertical feature maps to obtain the intersection image; (2-3) Coarse positioning: extract different connected domains in the intersection image, traverse each connected domain, obtain the coordinates and length and width information of the corner points of the maximum circumscribed rectangle of each connected domain; filter out some non-QR code areas based on the ratio of length to width, and filter out some non-QR code areas that are too large or too small based on the length of continuous identical pixel segments; select candidate QR code areas from the original image based on the corner point information.

4. A method for rapid positioning and enhanced recognition of a two-dimensional code as claimed in claim 1, characterized in that: Step (3) specifically includes: (3-1) Model improvement: Add CBAM attention mechanism to the AFF asymmetric feature fusion module of the MIMO-Unet network; (3-2) Deblurring: The trained improved MIMO-Unet model is used to perform inference deblurring on the candidate QR code image to obtain the deblurred QR code image.

5. A method for rapid positioning and enhanced recognition of a two-dimensional code as claimed in claim 1, characterized in that: Step (4) specifically includes: (4-1) Multi-threaded decoding: There are many methods for QR code decoding. We use the multi-threaded operation of three decoding libraries, zbar, opencv and zxing, to decode the same QR code image. (4-2) Decoding information processing: The decoding information of the three decoding methods is compared. If the decoding information is the same, the information content is returned; if it is different, the decoding failure is returned.

6. A device for rapid positioning and enhanced recognition of two-dimensional codes, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the method for rapid positioning and enhanced recognition of two-dimensional codes as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the method for rapid positioning and enhanced recognition of a two-dimensional code as described in any one of claims 1-5 is implemented.

Images