Scan code detection method, storage medium, device and training method

Abstract

The present invention relates to a scanning code detection method, a storage medium, a device and a training method. The scanning code detection method includes performing operations such as feature extraction, dimension reduction processing, normalization mapping, weighting processing, layer-by-layer fusion processing and fusion result dimension reduction processing on image pixel feature values ​​in sequence, so that the characteristics of each pixel feature value are continuously enhanced, thereby being converted into detection result information that can directly determine whether a scanning code exists and the type of the scanning code, and can significantly improve the accuracy of scanning code detection and classification.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a scanning code detection method for detecting whether there is a scanning code in an image and what kind of scanning code exists, and also to a storage medium, device and training method corresponding to the scanning code detection method. Background Art

[0002] An express delivery bill refers to a document used by the express industry to record relevant information such as the sender, recipient, address, contact information, etc. during the delivery of goods. It is used to be affixed to the surface of the outer packaging of the goods for easy viewing or scanning and detection. At present, most express delivery bills used in the logistics and transportation industry have barcodes and / or QR codes. The waybill number can be obtained by scanning the barcode or QR code for decoding. The corresponding routing information can be queried in the background of the logistics system based on the waybill number. Among them, due to the different encoding methods of barcodes and QR codes, the decoding methods of the two are also different. When scanning, it is necessary to distinguish between the two and adopt the corresponding decoding method.

[0003] The logistics and transportation industry has increasingly higher requirements for timeliness, and needs to be able to automatically and quickly detect and decode the routing information of goods from the goods sorting line in order to distribute the goods to the destination in a timely and automatic manner. Therefore, the detection, positioning and differentiation of barcodes and QR codes on express delivery bills are crucial working links of the sorting system.

[0004] At present, the detection and positioning of barcodes and QR codes are mainly based on two types of algorithms. The first type of algorithm is based on traditional computer vision and image processing technology. The surface of the goods on the sorting line is photographed by an industrial camera, and then each area on the photographed image is scanned to locate the position of the barcode and QR code respectively; the second type of algorithm is based on deep neural network calculation, which performs target detection on the image taken by the industrial camera, calculates the pixel features of the image using a deep neural network, directly regresses the position of the barcode and QR code, and distinguishes the barcode and QR code at the same time.

[0005] However, both of the above algorithms have shortcomings in real-world applications. The first type of algorithm has great limitations in application. Once the express delivery bill on the goods is folded, blocked, shadowed, contaminated, or the lighting changes, the processing effect will be greatly reduced. In addition, since the image size taken by the industrial cameras currently used on the sorting line is large, it takes a long time to scan the image, which makes it difficult to meet the needs of quickly obtaining the waybill number and routing information. The second type of algorithm usually consists of multiple stages of calculations in application, and it is difficult to achieve a satisfactory balance between accuracy and performance. In addition, the express delivery bills of goods on the sorting line usually have a certain rotation angle relative to the horizontal placement, so this type of algorithm cannot be applied. Summary of the invention

[0006] The object of the present invention is to provide a scan code detection method capable of improving detection accuracy.

[0007] The present invention provides a scan code detection method, comprising the following steps:

[0008] Obtain an image feature matrix composed of the feature values ​​of each pixel in the image to be detected, and perform several levels of scanning code feature extraction on the image feature matrix, and obtain a set of convolution feature matrices corresponding to each level;

[0009] Perform dimensionality reduction and normalization mapping on each group of convolution feature matrices to obtain the corresponding groups of attention feature matrices;

[0010] Perform pixel-by-pixel feature value weighting processing on each group of attention feature matrices and their corresponding convolution feature matrices at the same level to obtain each group of weighted feature matrices;

[0011] The weighted feature matrix of the highest level is fused layer by layer to the weighted feature matrix of the lower level, and a feature fusion result matrix is ​​obtained;

[0012] The feature fusion result matrix is ​​subjected to dimension reduction processing to obtain scan code detection result information.

[0013] The scanning code detection method provided by the present invention continuously enhances the characteristics of the feature values ​​of each pixel point by successively performing feature extraction, dimensionality reduction processing, normalization mapping, weighting processing (attention enhancement), layer-by-layer fusion processing, and fusion result dimensionality reduction processing on the feature values ​​of image pixel points, thereby converting them into detection result information that can directly determine whether a scanning code exists and the type of scanning code.

[0014] As a preferred solution, the scan code is a barcode and / or a QR code.

[0015] As a preferred solution, the step of performing dimensionality reduction processing on each group of convolution feature matrices is specifically: performing dimensionality reduction processing by a 1×1 convolution kernel; and the step of normalization mapping is specifically performing normalization mapping by the following mapping function:

[0016]

[0017] Among them, x is the eigenvalue of each pixel point after the convolution feature matrix is ​​processed by the dimensionality reduction, and φ(x) is the value after the normalized mapping of the eigenvalue of the corresponding pixel point.

[0018] As a preferred solution, the step of layer-by-layer fusion processing includes:

[0019] The weighted feature matrix of the highest level is obtained by an image expansion algorithm to obtain the same size as the weighted feature matrix of the second highest level, and then merged with the weighted feature matrix of the second highest level to obtain a first feature merged matrix;

[0020] The first feature merging matrix is ​​subjected to dimensionality reduction processing by a plurality of 1×1 convolution kernels and then a first feature fusion matrix is ​​obtained by feature extraction;

[0021] The first feature fusion matrix is ​​obtained by an image expansion algorithm to have the same size as the weighted feature matrix of the third high-level, and then merged with the weighted feature matrix of the third high-level to obtain a second feature merging matrix;

[0022] The second feature merging matrix is ​​subjected to dimensionality reduction processing by a plurality of 1×1 convolution kernels and then a second feature fusion matrix is ​​obtained by feature extraction;

[0023] By analogy, the feature fusion matrix output at the lowest level is obtained as the feature fusion result matrix.

[0024] As a preferred solution, the step of reducing the dimension of the feature fusion result matrix to obtain the scan code detection result information specifically includes:

[0025] The feature fusion result matrix is ​​subjected to dimensionality reduction processing through 9 1×1 convolution kernels so that each pixel point obtains a vector information of dimension 9 to represent the detection result; the vector information includes predicted coordinates of dimension 4, predicted category probability of dimension 3, predicted rotation angle of dimension 1, and predicted result confidence of dimension 1.

[0026] Correspondingly, the present invention also provides a computer-readable storage medium on which a computer program is stored. The computer program is loaded by a processor to execute the steps in any of the above scan code detection methods.

[0027] Accordingly, the present invention also provides a scan code detection device, comprising the following modules:

[0028] Backbone network module: used to obtain the image feature matrix composed of the feature values ​​of each pixel in the image to be detected, and perform several levels of scan code feature extraction on the image feature matrix, and obtain a set of convolution feature matrices corresponding to each level;

[0029] Attention enhancement module: used to perform dimensionality reduction and normalization mapping on each group of convolution feature matrices to obtain the corresponding groups of attention feature matrices; and to perform pixel-by-pixel feature value assignment processing on each group of attention feature matrices and their corresponding convolution feature matrices at the same level to obtain each group of weighted feature matrices;

[0030] Feature pyramid module: used to perform layer-by-layer fusion processing from the highest-level weighted feature matrix to the lower-level weighted feature matrix to obtain the feature fusion result matrix;

[0031] Network output layer module: used to reduce the dimension of the feature fusion result matrix to obtain scan code detection result information.

[0032] As a preferred solution, the attention enhancement module performs dimensionality reduction processing on each group of convolution feature matrices through a 1×1 convolution kernel; and normalizes and maps each group of convolution feature matrices through the following mapping function:

[0033]

[0034] Among them, x is the eigenvalue of each pixel after the convolution feature matrix is ​​processed by dimensionality reduction, and φ(x) is the value after the normalized mapping of the eigenvalue of the corresponding pixel.

[0035] As a preferred solution, the scanned codes detected include barcodes and QR codes;

[0036] The network output layer module is specifically used to: reduce the dimension of the feature fusion result matrix through 9 1×1 convolution kernels so that each pixel point obtains a vector information of dimension 9 to represent the detection result; the vector information includes the following information of the pixel point: the distance to the upper boundary of the scan code, the distance to the right boundary of the scan code, the distance to the lower boundary of the scan code, the distance to the left boundary of the scan code, the rotation angle, the prediction probability of the category to which it belongs, and the confidence of the prediction result.

[0037] Accordingly, the present invention also provides a training method based on the above-mentioned scan code detection device, comprising the following steps:

[0038] Compare the scan code detection result information output by the network output layer module with the real scan code information to obtain the loss value;

[0039] Based on the following loss value function, the algorithms of the backbone network module, the attention enhancement module, the feature pyramid module, and the network output layer module are adjusted accordingly until the total loss value reaches a preset threshold:

[0040] L total =αL s +βL c +γL g +μL θ

[0041] Among them, L total is the total loss value function, L s Prediction result confidence loss function, L c is the prediction probability loss function of the category, L gis the prediction coordinate loss function, L θ is the prediction angle loss function; α, β, γ and μ are L s , L c , L g The adjustable weight coefficient is used to control the weight of the corresponding loss function;

[0042] in, y i Is the logical value of whether the scan code exists in the real scan code information; p i is the probability that the scan code exists in the scan code detection result information;

[0043] in, y i,c is the logical value of the corresponding category in the real scan code information; p i,c is the probability of belonging to the corresponding category in the scan code detection result information;

[0044] in GIoU is the difference between the predicted coordinates in the scan code detection result information and the coordinates in the actual scan code information. and They respectively represent the scan code area in the scan code detection result information, the scan code area in the real scan code information, and the minimum closure area that contains both the predicted and real scan codes;

[0045] in, θ i P and θ i T They respectively represent the angle in the detection result information and the angle in the actual scan code information.

[0046] The beneficial effects of the present invention include at least:

[0047] (1) Through a series of processes such as attention increase and layer-by-layer fusion, the pixel feature values ​​are converted into detection result information that can directly determine whether there is a scan code and the type of scan code, significantly improving the detection and classification accuracy of scan codes, especially barcodes and QR codes;

[0048] (2) The same applies to the situation where the express delivery bill of goods on the sorting line has a rotation angle;

[0049] (3) It does not involve complicated intermediate processes and can achieve fast scan code detection and classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 It is a schematic diagram of the overall flow of the scan code detection method provided in Example 1;

[0052] Figure 2 is with Figure 1 One of the network structure schematics for use;

[0053] Figure 3 yes Figure 1 A sample schematic diagram of the convolution feature matrix obtained in step S1;

[0054] Figure 4 yes Figure 1 A schematic diagram of an operation in which step S2 obtains an attention feature matrix and step S3 obtains a weighted feature matrix;

[0055] Figure 5 yes Figure 4 A sample schematic diagram of obtaining an attention feature matrix and a weighted feature matrix;

[0056] Figure 6 yes Figure 1 A schematic diagram of an operation principle of obtaining a feature fusion result matrix in step S4;

[0057] Figure 7 This is a schematic diagram comparing the areas of the predicted and actual scan codes in the training method of the scan code detection device provided in the fifth embodiment. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0059] Embodiment 1

[0060] Please also read Figures 1 to 6 The scanning code detection method provided in the first embodiment takes the detection of barcode and QR code as an example, which mainly includes: Figure 1It should be noted that the contents of steps S1 to S5 are only a summary of the overall detection process. To make the following introduction more intuitive, the overall detection process can be specifically (but not limited to) as follows: Figure 2 The network structure shown in FIG. 1 is implemented by the backbone network module, the attention enhancement module, the feature pyramid module and the network output layer module.

[0061] S1. Obtain an image feature matrix composed of the feature values ​​of each pixel in the image to be detected, and perform several levels of scan code feature extraction, and obtain a set of convolution feature matrices corresponding to each level.

[0062] In this embodiment, the image to be detected refers to an image that needs to be used to detect whether it has a scan code; for example, images obtained by an industrial camera photographing the surface of goods (packages) on the automatic sorting line of a logistics transportation company need to be detected whether these images have a scan code and what kind of scan code they have.

[0063] Step S1 can extract the scan code features of the image using a filter with scan code features in the convolutional layer based on a well-trained convolutional neural network (CNN) model. For faster detection and easier understanding, Figure 2 The backbone network module uses a convolutional neural network with less complexity and only 4 levels. In practical applications, convolutional neural networks of any level can be used.

[0064] In the multi-level feature extraction algorithm of the convolutional neural network, the dimension of the output matrix of the high level is reduced relative to the dimension of the output matrix of the low level. In this embodiment, each time a level of scan code feature extraction is performed, the dimension of the obtained image feature matrix (convolution feature matrix) is reduced accordingly, and the corresponding image size is reduced accordingly. For example, Figure 2 In the example, the image size (height and width) is reduced by 2 times from C2 to C5 through feature extraction. Assuming that the dimension of the input image (RGB) is 8×8×3 (the image dimension in actual applications is higher, this example is only for the convenience of introduction and understanding), the dimension of the first set of convolution feature matrices C2 output at the lowest level is 4×4×2, and the dimension of the second set of convolution feature matrices C3 output at the second lowest level is 2×2×2, as shown in Figure 3 As shown, and so on.

[0065] according to Figure 2 The network structure can perform 4 levels of feature extraction from C2 to C5, and obtain 4 sets of convolution feature matrices.

[0066] S2. Perform dimensionality reduction and normalization mapping on each group of convolution feature matrices to obtain the corresponding groups of attention feature matrices.

[0067] As a preferred solution, the steps of performing dimensionality reduction processing on each group of convolution feature matrices in step S2 are specifically: dimensionality reduction processing through a 1×1 convolution kernel; during the training stage, the initial parameters of the 1×1 convolution kernel can all be configured to 1, which can be adjusted accordingly during the training process and applied to formal testing after becoming stable.

[0068] The normalization mapping step in step S2 can be specifically performed by the following mapping function:

[0069]

[0070] Among them, x is the eigenvalue of each pixel point after the convolution feature matrix is ​​processed by the dimensionality reduction, and φ(x) is the value after the normalized mapping of the eigenvalue of the corresponding pixel point.

[0071] Step S2 is as follows Figure 4 As shown in the figure, assuming that the dimension of the input image is h×w×c (h is the height, w is the width, and c is the number of channels), it is reduced in dimension by a 1×1 convolution kernel and normalized by a mapping function φ(x) to obtain an attention feature matrix with a dimension of h×w.

[0072] Assume that in step S1 Figure 3 The first set of convolution feature matrix C2 and the second set of convolution feature matrix C3 shown in FIG. 1 and the whole parameters of the 1×1 convolution kernel are 1, then based on step S2, the following can be obtained: Figure 5 The dimensionality reduction results and attention feature matrix are shown.

[0073] according to Figure 2 The network structure can obtain 4 sets of attention feature matrices.

[0074] S3. Perform pixel-by-pixel feature value assignment processing on each group of attention feature matrices and their corresponding convolution feature matrices at the same level to obtain each group of weighted feature matrices.

[0075] The attention feature matrix actually represents the importance of each pixel in the image. The processing of step S3 is as follows Figure 4 shown The essence of the function is to use the attention feature matrix to complete the weighting processing of the feature values ​​of the pixels at the corresponding positions, so that the entire network can better pay attention to the area where the barcode or QR code is located, and improve the detection and classification accuracy of barcodes and QR codes.

[0076] Assume that in step S2 Figure 5 The dimensionality reduction processing result and the attention feature matrix shown in step S3 can be obtained Figure 5 The weighted feature matrix shown. For example, Figure 5The output values ​​of the two channels in the upper right corner of the weighted feature matrix are 0.29 and 0.43, which are given by Figure 5 The weight 0.73 in the upper right corner of the attention feature matrix is ​​multiplied by Figure 3 The values ​​of the upper right corner of the first set of convolution feature matrix C2 are 0.4 and 0.6.

[0077] according to Figure 2 The network structure can obtain 4 sets of weighted feature matrices.

[0078] S4. Perform fusion processing from the weighted feature matrix of the highest level to the weighted feature matrix of the lower level in turn to obtain a feature fusion result matrix.

[0079] In convolutional neural networks, high-level feature matrices (such as Figure 2 The output matrix P5 in has a large receptive field and rich semantic features, so its features can better judge the category of the detected target; while the low-level feature matrix (such as Figure 2 The output matrix C2 in has a large resolution and can accurately locate the position of the barcode and the QR code. Therefore, this embodiment performs a layer-by-layer feature fusion process based on the feature pyramid structure, which can combine high-level rich semantic features and low-level spatial detail information to further improve the accuracy of detection and classification.

[0080] like Figure 2 For the network structure shown in the figure, the weighted feature matrix of the highest level is equal to the output matrix P5 of that level, and P4 to P2 are all obtained by fusion processing of the previous level in sequence, among which the output matrix P2 of the lowest level is the feature fusion result matrix.

[0081] As a preferred solution of the first embodiment, the step of layer-by-layer fusion processing in step S4 includes the following steps S41-S45:

[0082] S41, after the weighted feature matrix of the highest level is obtained to have the same size as the weighted feature matrix of the second highest level through an image expansion algorithm, it is merged with the weighted feature matrix of the second highest level to obtain a first feature merged matrix.

[0083] The weighted feature matrix P5 of the highest level (the weighted feature matrix of the highest level is equal to the output matrix P5 of this level, so P5 is used here to refer to it) and the weighted feature matrix C4 (the second highest level) of the third group of convolution feature matrices (hereinafter referred to as ) as an example, the size of P5 is equivalent to Therefore, P5 needs to be upsampled by 2 times to obtain the same Same size and then To merge, such as Figure 6 As shown (by Figure 2The network structure shown is Figure 6 where n is 2, 3 or 4).

[0084] S42, the first feature merging matrix is ​​subjected to dimensionality reduction processing by a plurality of 1×1 convolution kernels and feature extraction to obtain a first feature fusion matrix.

[0085] like Figure 6 As shown in the figure, the two feature matrices with dimensions of h×w×c are concatenated to obtain a feature merge matrix with dimensions of h×w×2c. Assume Figure 6 P5 and After concatenation and passing through 1×1 convolution kernel and 3×3 convolution kernel, we can get Figure 2 The output matrix P4 (first feature fusion matrix) in .

[0086] S43. After the first feature fusion matrix obtains the same size as the weighted feature matrix of the third high-level through an image expansion algorithm, it is merged with the weighted feature matrix of the third high-level to obtain a second feature merging matrix.

[0087] S44, the second feature merging matrix is ​​subjected to dimensionality reduction processing by a plurality of 1×1 convolution kernels and feature extraction to obtain a second feature fusion matrix.

[0088] S45, and so on, to obtain the feature fusion matrix output at the lowest level as the feature fusion result matrix.

[0089] Steps S43-S44 are similar to steps S41-S42, where the first feature fusion matrix P4 is combined with the third high-level weighted feature matrix After feature fusion processing, we can get Figure 2 The output matrix P3 (second feature fusion matrix) in.

[0090] The second feature fusion matrix P3 is combined with the weighted feature matrix of the fourth highest level (lowest level) After feature fusion processing, we can get Figure 2 The output matrix P2 (the third feature fusion matrix) in , at this time, the third feature fusion matrix P2 is used as the feature fusion result matrix.

[0091] S5. Reduce the dimension of the feature fusion result matrix to obtain scan code detection result information.

[0092] For a well-trained convolutional neural network model, its fully connected layer (FullyConnected) can output a multidimensional vector for representing various classification prediction probabilities according to the input matrix data. Therefore, in this embodiment, a scan code detection result information with prediction probabilities of three categories of barcode, QR code and background can be output. As a preferred solution of embodiment 1, in step S5, the feature fusion result matrix is ​​reduced in dimension, and the step of obtaining the scan code detection result information specifically includes:

[0093] The feature fusion result matrix is ​​processed by 9 1×1 convolution kernels to reduce the dimension, so that each pixel point obtains a vector information of dimension 9 to represent the detection result, that is, {d top ,d right ,d bottom ,d left ,θ,l1,l2,l3,score}; that is, the vector information includes the predicted coordinates {d top ,d right ,d bottom ,d left}, predicted category probabilities {l1,l2,l3} with a dimension of 3, predicted rotation angle θ with a dimension of 1, and predicted result confidence score with a dimension of 1.

[0094] The above predicted coordinates can be used to calculate the coordinates of the four corner points of the scanned code graph, where d top Indicates the distance from the pixel to the upper boundary of the scan code, d right Indicates the distance from the pixel to the right edge of the scan code, d bottom Indicates the distance from the pixel to the lower boundary of the scan code, d left Indicates the distance from the pixel to the left border of the scanned code. {l1,l2,l3} respectively represent the normalized probabilities of the scanned code being a barcode, a QR code, or a background. For example, when {l1,l2,l3}={0.2,0.7,0.1}, it means that the probability that the pixel belongs to a barcode is 0.2, the probability that it belongs to a QR code is 0.7, and the probability that it belongs to the image background is 0.1. Only results with a confidence score greater than a certain threshold are considered to be true and reliable.

[0095] Based on the above description, the scan code detection method provided in the first embodiment continuously enhances the characteristics of the feature values ​​of each pixel point by performing feature extraction, dimensionality reduction processing, normalization mapping, weighting processing (attention enhancement), layer-by-layer fusion processing, and fusion result dimensionality reduction processing on the feature values ​​of the image pixel points in sequence, and finally converts it into detection result information that can directly determine whether there is a scan code and the type of the scan code. Therefore, the first embodiment has at least the following beneficial effects:

[0096] (1) Through a series of processes such as attention increase and layer-by-layer fusion, the pixel feature values ​​are converted into detection result information that can directly determine whether there is a scan code and the type of scan code, significantly improving the detection and classification accuracy of scan codes, especially barcodes and QR codes;

[0097] (2) The same applies to the situation where the express delivery bill of goods on the sorting line has a rotation angle;

[0098] (3) It does not involve complicated intermediate processes and can achieve fast scan code detection and classification.

[0099] Embodiment 2

[0100] Based on the first embodiment, the second embodiment provides a scan code detection component, including:

[0101] One or more processors, memory, and one or more applications; wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the scan code detection method described in Embodiment 1. In this embodiment, the processor may be a single-core processor, a multi-core processor, or any processor that implements multiple nodes and / or multiple threads; the memory includes, but is not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0102] Embodiment 3

[0103] Based on the first embodiment, the third embodiment provides a computer-readable storage medium, on which a computer program is stored, and the computer program is loaded by the processor to execute the steps in the scan code detection method described in the first embodiment. In this embodiment, the computer-readable storage medium includes, but is not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0104] Embodiment 4

[0105] Based on the first embodiment, the fourth embodiment provides a scanning code detection device, which can be referred to in conjunction with Figure 2 , which includes a backbone network module, an attention enhancement module, a feature pyramid module, and a network output layer module; each module is specifically defined as follows:

[0106] Backbone network module: used to obtain the image feature matrix composed of the characteristic values ​​of each pixel in the image to be detected, and perform several levels of scanning code feature extraction on the image feature matrix, and obtain a set of convolution feature matrices corresponding to each level.

[0107] Attention enhancement module: used to perform dimensionality reduction and normalization mapping on each group of convolution feature matrices to obtain the corresponding groups of attention feature matrices; and to weight the pixel-by-pixel feature values ​​of each group of attention feature matrices and their corresponding convolution feature matrices at the same level to obtain each group of weighted feature matrices.

[0108] Feature pyramid module: used to fuse the weighted feature matrix from the highest level to the weighted feature matrix of the lower level layer by layer to obtain the feature fusion result matrix.

[0109] Network output layer module: used to reduce the dimension of the feature fusion result matrix to obtain scan code detection result information.

[0110] In this embodiment, the attention enhancement module performs dimensionality reduction processing on each group of convolution feature matrices through a 1×1 convolution kernel; and normalizes and maps each group of convolution feature matrices through the following mapping function:

[0111]

[0112] Among them, x is the eigenvalue of each pixel after the convolution feature matrix is ​​processed by dimensionality reduction, and φ(x) is the value after the normalized mapping of the eigenvalue of the corresponding pixel.

[0113] As a preferred scheme of Example 4, the network output layer module is specifically used to: reduce the dimension of the feature fusion result matrix through 9 1×1 convolution kernels, so that each pixel point obtains a vector information of dimension 9 to represent the detection result; the vector information includes the following information of the pixel point: the distance to the upper boundary of the scan code, the distance to the right boundary of the scan code, the distance to the lower boundary of the scan code, the distance to the left boundary of the scan code, the rotation angle, the predicted probability of the category to which it belongs, and the confidence of the predicted result.

[0114] Embodiment 4 may correspond to executing any solution of Embodiment 1 and has the corresponding beneficial effects of Embodiment 1, which will not be described in detail here.

[0115] Embodiment 5

[0116] The scan code detection device of Example 4 is recommended to be put into practical application only after multiple trainings are carried out to stabilize the various algorithms involved. Therefore, the following preparations can be made during the first training: randomly select the express delivery bill images taken by the industrial camera from the sorting line, manually annotate the coordinates of the four corner points of the barcode and QR code in the image in a clockwise direction, and distinguish whether the annotated target is a barcode or a QR code. These manually annotated information are used as the real scan code information for comparison in training.

[0117] Therefore, based on the fourth embodiment, the fifth embodiment correspondingly provides a training method for a scan code detection device, comprising the following steps H1 to H2:

[0118] H1. Compare the scan code detection result information output by the network output layer module with the real scan code information to obtain the loss value;

[0119] H2. Based on the following loss value function, adjust the algorithms of the backbone network module, attention enhancement module, feature pyramid module, and network output layer module accordingly until the total loss value reaches the preset threshold:

[0120] L total =αL s +βL c +γL g +μL θ

[0121] Among them, L total is the total loss value function, L s Prediction result confidence loss function, L c is the predicted category probability loss function, L g is the prediction coordinate loss function, L θ is the loss function for predicting the rotation angle; α, β, γ and μ are L s , L c , L g The adjustable weight coefficient is used to control the weight of the corresponding loss function;

[0122] in, y i Is the logical value of whether there is a scan code in the real scan code information. If there is a scan code, y i The value is 1, otherwise it is 0; p i is the probability that the scan code exists in the scan code detection result information;

[0123] in, y i,c is the logical value of the corresponding category in the actual scan code information. If the category is a barcode, y i,1 is 1, y i,2 is 0, yi,3 is 0; if the category is a QR code, y i,1 is 0, y i,2 is 1, y i,3 is 0; if the category is image background, y i,1 is 0, y i,2 is 0, y i,3 is 1; p i,c is the probability of belonging to the corresponding category in the scan code detection result information;

[0124] in GIoU is the difference between the predicted coordinates in the scan code detection result information and the coordinates in the actual scan code information. and They represent the scan code area in the scan code detection result information, the scan code area in the real scan code information, and the minimum closure area that contains both the predicted and real scan codes. Figure 7 As shown;

[0125] in, θ i P and θ i T They respectively represent the angle in the detection result information and the angle in the actual scan code information.

[0126] The above is only a part of the present invention, the purpose of which is to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation of the present invention.

Claims

1. A scan code detection method, characterized in that: The following steps are involved: Obtain an image feature matrix composed of the feature values ​​of each pixel in the image to be detected, and perform several levels of scanning code feature extraction on the image feature matrix, and obtain a set of convolution feature matrices corresponding to each level; Perform dimensionality reduction and normalization mapping on each group of convolution feature matrices to obtain the corresponding groups of attention feature matrices; Each group of attention feature matrices and their corresponding convolution feature matrices at the same level are weighted by pixel-by-pixel feature values ​​to obtain each group of weighted feature matrices; The weighted feature matrix of the highest level is fused layer by layer to the weighted feature matrix of the lower level in turn, to obtain a feature fusion result matrix, and the layer-by-layer fusion process includes: The weighted feature matrix of the highest level is obtained by an image expansion algorithm to obtain the same size as the weighted feature matrix of the second highest level, and then merged with the weighted feature matrix of the second highest level to obtain a first feature merged matrix; The first feature merging matrix is ​​subjected to dimensionality reduction processing by a plurality of 1×1 convolution kernels and then a first feature fusion matrix is ​​obtained by feature extraction; The first feature fusion matrix is ​​obtained by an image expansion algorithm to have the same size as the weighted feature matrix of the third high-level, and then merged with the weighted feature matrix of the third high-level to obtain a second feature merging matrix; The second feature merging matrix is ​​subjected to dimensionality reduction processing by a plurality of 1×1 convolution kernels and then a second feature fusion matrix is ​​obtained by feature extraction; By analogy, the feature fusion matrix output at the lowest level is obtained as the feature fusion result matrix; The feature fusion result matrix is ​​subjected to dimensionality reduction processing through 9 1×1 convolution kernels so that each pixel point obtains a vector information of dimension 9 to represent the detection result; the vector information includes predicted coordinates of dimension 4, predicted category probability of dimension 3, predicted rotation angle of dimension 1, and predicted result confidence of dimension 1.

2. The scan code detection method according to claim 1, characterized in that: The scan code is a barcode and / or a QR code.

3. The scan code detection method according to claim 1, characterized in that: The step of performing dimensionality reduction processing on each group of convolution feature matrices is specifically: performing dimensionality reduction processing by a 1×1 convolution kernel; the step of normalization mapping is specifically performing normalization mapping by the following mapping function: Among them, x is the eigenvalue of each pixel after the convolution feature matrix is ​​processed by dimensionality reduction, and φ(x) is the value after the normalized mapping of the eigenvalue of the corresponding pixel.

4. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the scan code detection method according to any one of claims 1 to 3.

5. A scanning code detection device, characterized in that: Includes the following modules: Backbone network module: used to obtain the image feature matrix composed of the feature values ​​of each pixel in the image to be detected, and perform several levels of scan code feature extraction on the image feature matrix, and obtain a set of convolution feature matrices corresponding to each level; Attention enhancement module: used to perform dimensionality reduction and normalization mapping on each group of convolution feature matrices to obtain the corresponding attention feature matrices of each group; And weighting the pixel-by-pixel feature values ​​of each group of attention feature matrices and their corresponding convolution feature matrices at the same level to obtain each group of weighted feature matrices; Feature pyramid module: used for fusing the weighted feature matrix of the highest level to the weighted feature matrix of the lower level level in turn, to obtain a feature fusion result matrix, wherein the layer-by-layer fusion process includes: applying an image expansion algorithm to the weighted feature matrix of the highest level to obtain the same size as the weighted feature matrix of the second highest level, and then merging the weighted feature matrix of the second highest level to obtain a first feature merging matrix; applying a dimensionality reduction process to the first feature merging matrix through a plurality of 1×1 convolution kernels and extracting features to obtain a first feature fusion matrix; applying an image expansion algorithm to the first feature fusion matrix to obtain the same size as the weighted feature matrix of the third highest level, and then merging the weighted feature matrix of the third highest level to obtain a second feature merging matrix; applying a dimensionality reduction process to the second feature merging matrix through a plurality of 1×1 convolution kernels and extracting features to obtain a second feature fusion matrix; and so on, obtaining a feature fusion matrix output at the lowest level as a feature fusion result matrix; Network output layer module: used to reduce the dimension of the feature fusion result matrix through 9 1×1 convolution kernels so that each pixel point obtains a vector information of dimension 9 to represent the detection result; the vector information includes predicted coordinates of dimension 4, predicted category probability of dimension 3, predicted rotation angle of dimension 1 and predicted result confidence of dimension 1.

6. The scanning code detection device according to claim 5, characterized in that: The attention enhancement module performs dimensionality reduction processing on each group of convolution feature matrices through a 1×1 convolution kernel; and normalizes and maps each group of convolution feature matrices through the following mapping function: Among them, x is the eigenvalue of each pixel after the convolution feature matrix is ​​processed by dimensionality reduction, and φ(x) is the value after the normalized mapping of the eigenvalue of the corresponding pixel.

7. The scanning code detection device according to claim 6, characterized in that: The scanned codes detected include barcodes and QR codes; The network output layer module is specifically used to: reduce the dimension of the feature fusion result matrix through 9 1×1 convolution kernels so that each pixel point obtains a vector information of dimension 9 to represent the detection result; the vector information includes predicted coordinates of dimension 4, predicted category probability of dimension 3, predicted rotation angle of dimension 1 and predicted result confidence of dimension 1.

8. A training method for a scanning code detection device as claimed in claim 7, characterized in that: The following steps are involved: Compare the scan code detection result information output by the network output layer module with the real scan code information to obtain the loss value; Based on the following loss value function, the algorithms of the backbone network module, the attention enhancement module, the feature pyramid module, and the network output layer module are adjusted accordingly until the total loss value reaches a preset threshold: L total =αL s +βL c +γL g +μL θ Among them, L total is the total loss value function, L s Prediction result confidence loss function, L c is the predicted category probability loss function, L g is the prediction coordinate loss function, L θ is the loss function for predicting the rotation angle; α, β, γ and μ are L s , L c , L g The adjustable weight coefficient is used to control the weight of the corresponding loss function; in, y i Is the logical value of whether the scan code exists in the real scan code information; p i is the probability that the scan code exists in the scan code detection result information; in, y i,c is the logical value of the corresponding category in the real scan code information; p i,c is the probability of belonging to the corresponding category in the scan code detection result information; in GIoU is the difference between the predicted coordinates in the scan code detection result information and the coordinates in the actual scan code information. and They respectively represent the scan code area in the scan code detection result information, the scan code area in the real scan code information, and the minimum closure area that contains both the predicted and real scan codes; in, and They respectively represent the angle in the detection result information and the angle in the actual scan code information.

Images