Graphic code detection method, device and equipment based on improved Mama model

Abstract

The invention relates to the technical field of graphic codes, and discloses a graphic code detection method, device and equipment based on an improved Mama model, and the method comprises the steps: inputting a to-be-detected image into a backbone network in a target detection model, obtaining a plurality of first feature maps of different scales, and enabling the backbone network to be used for carrying out the feature extraction of the to-be-detected image, the backbone network comprises a randomized regularized deep convolutional neural network module and a visual clue fusion module which are cascaded; inputting the plurality of first feature maps into a path aggregation feature pyramid network in a target detection model to obtain a plurality of second feature maps of different scales, the path aggregation feature pyramid network being used for performing feature fusion processing on the plurality of first feature maps, and a convolutional layer in the path aggregation feature pyramid network being a deep separable convolutional layer; and inputting the plurality of second feature maps into a detection head in the target detection model to obtain position information of the graphic code in the to-be-detected image. According to the invention, the graphic code area in the image can be accurately identified.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the technical field of graphic code, in particular to a graphic code detection method, device and equipment based on an improved Mamba model. BACKGROUND

[0002] Artificial intelligence (AI) enables computers or machines to simulate, extend and expand human intelligence, allowing machines to have perception, knowledge acquisition and decision-making capabilities. Its research scope includes robotics, natural language processing, computer vision, etc. Deep learning technology, especially large models that have emerged in recent years, has become a key driving force for the development of artificial intelligence. Large models can mine complex patterns and correlations within big data, thereby significantly improving the ability of machines to process complex information and have been widely applied in the field of computer vision.

[0003] Target detection technology in computer vision is an application example of deep learning large models. Unlike simple image classification, target detection not only determines which object categories exist in an image, but also accurately locates their specific positions in the image. This task poses higher challenges to algorithms, and convolution algorithms, as the core component of deep learning, play a crucial role in it. By constructing a multi-layer convolutional neural network, the target detection model can automatically learn the hierarchical features of images from edges, textures to complex object shapes from big data, just like the human visual system. It is this powerful feature extraction capability that enables the model to achieve accurate recognition and positioning of objects in images.

[0004] One of the most representative application scenarios of target detection models is the detection of graphic codes (such as two-dimensional codes and bar codes). In modern commercial activities such as logistics and retail, these graphic codes are indispensable information carriers that carry the unique identification of goods. Using deep learning-based large models for target detection can efficiently identify the specific area where the graphic code is located from various complex lighting, angles and background interference.

[0005] However, despite the significant progress made by AI technology in target detection, in practical applications, how to use the target detection model to accurately identify the area where the graphic code is located from the image is still a key problem that needs continuous research and optimization. Especially in complex situations such as extremely small graphic code size, various poses or partial occlusion, the robustness, accuracy and speed of the model are all severely challenged.

[0006] In recent years, Mamba model has received extensive attention as a new type of deep learning model. Based on the theoretical framework of State Space Model (SSM), Mamba model can effectively model long-distance dependencies while maintaining linear time complexity by introducing the characteristics of SSM. This makes Mamba model have unique advantages in processing complex image sequences and long-distance context information. However, Mamba model is relatively weak in modeling local dependencies and cannot accurately detect small targets. SUMMARY

[0007] In view of the above problems, the embodiments of the present application provide a graphic code detection method and device based on an improved Mamba model, equipment and a storage medium, which are used to solve the problem that the prior art cannot accurately identify the region of the graphic code in the image.

[0008] According to an aspect of an embodiment of the present application, a graphic code detection method based on an improved Mamba model is provided, the method comprising: inputting a to-be-detected image into a backbone network in a target detection model to obtain a plurality of first feature maps of different scales output by the backbone network, wherein the backbone network is used for feature extraction of the to-be-detected image, and the backbone network comprises a cascaded randomized regularization deep convolutional neural network module and a visual clue fusion module; inputting a plurality of the first feature maps into a path aggregation feature pyramid network in the target detection model to obtain a plurality of second feature maps of different scales output by the path aggregation feature pyramid network, wherein the path aggregation feature pyramid network is used for feature fusion processing of a plurality of the first feature maps, and the convolutional layer in the path aggregation feature pyramid network is a depth separable convolutional layer; inputting a plurality of the second feature maps into a detection head in the target detection model to obtain a detection result output by the detection head, wherein the detection result comprises position information of a graphic code in the to-be-detected image.

[0009] In an optional manner, the backbone network comprises a down-sampling module, four backbone units and a spatial pyramid pooling fast module which are sequentially cascaded, each of the backbone units comprises a cascaded randomized regularization deep convolutional neural network module and a visual clue fusion module; the randomized regularization deep convolutional neural network module in the second backbone unit of the four backbone units is used to output a first feature map of a first scale to the path aggregation feature pyramid network; the randomized regularization deep convolutional neural network module in the third backbone unit of the four backbone units is used to output a first feature map of a second scale to the path aggregation feature pyramid network; and the spatial pyramid pooling fast module is used to output a first feature map of a third scale to the path aggregation feature pyramid network.

[0010] In an alternative manner, the randomized regularized deep convolutional neural network module comprises a cascaded feature channel split layer, a first convolutional layer, a first deep separable convolutional layer, a second convolutional layer, a first feature concatenation layer and a first channel random layer, wherein the feature channel split layer is configured to receive a feature map, split the received feature map into a first part feature map and a second part feature map in a channel dimension, output the first part feature map to the first convolutional layer, and output the second part feature map to the first feature concatenation layer; or the randomized regularized deep convolutional neural network module comprises a first branch, a second branch and a cascaded second feature concatenation layer and a second channel random layer, wherein the first branch comprises a cascaded second deep separable convolutional layer and a third convolutional layer, the second branch comprises a cascaded fourth convolutional layer, a third deep separable convolutional layer and a fifth convolutional layer, and the second feature concatenation layer is configured to receive feature maps output by the third convolutional layer and the fifth convolutional layer.

[0011] In an optional mode, the plurality of first feature maps output by the backbone network comprises first feature maps of a first scale, first feature maps of a second scale, and first feature maps of a third scale; the plurality of second feature maps output by the path aggregation feature pyramid network comprises second feature maps of the first scale, second feature maps of the second scale, and second feature maps of the third scale; the path aggregation feature pyramid network processes the plurality of first feature maps to obtain the plurality of second feature maps by: up-sampling the first feature maps of the third scale to obtain third feature maps of the second scale, fusing the first feature maps of the second scale and the third feature maps of the second scale to obtain first fused feature maps of the second scale; processing the first fused feature maps of the second scale by the optimization depth state space module to obtain fourth feature maps of the second scale, up-sampling the fourth feature maps of the second scale to obtain fifth feature maps of the first scale, fusing the first feature maps of the first scale and the fifth feature maps of the first scale to obtain second fused feature maps of the first scale, processing the second fused feature maps of the first scale by the optimization depth state space module to obtain the second feature maps of the first scale; down-sampling the second feature maps of the first scale by the depth separable convolution layer to obtain sixth feature maps of the second scale, fusing the fourth feature maps of the second scale and the sixth feature maps of the second scale to obtain third fused feature maps of the second scale, processing the third fused feature maps of the second scale by the depth separable convolution layer to obtain the second feature maps of the second scale; down-sampling the second feature maps of the second scale by the depth separable convolution layer to obtain seventh feature maps of the third scale, fusing the first feature maps of the third scale and the seventh feature maps of the third scale to obtain fourth fused feature maps of the third scale, processing the fourth fused feature maps of the third scale by the depth separable convolution layer to obtain the second feature maps of the third scale.

[0012] In an optional mode, the path aggregation feature pyramid network fuses two feature maps by one or more of the parallel guided selection scanning algorithm module, the serial guided selection scanning algorithm module, and the adaptive gated feature fusion module.

[0013] In an optional mode, the path aggregation feature pyramid network fuses two feature maps by the cascaded parallel guided selection scanning algorithm module, the serial guided selection scanning algorithm module, and the adaptive gated feature fusion module.

[0014] In an optional manner, the method further comprises: deleting redundant position information in a group of position information in the detection result by a non-maximum suppression algorithm to retain one position information in the group of position information, wherein the position information corresponding to the same graphic code in the detection result belongs to the group of position information.

[0015] According to another aspect of the embodiments of the present application, an improved Mamba model-based graphic code detection device is provided, comprising: a first processing module configured to input a to-be-detected image into a backbone network in a target detection model to obtain a plurality of first feature maps of different scales output by the backbone network, wherein the backbone network is configured to perform feature extraction on the to-be-detected image, and the backbone network comprises a cascaded randomized regularization deep convolutional neural network module and a visual clue fusion module; a second processing module configured to input the plurality of first feature maps into a path aggregation feature pyramid network in the target detection model to obtain a plurality of second feature maps of different scales output by the path aggregation feature pyramid network, wherein the path aggregation feature pyramid network is configured to perform feature fusion processing on the plurality of first feature maps, and a convolution layer in the path aggregation feature pyramid network is a depth separable convolution layer; and a third processing module configured to input the plurality of second feature maps into a detection head in the target detection model to obtain a detection result output by the detection head, wherein the detection result comprises position information of a graphic code in the to-be-detected image.

[0016] According to another aspect of the embodiments of the present application, an improved Mamba model-based graphic code detection device is provided, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the improved Mamba model-based graphic code detection method as described above.

[0017] According to yet another aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the improved Mamba model-based graphic code detection method as described above.

[0018] In the embodiments of the present application, the backbone network in the target detection model comprises a cascaded randomized regularization deep convolutional neural network module and a visual clue fusion module, and the visual clue fusion module is a module in the Mamba model. Therefore, the target detection model in the embodiments of the present application fuses the randomized regularization deep convolutional neural network module and the visual clue fusion module, so as to improve the Mamba model, and the improved Mamba model is called SFMamba. In the embodiments of the present application, the model is improved by the above-mentioned manner, which can reduce the complexity of the model structure, so that the improved model is more suitable for actual detection tasks.

[0019] In some embodiments, the path aggregation feature pyramid network in the target detection model comprises an optimized deep state space module, which belongs to the modules in the Mamba model, and the optimized deep state space module is a self-regressive model based on SSM, which can effectively capture global dependencies, and the optimized deep state space module comprises a local spatial feature extraction module and a residual lock module, wherein the local spatial feature extraction module extracts local spatial information by using multi-layer convolution and pooling operations, making up for the shortcomings of SSM in processing images; and the residual lock module combines high-dimensional representation and residual connection, effectively capturing local dependencies and further improving the performance and robustness of the model.

[0020] The above description is only a summary of the technical solutions of the embodiments of the present application. In order to more clearly understand the technical means of the embodiments of the present application, the embodiments of the present application can be implemented according to the content of the specification, and in order to make the above and other purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the specific embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings are only used to illustrate the embodiments and are not considered as limiting the present application. Moreover, the same reference signs are used to represent the same components throughout the drawings. In the drawings:

[0022] Figure 1 A structure schematic diagram of a target detection model provided by the embodiments of the present application is shown;

[0023] Figure 2 A flowchart of a graphical code detection method based on an improved Mamba model provided by the embodiments of the present application is shown;

[0024] Figure 3 A structure schematic diagram of a ShuffleBlock module provided by the embodiments of the present application is shown;

[0025] Figure 4 A structure schematic diagram of an ODSSBlock module provided by the embodiments of the present application is shown;

[0026] Figure 5 A structure schematic diagram of an LS Block module provided by the embodiments of the present application is shown;

[0027] Figure 6 A structure schematic diagram of an RG Block module provided by the embodiments of the present application is shown;

[0028] Figure 7 A structure schematic diagram of an SS2D module provided by the embodiments of the present application is shown;

[0029] Figure 8A schematic diagram of the structure of a graphic code detection device based on an improved Mamba model provided in an embodiment of the present application is shown;

[0030] Figure 9 A schematic structural diagram of a graphic code detection device based on an improved Mamba model provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0032] When using a camera to capture a graphic code on a product in order to decode the graphic code in the image, if the camera has a wide shooting range or the graphic code is far away from the camera, the graphic code area in the captured image will be smaller, that is, the proportion of the graphic code area to the total image area will be smaller.

[0033] For example, in modern industrial production, the use of conveyor lines (also known as assembly lines) is a key measure to improve production efficiency and reduce costs. Each product on the conveyor line is labeled with a graphic code, which is typically captured by a camera so that the graphic code in the captured image can be decoded. To improve decoding efficiency, the camera must simultaneously capture the graphic codes of multiple products. This requires a wide range of imaging, resulting in a smaller image area for each graphic code, even if the image contains multiple graphic codes.

[0034] For the above-mentioned image, since the graphic code area is small, it may not be possible to use conventional target detection methods to detect the area where the graphic code is located in the image. Or, if conventional target detection methods are used to detect the area where the graphic code is located in the image, missed detection or false detection may occur, resulting in inaccurate detection results.

[0035] Specifically, when the graphic code occupies a very small proportion of the image, the number of pixels corresponding to it is very limited, making it difficult for the object detection model to extract sufficiently stable and discriminative features to identify it. Object detection models generally perform better for large objects because large objects provide richer texture, shape, and contextual information. Therefore, small objects can easily be overlooked or misclassified during the detection process.

[0036] Moreover, if the image to be detected contains a large amount of non-target content (such as text, other objects, complex textures, etc.) in addition to the target (graphic code), these non-target contents will also be processed by the target detection model, causing the target detection model to fail to accurately focus on the true graphic code area, especially in cases where the graphic code itself is not sufficiently prominent (such as color, shape, and background difference is not large), background interference will be more serious.

[0037] Currently, deep learning techniques, especially target detection algorithms based on Convolutional Neural Network (CNN), such as the YOLO (You Only Look Once) series, are widely used in target detection tasks due to their efficiency and accuracy. The YOLO series algorithm can directly map the input image to the target detection result through end-to-end training, with the advantages of fast detection speed and strong real-time performance. However, traditional YOLO models still have some limitations in handling complex scenes. For example, when facing situations such as partial occlusion of target regions in the image to be detected, changes in lighting conditions such as strong light and shadows during image capture, resulting in reduced contrast of the image to be detected, and background content interference in the image to be detected, the detection accuracy of the model may be affected. In addition, the YOLO model has relatively weak ability in handling long-distance dependencies, which may lead to suboptimal detection performance in scenarios that require consideration of contextual information.

[0038] YOLO-OBB is a variant of the YOLO algorithm, specifically designed for handling target detection tasks of oriented bounding boxes (OBB), and can achieve multi-angle target detection in images to be detected. It improves the representation of bounding boxes based on traditional YOLO, which can more accurately describe the orientation and shape of target objects. However, YOLO-OBB still has limitations in handling very small targets. Due to its single-stage detection characteristics, it may be limited in feature extraction and positioning accuracy for small targets. In addition, YOLO-OBB may also have false positives or missed targets when dealing with complex scenes and background interference.

[0039] In recent years, Mamba model has received extensive attention as a new type of deep learning model. Based on the theoretical framework of State Space Model (SSM), Mamba model can effectively model long-distance dependencies while maintaining linear time complexity by introducing the characteristics of SSM. This makes Mamba model have unique advantages in processing complex image sequences and long-distance context information. However, Mamba model is relatively weak in modeling local dependencies, and its computational efficiency still needs to be further optimized to meet the real-time requirements in practical applications.

[0040] In view of the advantages and disadvantages of YOLO-OBB and Mamba algorithm, the two algorithms are fused to complement each other, so as to better solve the problem of small target detection. The advantages of YOLO-OBB in target detection framework and real-time performance can provide fast detection capability for small target detection; while the advantages of Mamba in processing complex dynamic behavior and long sequence data can enhance the feature extraction capability and robustness of the model for small targets. By fusing the two algorithms, their advantages can be fully utilized, their shortcomings can be overcome, and the accuracy and reliability of small target detection in the image to be detected can be improved.

[0041] Based on this, the present application provides a target detection model, Figure 1 The structure of the target detection model provided by the embodiment of the present application is shown. As shown in the figure, Figure 1 The target detection model includes a backbone network (Backbone), a path aggregation feature pyramid network (PAFPN) and a detection head (Head). The Backbone includes a down-sampling module (simplestem), four backbone units and a spatial pyramid pooling fast module (SSPF) in turn. Each backbone unit includes a Shuffle to Regularize Deep Convolutional Neural Networks (ShuffleBlock) and a Vision Clue Merge module. The PAFPN includes a feature fusion module (C), an up-sampling module (U), a depthwise separable convolution layer (Depthwise Separable Convolution, DWConvblock) and an optimized deep state space module (Optimized Deep State Space Block, ODSSBlock).

[0042] Figure 2A flowchart of a method for detecting a graphic code based on an improved Mamba model is shown. The method is executed by a terminal device, which can be a terminal device including one or more processors, such as a computer, a mobile phone, a tablet computer, or other electronic devices. The processor can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application, without limitation. The one or more processors included in the terminal device can be the same type of processors, such as one or more CPUs; or different types of processors, such as one or more CPUs and one or more ASICs, without limitation. As shown in Figure 2 the method includes the following steps:

[0043] Step 110: inputting the image to be detected into a backbone network in a target detection model to obtain a plurality of first feature maps of different scales output by the backbone network.

[0044] The terminal device can obtain an image to be detected captured by a camera from an external camera or a self camera. As shown in Figure 1 after the image to be detected is input into the backbone network in the target detection model, a first feature map P 11 of a first scale output by a ShuffleBlock module in a second backbone unit, a first feature map P 21 of a second scale output by a ShuffleBlock module in a third backbone unit, and a first feature map P 31 of a third scale output by an SSPF module in the backbone network are obtained. The first scale, the second scale, and the third scale can be set according to an actual scene. In this embodiment, the first scale is 80x80, the second scale is 40x40, and the third scale is 20x20.

[0045] Figure 3 A structure diagram of a ShuffleBlock module is shown. As shown in Figure 3 (a) and (b) in Figure 3 are structure diagrams of the ShuffleBlock module. (a) is a basic module without downsampling the input feature, and (b) is a module that downsamples the input feature by two times. For a small-sized image to be detected, Figure 1 the ShuffleBlock module shown in (a) can be used in the target detection model of Figure 1The ShuffleBlock module in the target detection model can use the ShuffleBlock module shown in structure (b). And the number of parameters of the ShuffleBlock module shown in (a) is less than that of the ShuffleBlock module shown in (b), so the efficiency of the ShuffleBlock module shown in (a) is higher than that of the ShuffleBlock module shown in (b), but the accuracy of the ShuffleBlock module shown in (b) is higher than that of the ShuffleBlock module shown in (a), so the ShuffleBlock module in the target detection model can be set to the structure shown in (a) or the structure shown in (b) as needed. Figure 1 The ShuffleBlock module in the target detection model can use the ShuffleBlock module shown in structure (b). And the number of parameters of the ShuffleBlock module shown in (a) is less than that of the ShuffleBlock module shown in (b), so the efficiency of the ShuffleBlock module shown in (a) is higher than that of the ShuffleBlock module shown in (b), but the accuracy of the ShuffleBlock module shown in (b) is higher than that of the ShuffleBlock module shown in (a), so the ShuffleBlock module in the target detection model can be set to the structure shown in (a) or the structure shown in (b) as needed.

[0046] Figure 3 In (a) in the above formula, the ShuffleBlock module includes a cascaded feature channel split layer (Channel Split), a first convolutional layer (1x1 Conv), a first depth separable convolutional layer (3x3 DWConv), a second convolutional layer (1x1 Conv), a first feature concatenation layer (Concat), and a first channel random layer (Channel Shuffle). Among them, the ChannelSplit layer receives the feature map, splits the received feature map into a first part feature map and a second part feature map in the channel dimension, and outputs the first part feature map to the first convolutional layer and the second part feature map to the first feature concatenation layer. The first part feature map is input to the first feature concatenation layer after being processed by the first convolutional layer, the first depth separable convolutional layer, and the second convolutional layer, then the first feature concatenation layer performs feature splicing processing on the two received feature maps, and finally the first channel random layer is used to shuffle the order of the feature channel dimension after splicing. For the Channel Shuffle operation, the input feature matrix is expanded into a four-dimensional matrix of gxnwxh, transposed along the output channel dimension g axis and the input dimension n axis, tiled according to the matrix g axis and n axis, and finally grouped and 1x1 convolution within the group to obtain a new output feature.

[0047] Figure 3In (b), the ShuffleBlock module includes a first branch (i.e., the branch on the left), a second branch (i.e., the branch on the right), and a cascaded second feature concatenation layer (Concat) and a second channel randomization layer (Channel Shuffle). The first branch includes a cascaded second depthwise separable convolution layer (3×3 DWConv) and a third convolution layer (1×1 Conv), with the stride of the second depthwise separable convolution layer being 2. The second branch includes a cascaded fourth convolution layer (1×1 Conv), a third depthwise separable convolution layer (3×3 DWConv), and a fifth convolution layer (1×1 Conv), with the stride of the third depthwise separable convolution layer being 2. After the third and fifth convolution layers output feature maps to the second feature concatenation layer, the second feature concatenation layer concatenates the received feature maps. The second channel randomization layer has the same function and principle as the first channel randomization layer, so the function and principle of the second channel randomization layer can refer to the function and principle of the first channel randomization layer in (a). It is worth noting that the strides of the three convolutional layers in the right branch of (b) are 1, 2, and 1, respectively, while the left branch has one less 1×1 convolutional layer. Note in particular that the output feature dimension of the 1×1 convolutional layer in the left branch is half the output dimension of the entire ShuffleBlock module, while the output dimension of the first 1×1 convolutional layer in the right branch is also half the output dimension of the entire ShuffleBlock module.

[0048] In the embodiment of the present application, by Figure 1 The ShuffleBlock module is used in the backbone unit of the provided target detection model. Since the number of parameters of the ShuffleBlock module is less than that of other modules (such as the ODSSBlock module), the feature extraction efficiency can be improved when the backbone network is used to extract features of the input image to be detected.

[0049] Step 120: Input the multiple first feature maps into a path aggregation feature pyramid network in the target detection model to obtain multiple second feature maps of different scales output by the path aggregation feature pyramid network.

[0050] like Figure 1 As shown, the first feature map P of the third scale 31 Input upsampling module (U) to upsample the first feature map P of the third scale 31 After upsampling, the third feature map P of the second scale output by the upsampling module is obtained 22 , the first feature map P of the second scale 21 and the third feature map P of the second scale 22 Input feature fusion module (C) to the first feature map P of the second scale21 and the third feature map P of the second scale 22 Perform feature fusion to obtain the first fusion feature map P of the second scale output by the feature fusion module 23 , the first fusion feature map P of the second scale 23 Input the ODSSBlock module to obtain the fourth feature map P of the second scale output by the ODSSBlock module 24 , the fourth feature map P of the second scale is processed by the upsampling module (U) 24 Upsample to obtain the fifth feature map P of the first scale 12 , the fifth feature map P of the first scale is fused by the feature fusion module (C) 12 and the first feature map P of the first scale 11 Perform feature fusion to obtain the second fusion feature map P of the first scale 13 , the second fusion feature map P of the first scale 13 Input the ODSSBlock module and obtain the second feature map P of the first scale output by the ODSSBlock module 14 .

[0051] The second feature map P of the first scale is processed by DWConvblock 14 Processing is performed to obtain the sixth feature map P of the second scale 25 , the sixth feature map P of the second scale is fused by the feature fusion module (C) 25 and the fourth feature map P of the second scale 24 Perform feature fusion to obtain the third fusion feature map P of the second scale 26 , the third fusion feature map P of the second scale 26 Input the ODSSBlock module and obtain the second feature map P of the second scale output by the ODSSBlock module 27 .

[0052] The second feature map P of the second scale is processed by DWConvblock 27 Processing is performed to obtain the seventh feature map P of the third scale 32 , the first feature map P of the third scale is fused by the feature fusion module (C) 31 and the seventh feature map P of the third scale 32 Perform feature fusion to obtain the seventh fusion feature map P of the third scale 33 , the seventh fusion feature map P of the third scale 33 Input the ODSSBlock module and obtain the second feature map P of the third scale output by the ODSSBlock module 34 .

[0053] Step 130: input the plurality of second feature maps into a detection head in the target detection model to obtain a detection result output by the detection head.

[0054] As shown in Figure 1 , the first scale second feature map P 14 , the second scale second feature map P 27 , and the third scale second feature map P 34 are input into the detection head, and the detection result (Output) output by the detection head can be obtained. Among them, the detection result includes the position information of the graphic code in the to-be-detected image.

[0055] In the embodiment of the application, the backbone network in the target detection model includes a cascaded randomized regularization deep convolutional neural network module and a visual clue fusion module, and the visual clue fusion module is a module in the Mamba model. Therefore, the target detection model in the embodiment of the application fuses the randomized regularization deep convolutional neural network module and the visual clue fusion module, so as to improve the Mamba model, and the improved Mamba model is called SFMamba. In the embodiment of the application, the model is improved by the above-mentioned method, which can reduce the complexity of the model structure, so that the improved model is more suitable for actual detection tasks.

[0056] Furthermore, Figure 1 The state space model (SSM) of the Mamba model is combined with the target detection framework of YOLO in the provided target detection model, and the ShuffleBlock and DwconvBlock modules are fused, and the improved Mamba model is called SFMamba. Since SSM can efficiently model long-distance dependencies with linear complexity, the SFMamba model makes up for the shortcomings of the traditional YOLO model in handling complex scenes, while retaining the efficient detection capability of YOLO. Moreover, the ShuffleBlock module and the DwconvBlock module have a low number of parameters, thereby reducing the number of parameters of the entire SFMamba model and speeding up the model inference speed. Therefore, the above-mentioned fusion method significantly improves the precision and efficiency of ultra-small code detection.

[0057] In order to further improve the detection precision, in the embodiment of the application, Figure 1 the PAFPN network in the PAFPN network fuses two feature maps through one or more of the parallel guided selection scanning algorithm (PGSS) module, the serial guided selection scanning algorithm module (SGSS), and the adaptive gating feature fusion module (AGFF). That is, the feature fusion module (C) in the PAFPN network is a module formed by cascading one or more of the parallel guided selection scanning algorithm module, the serial guided selection scanning algorithm module, and the adaptive gating feature fusion module.

[0058] The parallel guided selective scanning algorithm is a selective scanning method for cross-modal input sequences and Mamba hidden states. The purpose is to enhance the effect of feature fusion by processing multiple input sequences in parallel while utilizing the hidden states of the Mamba model. The processing flow of the parallel guided selective scanning algorithm is as follows: the feature maps of the input images (such as the feature maps obtained from the feature extraction layer of YOLO) and the hidden states of the Mamba model are taken as input sequences, then the input sequences are processed in parallel, each input sequence is processed by an independent scanning module, these scanning modules share weights but run independently, each scanning module selects key information in the input sequence according to the current hidden state, controls the flow of information through a gating mechanism, then the features after parallel processing are fused to obtain enhanced feature representations, and finally the hidden states of the Mamba model are updated according to the fused features.

[0059] The serial guided selective scanning algorithm is a method of sequentially processing input sequences, which enhances the effect of feature fusion by gradually utilizing the hidden states of the Mamba model. The processing flow of the serial guided selective scanning algorithm is as follows: the feature maps of the input images and the hidden states of the Mamba model are taken as input sequences, the input sequences are processed sequentially, the processing result of each input sequence will affect the processing of subsequent sequences, then at each time step, key information in the input sequence is dynamically selected according to the current hidden state, the flow of information is controlled through a gating mechanism, finally the features after sequential processing are fused to obtain enhanced feature representations, and the hidden states of the Mamba model are updated according to the fused features.

[0060] The adaptive gated multi-branch focus fusion module (AGFF) is a feature fusion method for optimizing the head part of YOLO. By controlling the flow of information, the efficiency of feature integration is improved. First, the feature maps output by the PAFPN module are taken as input; second, the adaptive gating mechanism is used to dynamically adjust the information flow between different feature maps, and the features are selectively fused according to the requirements of the current task; finally, the features adjusted by the gating mechanism are integrated to obtain the final detection features. Specifically, the AGMF realizes adaptive fusion of features through the following steps: first, feature input and branch processing, specifically, the input feature maps are sent to three branches for processing, and each branch outputs features of corresponding dimensions (such as spatial details, multi-scale information, and attention weights); second, dynamic weight generation, specifically, the gating unit generates weight coefficients for each branch according to the input features, and the weights are normalized by the Softmax function to ensure that the sum of the weights is 1; finally, weighted fusion, specifically, the features output by each branch are multiplied by the corresponding weights and then superimposed to form the final fused feature map.

[0061] In the embodiments of the present application, through the above feature fusion mechanism, the advantages of the Mamba model in long-distance dependency modeling and the YOLO model in real-time detection can be fully utilized, and the precision and efficiency of ultra-small code detection can be effectively improved.

[0062] In the embodiments of the present application, preferably, the feature fusion module (C) in the PAFPN network includes a cascaded parallel guided selection scanning algorithm module, a serial guided selection scanning algorithm module, and an adaptive gated feature fusion module. By using the parallel guided selection scanning algorithm and the serial guided selection scanning algorithm, selective scanning of the cross-modal input sequence and the Mamba hidden state is realized, and the effect of feature fusion is enhanced. At the same time, the adaptive gated feature fusion module is introduced to optimize the feature integration efficiency of the head part of YOLO and further improve the detection performance.

[0063] Figure 4 The structure diagram of the ODSSBlock module provided by the embodiments of the present application is shown. As shown in Figure 4 For the feature map input into the ODSSBlock module, the feature map is first subjected to 3x3 convolution (Conv2D) for spatial feature extraction, and then batch normalization (BN) operation is performed to stabilize the feature distribution and speed up the training convergence. The output feature dimension maintains the same number of channels (CxHxW) as the input. The batch normalization and the activation layer (i.e., the activation function in the figure) perform batch normalization (BN) operation on the feature map output by the convolution layer, and then use the GELU activation function to enhance the nonlinear expression capability. The output feature dimension is CxHxW (i.e., the spatial size remains unchanged).

[0064] In order to extract more local features when Mamba solves tasks with complex scale changes, the embodiment of the present application proposes to use a local spatial feature extraction (Local Spatial Block, LS Block) module to enhance the capture of local features, and to use layer normalization (Layer Norm) to process the feature map after LS Block extraction. The LS Block layer in the ODSSBlock module uses depthwise convolution (Depthwise Conv) to extract local features channel by channel, reducing the number of parameters while preserving spatial details. Specifically, it first performs 3x3 convolution channel by channel, and then fuses channel information through 1x1 convolution. The LS Block module, residual gated block (Residual Gated Block, RG Block), and two-dimensional selective scanning module (State Space Two Dimensional, SS2D) will be introduced in conjunction with the drawings below. Figure 4

[0065] Figure 5 The structure diagram of the LS Block module provided by the embodiment of the present application is shown. As shown in Figure 5 The LS Block module for a given input feature first performs depthwise convolution (DW-conv), which operates on each input channel separately, effectively extracting local spatial information from the input feature map while reducing computational cost and parameter quantity. Then, batch normalization (Batch Norm) is performed to provide a certain degree of regularization effect while reducing overfitting. In the depthwise convolution (Conv 2D), the channel information is mixed through 1x1 convolution, and the distribution of information is better preserved through the activation function (Activation), so that the model can learn more complex feature representations, thereby being able to extract rich multi-scale context information from the input feature map. In the LS Block, the activation function uses a non-linear GeLU, which changes the number of channels of the feature without changing the spatial dimension, thereby enhancing the feature representation. Finally, the original input and the processed feature are fused through residual concatenation, so that the model can understand and fuse different dimensional features in the image, thereby enhancing the robustness to scale changes.

[0066] Figure 6 The structure diagram of the RG Block module provided by the embodiment of the present application is shown. As shown in Figure 6 ​As shown, the depth separable convolution is used as the position encoding module on the X1-2 branch (the branch in the dashed line on the left in the figure), and the gradient is more effectively backflowed in the training process through the residual cascade manner, has a lower calculation cost, and significantly improves the performance by retaining and utilizing the spatial structure information of the image. The RG Block adopts a nonlinear GeLU as an activation function to control the flow of information in each layer, then merges with one branch of XI-2 through element multiplication, refines the global features through 1x1 convolution to fuse channel information, and finally adds the original input XI-2 and the features in the hidden layer through residual cascade. The RG Block can capture more global features while only slightly increasing the calculation cost.

[0067] The Multi-Layer Perceptron (MLP) in the Visual State Space Model (VMamba) architecture follows the Transformer design to perform nonlinear transformation on the input sequence to enhance the expression ability of the model. Among them, the SS2D module is proposed based on VMamba. Figure 7 The structure diagram of the SS2D module provided by the embodiment of the application is shown. As shown in Figure 7 The SS2D module includes a linear layer (Linear Layer), a depth separable convolution layer (DW-conv) fused with an activation function (Activation), and a selective scanning layer (Scan). SS2D is a convolution operation based on State Space Model. The operation expands the input feature map by scanning to obtain sequences in four directions, and then merges these sequences into global feature representation. The main role of SS2D is to expand the learnable range of the model without increasing the calculation amount, thereby improving the performance of the model.

[0068] In the embodiment of the application, by simplifying the network structure (such as the Simple Stem module) and optimizing the PAFPN structure (replacing the C2f module with the ODSS Block), the calculation complexity of the model is significantly reduced while maintaining the detection accuracy. In addition, the macro structure is designed to optimize the SSM and related modules to ensure the efficient operation of the model in real-time tasks, making it suitable for resource-constrained devices and scenarios.

[0069] In general, for Figure 1 The target detection model provided by the embodiment of the application, the input image is preliminarily down-sampled by the Simple Stem module, which adopts two convolution layers with a step size of 2 and a kernel size of 3 (non-traditional non-overlapping block), thereby retaining more spatial information. After down-sampling, a resolution of 1 / 8 of the original image is generated. Figure 1The feature map of / 4 (such as input 640x640 and output 160x160) is taken as the starting point of subsequent processing. The backbone network is constructed by fusing ShuffleBlock modules and Vision Clue Merge. After passing through two ShuffleBlock modules, the features are merged through the Vision Clue Merge module: the dimension mapping is split, the redundant features are attached to the channel dimension, and the key visual clues selected by SS2D are retained. Pointwise convolution with 4 times compression is used for downsampling, replacing the traditional 3x3 convolution with a step of 2, to reduce information loss. The end of the backbone network is connected to the SSPF module, which fuses features of different receptive fields through multi-scale pooling, and outputs multi-level feature maps of 20x20, 40x40, and 80x80 to the PAFPN.

[0070] The PAFPN adopts a top-down and bottom-up semantic transmission method. High-level features (such as 20x20) are aligned in size with low-level features (such as 40x40) through upsampling (bilinear interpolation or transpose convolution), and the superimposed features are fused through an upsampling module (1x1 convolution to adjust the number of channels) to integrate semantic information. Low-level features (such as 80x80) are aligned in size with middle-level features (such as 40x40) through a step-2 convolution downsampling module, and the traditional C2f module is replaced by an ODSSBlock to strengthen the gradient information flow.

[0071] By using an improved weighted bidirectional (top-down + bottom-up) feature pyramid network (BiFPN) structure, the contribution of different levels of features is dynamically adjusted through a weighted fusion mechanism to balance semantic and detail information. For example, 80x80 features are added to upsampled 40x40 features, and then a 3x3 convolution is used to eliminate aliasing effects. In the PAFPN, DWConvblock modules are used instead of ordinary convolution layers, which reduces the amount of calculation and the number of parameters while improving the efficiency and speed of the model.

[0072] The three-level feature maps output by the PAFPN are input into double ODSSBlock stacking layers to further extract task-related features. The first layer of ODSSBlock is used to enhance spatial information sensitive to positioning, and the second layer of ODSSBlock is used to extract semantic information related to classification.

[0073] The refined feature maps are output by the detection head parallel branch to predict the results. The classification branch is used to predict class probabilities through full convolution layers, and FocalLoss is used to alleviate class imbalance. The regression branch is used to predict the offset of the bounding box based on anchor or anchor-free mechanisms, and the CIoU loss is used to enhance the positioning accuracy.

[0074] In some embodiments, the detection result output by the detection head is also post-processed and optimized, specifically including that for a group of position information in the detection result corresponding to the position information of the same graphic code, redundant position information in each group of position information in the detection result is deleted by a non-maximum suppression algorithm to retain one position information in each group of position information. That is, the non-maximum suppression (NMS) method is applied to filter the overlapping detection boxes. And if the confidence of each detection box is also included in the detection result, the confidence threshold (such as 0.5) can also be combined to screen the final result, for example, the detection box with a confidence greater than the confidence threshold is taken as the final detection result.

[0075] Figure 8 The structure schematic diagram of the graphic code detection device based on the improved Mamba model provided by the embodiment of the present application is shown. As shown in the figure, the graphic code detection device 200 includes a first processing module 201, a second processing module 202, and a third processing module 203. Figure 8 The first processing module 201 inputs the to-be-detected image into the backbone network in the target detection model to obtain a plurality of first feature maps of different scales output by the backbone network, wherein the backbone network is used for feature extraction of the to-be-detected image, and the backbone network includes a cascaded randomized regularization deep convolutional neural network module and a visual clue fusion module.

[0076] The first processing module 201 inputs the to-be-detected image into the backbone network in the target detection model to obtain a plurality of first feature maps of different scales output by the backbone network, wherein the backbone network is used for feature extraction of the to-be-detected image, and the backbone network includes a cascaded randomized regularization deep convolutional neural network module and a visual clue fusion module.

[0077] The second processing module 202 inputs the plurality of first feature maps into the path aggregation feature pyramid network in the target detection model to obtain a plurality of second feature maps of different scales output by the path aggregation feature pyramid network, wherein the path aggregation feature pyramid network is used for feature fusion processing of the plurality of first feature maps, and the convolutional layer in the path aggregation feature pyramid network is a depth separable convolutional layer.

[0078] The third processing module 203 inputs the plurality of second feature maps into the detection head in the target detection model to obtain a detection result output by the detection head, wherein the detection result includes position information of the graphic code in the to-be-detected image.

[0079] The graphic code detection device 200 provided by the embodiment is used to execute the graphic code detection method in the foregoing method embodiment, and has similar implementation principles and technical effects, which will not be described here.

[0080] It is worth noting that the graphic code detection device 200 provided by the embodiment also includes other modules for executing each step of the above graphic code detection method embodiment, which will not be described here.

[0081] Figure 9A structure schematic diagram of the graphical code detection device based on the improved Mamba model is shown, and the specific implementation of the graphical code detection device is not limited in the embodiments of the present application.

[0082] As shown in Figure 9 The graphical code detection device 300 can include a processor 302 and a memory 304.

[0083] The memory 304 is configured to store a computer program 306. The memory 304 can include a high-speed RAM memory, and can also include a non-volatile memory such as at least one disk memory. The computer program 306 can include computer executable instructions.

[0084] The processor 302 is configured to execute the computer program 306 to implement the above-mentioned graphical code detection method embodiments.

[0085] The processor 302 can be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the graphical code detection device 300 can be the same type of processor, such as one or more CPUs; or can be different types of processors, such as one or more CPUs and one or more ASICs.

[0086] The embodiments of the present application provide a computer readable storage medium, and the storage medium stores a computer program. The computer program is executed by a processor to implement the above-mentioned graphical code detection method embodiments.

[0087] The embodiments of the present application provide a computer program, and the computer program can be executed by a processor to implement the above-mentioned graphical code detection method embodiments.

[0088] The embodiments of the present application provide a computer program product, and the computer program product includes a computer program. The computer program is executed by a processor to implement the above-mentioned graphical code detection method embodiments.

[0089] In several embodiments provided by the present application, any function if realized in the form of a software function module / unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the technical solutions of the present application can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or an electronic device) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media capable of storing computer program codes.

[0090] The algorithms and displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems can be used with these teachings, based on the description as provided herein. In terms of input / output, the structure of such systems and other apparatus can be apparent to those skilled in the art from the description herein. Also, the present embodiments are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the teachings of the present application as described herein, and any references below to specific languages are provided for disclosure of enablement only.

[0091] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the claims, the word "first", "second", and "third", etc. does not limit the number for these elements. These words are only used to distinguish between two entities or steps. The steps of the methods described herein do not have to be performed in the exact order described.

[0092] The above-described embodiments are merely illustrative for the present application and are not intended to limit the scope of the present application. It should be pointed out that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these modifications and improvements shall fall within the scope of the present application. Therefore, the scope of the present application shall be subject to the appended claims.

Claims

1. A method for detecting graphical codes based on an improved Mamba model, characterized in that, The method comprises: inputting a to-be-detected image into a backbone network in a target detection model to obtain a plurality of first feature maps of different scales output by the backbone network, wherein the backbone network is used for feature extraction on the to-be-detected image, and the backbone network comprises a cascaded randomized regularization deep convolutional neural network module and a visual clue fusion module; inputting the plurality of first feature maps into a path aggregation feature pyramid network in the target detection model to obtain a plurality of second feature maps of different scales output by the path aggregation feature pyramid network, wherein the path aggregation feature pyramid network is used for feature fusion processing on the plurality of first feature maps, and a convolutional layer in the path aggregation feature pyramid network is a depth separable convolutional layer; inputting the plurality of second feature maps into a detection head in the target detection model to obtain a detection result output by the detection head, wherein the detection result comprises position information of a graphical code in the to-be-detected image.

2. The method of claim 1, wherein, The backbone network comprises a down-sampling module, four backbone units and a spatial pyramid pooling fast module which are sequentially cascaded, each of the backbone units comprises a cascaded randomized regularization deep convolutional neural network module and a visual clue fusion module; the randomized regularization deep convolutional neural network module in the second backbone unit of the four backbone units is used for outputting a first feature map of a first scale to the path aggregation feature pyramid network; the randomized regularization deep convolutional neural network module in the third backbone unit of the four backbone units is used for outputting a first feature map of a second scale to the path aggregation feature pyramid network; the spatial pyramid pooling fast module is used for outputting a first feature map of a third scale to the path aggregation feature pyramid network.

3. The method of claim 1, wherein, The randomized regularization deep convolutional neural network module comprises a feature channel splitting layer, a first convolutional layer, a first depth separable convolutional layer, a second convolutional layer, a first feature splicing layer and a first channel random layer, wherein the feature channel splitting layer is used for receiving a feature map, splitting the received feature map into a first feature map and a second feature map in a channel dimension, outputting the first feature map to the first convolutional layer, and outputting the second feature map to the first feature splicing layer; or The randomized regularization deep convolutional neural network module comprises a first branch, a second branch and a cascaded second feature splicing layer and a second channel random layer, wherein the first branch comprises a cascaded second depth separable convolutional layer and a third convolutional layer, the second branch comprises a cascaded fourth convolutional layer, a third depth separable convolutional layer and a fifth convolutional layer, and the second feature splicing layer is used for receiving feature maps output by the third convolutional layer and the fifth convolutional layer.

4. The method of claim 1, wherein, The plurality of first feature maps output by the backbone network comprises a first feature map of a first scale, a first feature map of a second scale and a first feature map of a third scale; The plurality of second feature maps output by the path aggregation feature pyramid network comprises a second feature map of a first scale, a second feature map of a second scale and a second feature map of a third scale; The path aggregation feature pyramid network processes the multiple first feature maps to obtain multiple second feature maps through the following steps: The first feature map of the third scale is up-sampled to obtain a third feature map of a second scale, and the first feature map of the second scale and the third feature map of the second scale are fused to obtain a first fused feature map of the second scale; The first fused feature map of the second scale is processed by the optimized deep state space module to obtain a fourth feature map of the second scale, the fourth feature map of the second scale is up-sampled to obtain a fifth feature map of a first scale, the first feature map of the first scale and the fifth feature map of the first scale are fused to obtain a second fused feature map of the first scale, and the second fused feature map of the first scale is processed by the optimized deep state space module to obtain the second feature map of the first scale; The second feature map of the first scale is down-sampled by the deep separable convolution layer to obtain a sixth feature map of a second scale, the fourth feature map of the second scale and the sixth feature map of the second scale are fused to obtain a third fused feature map of the second scale, and the third fused feature map of the second scale is processed by the deep separable convolution layer to obtain the second feature map of the second scale; The second feature map of the second scale is down-sampled by the deep separable convolution layer to obtain a seventh feature map of a third scale, the first feature map of the third scale and the seventh feature map of the third scale are fused to obtain a fourth fused feature map of the third scale, and the fourth fused feature map of the third scale is processed by the deep separable convolution layer to obtain the second feature map of the third scale.

5. The method of claim 4, wherein, The path aggregation feature pyramid network fuses two feature maps through one or more of the parallel guided selection scanning algorithm module, the serial guided selection scanning algorithm module and the adaptive gating feature fusion module.

6. The method of claim 5, wherein, The path aggregation feature pyramid network fuses two feature maps through the cascaded parallel guided selection scanning algorithm module, the serial guided selection scanning algorithm module and the adaptive gating feature fusion module.

7. The method of claim 1, wherein, The method further comprises: Extraneous position information in a group of position information in the detection result is deleted by a non-maximum suppression algorithm to retain one position information in the group of position information, wherein the position information corresponding to the same graphic code in the detection result belongs to the group of position information.

8. A pattern code detection apparatus based on an improved Mamba model, characterized by, The device comprises: The first processing module is configured to input a to-be-detected image into a backbone network in a target detection model to obtain multiple first feature maps of different scales output by the backbone network, wherein the backbone network is configured to extract features of the to-be-detected image, and the backbone network comprises a cascaded randomized regularization deep convolutional neural network module and a visual clue fusion module. The first processing module is configured to input a to-be-detected image into a backbone network in a target detection model to obtain multiple first feature maps of different scales output by the backbone network, wherein the backbone network is configured to extract features of the to-be-detected image, and the backbone network comprises a cascaded randomized regularization deep convolutional neural network module and a visual clue fusion module. The second processing module is configured to input the multiple first feature maps into a path aggregation feature pyramid network in the target detection model to obtain multiple second feature maps of different scales output by the path aggregation feature pyramid network, wherein the path aggregation feature pyramid network is configured to perform feature fusion processing on the multiple first feature maps, and a convolution layer in the path aggregation feature pyramid network is a depth separable convolution layer. The third processing module is configured to input the multiple second feature maps into a detection head in the target detection model to obtain a detection result output by the detection head, wherein the detection result comprises position information of the graphic code in the to-be-detected image.

9. A device for detecting graphical codes based on an improved Mamba model, comprising a memory, a processor and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the graphic code detection method based on the improved Mamba model according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the graphic code detection method based on the improved Mamba model according to any one of claims 1-7.

Images