Graphic code detection method and device and computer readable storage medium

Abstract

The embodiment of the invention provides a graphic code detection method and device and a computer readable storage medium. The method comprises the steps that a first picture containing at least one graphic code is acquired; the first picture is input into a target detection model, a detection result of the at least one graphic code in the first picture is obtained, the target detection model adopts a lightweight network structure, and the detection result comprises coordinates and a decoding result of the at least one graphic code in the first picture; and outputting the detection result. Therefore, a plurality of graphic codes can be quickly and effectively identified at the same time.

Description

Technical Field

[0001] This document relates to the field of terminals, and in particular to a method, device and computer-readable storage medium for detecting graphic codes. Background Technology

[0002] With the widespread use of mobile devices, people can scan and recognize various graphic codes using their mobile devices.

[0003] However, existing graphic code detection technologies can usually only recognize a single graphic code at a time. When recognizing multiple graphic codes, the recognition device is prone to lag or failure to recognize the code, resulting in low recognition efficiency. Summary of the Invention

[0004] This application provides a graphic code detection method, device, and computer-readable storage medium to solve the problem that existing graphic code detection technologies cannot effectively identify multiple graphic codes.

[0005] To solve the above-mentioned technical problems, the embodiments of this application are implemented as follows: Firstly, a method for detecting graphic codes is provided, the method comprising: Obtain a first image containing at least one graphic code; The first image is input into the target detection model to obtain the detection result of at least one graphic code in the first image. The target detection model adopts a lightweight network structure, and the detection result includes the coordinates and decoding result of at least one graphic code in the first image. Output the detection results.

[0006] Secondly, an electronic device is provided, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in the first aspect.

[0007] Thirdly, a computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method described in the first aspect.

[0008] In this embodiment of the application, when performing the identification of at least one graphic code on the first image, the target detection model uses a lightweight network structure, which requires less computing resources, is friendly to the recognition device, and will not cause the recognition device to lag or fail to recognize during the recognition process. Therefore, it can quickly and effectively identify multiple graphic codes at the same time. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic flowchart of a graphic code detection method provided in one embodiment of this application.

[0011] Figure 2 This is a schematic diagram of the shooting interface of the mobile terminal provided in this application, which captures a first image containing at least one graphic code located on a non-planar surface.

[0012] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0013] Figure 4 This is a schematic diagram of the structure of a graphic code detection device provided in one embodiment of this application. Detailed Implementation

[0014] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in one or more embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the protection scope of this document.

[0015] The terms "first," "second," etc., used in this application and claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in this application and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0016] With the widespread adoption of smartphones and tablets, mobile devices have become an indispensable part of daily life, providing a broad user base for image barcode scanning technology. Users can easily scan and recognize image barcodes using their own mobile devices without additional hardware or equipment. However, existing image barcode scanning technologies only work for single QR codes or image barcodes located on a flat surface, such as multiple image barcodes on a poster, where users can choose to scan only one as needed. For a large number of image barcodes (e.g., more than 10), especially when these image barcodes are located on non-planar surfaces, there is currently no good solution.

[0017] For example, a box may have more than 10 or more non-planar graphic codes arranged in a staggered pattern. Furthermore, some devices may have even more complex non-planar graphic codes. For instance, passive optical transmission devices may have at least one row of terminals at both ends, each with graphic codes affixed to it, all located on non-planar surfaces. These devices may be installed in difficult-to-operate environments, such as utility poles, requiring operators to climb ladders. If these devices are used in harsh environments, such as low light, strong light, or rainy weather, existing graphic code scanning technology may be unable to recognize them.

[0018] To address at least one of the aforementioned problems, this application proposes a method, apparatus, and computer-readable storage medium for detecting graphic codes. The method can be executed by an electronic device or software (such as an SDK) installed in the electronic device. The electronic device includes, but is not limited to, any one of the following mobile terminals: smartphones, personal computers (PCs), laptops, tablets, e-readers, wearable devices, etc.

[0019] It should be noted that, in the embodiments of this application, the graphic code may include, but is not limited to, at least one of QR codes and barcodes.

[0020] It should be noted that, in the embodiments of this application, a graphic code located on a non-planar surface can refer to a graphic code that is affixed or printed on a curved surface (non-planar surface) in the actual shooting scene, and a graphic code located on a planar surface can refer to a graphic code that is affixed or printed on a planar surface in the actual shooting scene.

[0021] It should also be noted that the graphic code detection method proposed in this application can identify multiple graphic codes with complex arrangements located in a planar or non-planar manner.

[0022] The following description, in conjunction with the accompanying drawings, illustrates a graphic code detection method provided in an embodiment of this application.

[0023] like Figure 1As shown, one embodiment of this application provides a graphic code detection method that can be applied to a mobile terminal, i.e., the graphic code recognition device is a mobile terminal. This mobile terminal can be a smart mobile device with image capture and display functions. The method may include: Step 101: Obtain a first image containing at least one graphic code.

[0024] In some embodiments, the mobile terminal may capture a first image containing at least one graphic code using its own camera, that is, the first image is captured by the mobile terminal.

[0025] In other embodiments, the mobile terminal may also receive a first image containing at least one graphic code sent by another device, i.e., the first image may be captured by another device and then sent to the mobile terminal.

[0026] Of course, the first image containing at least one graphic code can also be obtained through other means. This application embodiment does not limit the specific method of obtaining the first image.

[0027] Optionally, regardless of which device captures the first image, to ensure the recognition effect of the graphic code, it is necessary to ensure that the graphic code contained in the first image is clear when capturing it. To achieve this, during the capture, the photographer should maintain a distance of approximately 20cm between the camera of the capturing device and the object to be photographed, ensuring that the graphic code is clearly visible and not obstructed by other objects, while also ensuring that each graphic code to be recognized is included in the preview photo. Furthermore, to ensure the recognition effect, during the capture, it is also possible to ensure that only the graphic code is present in the preview photo, minimizing the presence of other useless objects.

[0028] Optionally, at least one graphic code contained in the first image is located on a non-planar surface in the actual shooting scene.

[0029] Step 102: Input the first image into the target detection model to obtain the detection result of at least one graphic code in the first image, wherein the target detection model adopts a lightweight network structure, and the detection result includes the coordinates and decoding result of at least one graphic code in the first image.

[0030] The coordinates of the graphic code can be Rect coordinates, and the decoding result of the graphic code can include the decoding result of the information contained in the graphic code.

[0031] The object detection model may include, but is not limited to, one of the following object detection models: EfficientDet, YOLO (You Only LookOnce) series models, RetinaNet, and Faster R-CNN.

[0032] As an optional example, the object detection model can be YOLO V7 (You Only Look Once Version 7), and the YOLO V7 uses the lightweight ShuffleNet as its backbone network.

[0033] YOLO v7, as an object detection model, includes a backbone network, a neck network, and a head network. The neck network connects the backbone network and the head network, where: The back backbone network is used to extract features from an input image containing at least one graphic code located on a non-planar surface; The neck network is used to further extract features from the features extracted from the posterior backbone network; The prediction network is used to predict the target detection result based on the features extracted by the neck network.

[0034] Furthermore, YOLO v7 can use the lightweight ShuffleNet as a post-backbone network to improve the model's speed.

[0035] In some embodiments, a target detection model capable of recognizing at least one graphic code located on a non-planar surface is trained by inputting a large number of first sample images into YOLO v7. Alternatively, a target detection model capable of recognizing at least one graphic code located on a planar surface can be trained by inputting a large number of third sample images into YOLO v7. It is understood that by inputting both a large number of first sample images and a large number of third sample images as training data into YOLO v7, the trained target detection model possesses the ability to recognize not only at least one graphic code located on a non-planar surface, but also the ability to recognize at least one graphic code located on a planar surface.

[0036] Optionally, in order to further improve the object detection model's ability to recognize repetitive and / or overlapping non-planar graphic codes, the training data for training the object detection model may also include multiple second sample images, each containing at least two graphic codes that are non-planar and satisfy the first condition.

[0037] Optionally, in order to further improve the object detection model's ability to recognize repetitive and / or overlapping planar graphic codes, the training data for training the object detection model may also include multiple fourth sample images, wherein the fourth sample images contain at least two graphic codes that are planar and satisfy the first condition.

[0038] The first condition includes at least one of the following: At least one pair of the at least two graphic codes overlaps; At least one pair of the at least two graphic codes is repeated.

[0039] Optionally, to enable the object detection model to better recognize multiple graphic codes arranged in complex, non-planar configurations, both the first and second sample images used to train the model contain multiple graphic codes arranged in a regular or irregular manner on a non-planar surface. Correspondingly, if the first image contains multiple graphic codes arranged in a regular, non-planar configuration, for example... Figure 2 As shown, the first image obtained contains a row of multiple barcodes affixed to multiple wires. These multiple barcodes are all located on a non-planar surface. Then, by inputting the first image into the trained target detection model, the recognition result of at least one row of graphic codes located on a non-planar surface can be obtained.

[0040] Optionally, in order to enable the object detection model to have a better recognition ability for multiple graphic codes arranged in a complex plane, the third sample image and the fourth sample image used to train the object detection model both contain multiple graphic codes arranged in a regular or irregular plane.

[0041] Optionally, during the training of the object detection model, before inputting the sample images (such as at least one of the first, second, third, and fourth sample images) from the training data into the object detection model, the sample images may be preprocessed. The preprocessing may include at least one of format conversion, compression, sharpening, and enhancement. For example, the preprocessing of the sample images may include: converting the sample images to bitmap format; and compressing the converted bitmap format sample images to prevent excessively large images from causing mobile terminal crashes.

[0042] Accordingly, during the prediction process using the target detection model, before inputting the first image into the target detection model, Figure 1 The method may further include: preprocessing the first image, wherein the preprocessing may include at least one of format conversion, compression, sharpening, and enhancement. For example, the preprocessing of the first image may include: converting the first image into a bitmap format; and compressing the first image converted to a bitmap format to prevent the first image from being too large and causing the mobile terminal to freeze, for example, the first image may be compressed to within 100k.

[0043] For example, preprocessing of sample images may include: converting the sample image into a bitmap format; compressing the converted bitmap format sample image; sharpening and enhancing the compressed sample image to improve image resolution; then converting the sharpened and enhanced bitmap format sample image into an image format, and then having the object detection model detect the graphic code in it.

[0044] Accordingly, the preprocessing of the first image may include: converting the first image into a bitmap format; compressing the first image converted into a bitmap format; sharpening and enhancing the compressed first image to improve the image resolution; then converting the sharpened and enhanced bitmap format first image into an image format, and then having the object detection model detect the graphic code in it.

[0045] Optionally, partial preprocessing of the sample image or the first image can be performed by the object detection model before detecting the graphic code within it.

[0046] Based on this, further, the step of inputting the first image into the target detection model to obtain the detection result of at least one graphic code in the first image may include: The first image in image format is segmented using the target detection model. The target detection model is used to detect graphic codes in the segmented first image to obtain the detection results of at least one graphic code in the first image.

[0047] For example, suppose Figure 1 The method shown is executed by a specified SDK. After the mobile terminal captures the first image, it converts the first image into a bitmap format and performs certain technical processing on the bitmap image, such as compression, to prevent the mobile terminal from freezing due to excessively large images. The first image can be compressed to within 100k. Then, the first image is uploaded to the specified SDK. After receiving the first image, the specified SDK uses an improved YOLO V7 to perform sharpening and data enhancement operations on the bitmap image to improve its resolution. Then, the processed bitmap image is converted back into an image format. The improved YOLO V7 then segments the processed first image and calculates the coordinates of each graphic code to generate Rect coordinates. This allows for more accurate detection of the coordinates of the graphic codes in the first image.

[0048] Step 103: Output the detection results.

[0049] For example, suppose Figure 1If the method shown is executed by a specified SDK, the specified SDK will return the graphic code detection results obtained in step 102 (the coordinates and decoding results of at least one graphic code in the first image) to the mobile terminal. The mobile terminal will then use drawing technology to draw and display each graphic code at a specified position.

[0050] The graphic code detection method proposed in this application uses a lightweight network structure in the target detection model when detecting at least one graphic code on a first image. This requires less computing resources, is friendly to recognition devices, and will not cause the recognition device to lag or fail to recognize the graphic code during the recognition process. Therefore, it can quickly and effectively identify multiple graphic codes at the same time.

[0051] Furthermore, some embodiments of this application propose a graphic code detection method that can use a first sample image containing graphic codes located on a non-planar surface as training data to train a target detection model that can detect graphic codes located on a non-planar surface. Then, by applying the target detection model, at least one graphic code located on a non-planar surface in the input first image can be detected, thereby achieving the purpose of effectively identifying graphic codes located on a non-planar surface.

[0052] Therefore, it can be seen that the graphic code detection method proposed in some embodiments of this application can quickly provide correct recognition results for a large number of graphic codes (such as more than 10), especially when these graphic codes are located on a non-planar surface.

[0053] Optionally, the graphic code detection method proposed in this application embodiment may further include: If the detection result contains a first graphic code with an incorrect recognition, a second image containing the first graphic code is re-acquired. The second image is input into the target detection model to obtain the detection result of the first graphic code in the second image.

[0054] In other words, if an incorrect graphic code is encountered, the operator can perform an incremental scan on the graphic code separately to identify the correct graphic code.

[0055] Optionally, step 101 may specifically include: In response to an operation on a target button in the image capture interface, the capture light source is adjusted, wherein the target button includes one of a first button and a second button, the first button being a trigger button for a light source adjustment strategy set for an outdoor shooting environment, and the second button being a trigger button for a light source adjustment strategy set for an indoor shooting environment; In response to an operation of the capture button on the image capture interface, a first image containing at least one graphic code is captured.

[0056] Optionally, before step 101, the graphic code detection method proposed in this application may further include: In response to the operation of the start button 21 in the image capture interface, the graphic code detection method proposed in this application embodiment is activated.

[0057] For example, such as Figure 2 As shown, the mobile terminal's camera interface includes a start button 21 for a graphic code detection method proposed in this application embodiment. When an operation on the first button 21 is detected, the graphic code detection method proposed in this application embodiment is activated. Optionally, if no operation on the first button 21 is detected, existing graphic code recognition technologies (graphic code recognition technologies using non-lightweight network structure target detection models or other existing graphic code recognition technologies) can be used to recognize the graphic codes, allowing for flexible control of the graphic code recognition technology used by the mobile terminal and improving the recognition efficiency. For example, when encountering a construction worker using a passive optical transmission device with poor device performance, this button can be activated. In this case, a super-resolution algorithm (the improved YOLOv7 in this application embodiment) can be enabled to improve the recognition performance of the mobile terminal, so that all graphic codes and QR codes to be recognized can be quickly identified.

[0058] Furthermore, such as Figure 2 As shown, the mobile terminal's shooting interface includes a first button 22 and a second button 23. Upon detecting an operation on the first button 22, the shooting light source of the mobile terminal is adjusted. The second button 22 is a trigger button for a light source adjustment strategy set for outdoor shooting environments. For example, in outdoor construction scenarios with passive optical transmission equipment, if construction workers encounter adverse environments, such as low light or strong light, they can activate this button to supplement or reduce the light, facilitating the capture of clear images containing graphic codes for quick identification. Alternatively, as... Figure 2 As shown, the mobile terminal's shooting interface includes a second button 23. Upon detecting an operation on the second button 23, the shooting light source of the mobile terminal is adjusted. The second button 23 is a trigger button for a light source adjustment strategy set for indoor shooting environments. For example, in an indoor construction scenario with a passive optical transmission device, if construction workers encounter dim lighting, they can turn on this button to increase brightness, making it easier for them to capture clear images of graphic codes for quick identification. Subsequently, in response to an operation on the shooting button 24 in the image shooting interface, a first image containing at least one graphic code is captured.

[0059] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0060] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 3 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0061] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0062] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0063] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a graphic code detection device at the logical level. The processor executes the program stored in memory and specifically performs the following operations: Obtain a first image containing at least one graphic code; The first image is input into the target detection model to obtain the detection result of at least one graphic code in the first image. The target detection model adopts a lightweight network structure, and the detection result includes the coordinates and decoding result of at least one graphic code in the first image. Output the detection results.

[0064] The above is as stated in this application. Figure 3 The method executed by the graphic code detection device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0065] The electronic device can also perform Figure 1 The method, and implement the graphic code detection device in Figure 1 The functions described in the illustrated embodiments will not be repeated here.

[0066] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0067] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device comprising multiple target applications, enable the portable electronic device to perform... Figure 1 The method of the illustrated embodiment is specifically used to perform the following operations: Obtain a first image containing at least one graphic code; The first image is input into the target detection model to obtain the detection result of at least one graphic code in the first image. The target detection model adopts a lightweight network structure, and the detection result includes the coordinates and decoding result of at least one graphic code in the first image. Output the detection results.

[0068] Figure 4 This is a schematic diagram of the structure of a graphic code detection device 400 according to an embodiment of this application. Please refer to... Figure 4 In one software implementation, the graphic code detection device 400 can be applied to a mobile terminal, and the graphic code detection device 400 may include: a first image acquisition module 401, a first input module 402, and a result output module 403.

[0069] The first image acquisition module 401 is used to acquire a first image containing at least one graphic code.

[0070] In some embodiments, the image acquisition module 401 can acquire a first image containing at least one graphic code located on a non-planar surface by taking a picture, that is, the first image is captured by the mobile terminal.

[0071] In other embodiments, the image acquisition module 401 may also receive a first image sent by another device that contains at least one graphic code located on a non-planar surface, i.e., the first image may be captured by another device and then sent to the mobile terminal.

[0072] Of course, the first image containing at least one graphic code located on a non-planar surface can also be obtained through other means. This application embodiment does not limit the specific method of obtaining the first image.

[0073] Optionally, regardless of which device captures the first image, to ensure the recognition effect of the graphic code, it is necessary to ensure that the graphic code contained in the first image is clear when capturing it. To achieve this, during the capture, the photographer should maintain a distance of approximately 20cm between the camera of the capturing device and the object to be photographed, ensuring that the graphic code is clearly visible and not obstructed by other objects, while also ensuring that each graphic code to be recognized is included in the preview photo. Furthermore, to ensure the recognition effect, during the capture, it is also possible to ensure that only the graphic code is present in the preview photo, minimizing the presence of other useless objects.

[0074] Optionally, at least one graphic code contained in the first image is located on a non-planar surface in the actual shooting scene.

[0075] The first input module 402 is used to input the first image into the target detection model to obtain the detection result of at least one graphic code in the first image, wherein the target detection model adopts a lightweight network structure, and the detection result includes the coordinates and decoding result of at least one graphic code in the first image.

[0076] The coordinates of the graphic code can be Rect coordinates, and the decoding result of the graphic code can include the decoding result of the information contained in the graphic code.

[0077] The object detection model may include, but is not limited to, one of the following object detection models: EfficientDet, YOLO (You Only LookOnce) series models, RetinaNet, and Faster R-CNN.

[0078] As an optional example, the object detection model can be YOLO V7 (You Only Look Once Version 7), and the YOLO V7 uses the lightweight ShuffleNet as its backbone network.

[0079] YOLO v7, as an object detection model, includes a backbone network, a neck network, and a head network. The neck network connects the backbone network and the head network, where: The back backbone network is used to extract features from an input image containing at least one graphic code located on a non-planar surface; The neck network is used to further extract features from the features extracted from the posterior backbone network; The prediction network is used to predict the target detection result based on the features extracted by the neck network.

[0080] Furthermore, YOLO v7 can use the lightweight ShuffleNet as a post-backbone network to improve the model's speed.

[0081] In some embodiments, a target detection model capable of recognizing at least one graphic code located on a non-planar surface is trained by inputting a large number of first sample images into YOLO v7. Alternatively, a target detection model capable of recognizing at least one graphic code located on a planar surface can be trained by inputting a large number of third sample images into YOLO v7. It is understood that by inputting both a large number of first sample images and a large number of third sample images as training data into YOLO v7, the trained target detection model possesses the ability to recognize not only at least one graphic code located on a non-planar surface, but also the ability to recognize at least one graphic code located on a planar surface.

[0082] Optionally, in order to further improve the object detection model's ability to recognize repetitive and / or overlapping non-planar graphic codes, the training data for training the object detection model may also include multiple second sample images, each containing at least two graphic codes that are non-planar and satisfy the first condition.

[0083] Optionally, in order to further improve the object detection model's ability to recognize repetitive and / or overlapping planar graphic codes, the training data for training the object detection model may also include multiple fourth sample images, wherein the fourth sample images contain at least two graphic codes that are planar and satisfy the first condition.

[0084] The first condition includes at least one of the following: At least one pair of the at least two graphic codes overlaps; At least one pair of the at least two graphic codes is repeated.

[0085] Optionally, to enable the object detection model to better recognize multiple graphic codes arranged in complex, non-planar configurations, both the first and second sample images used to train the model contain multiple graphic codes arranged in a regular or irregular manner on a non-planar surface. Correspondingly, if the first image contains multiple graphic codes arranged in a regular, non-planar configuration, for example... Figure 2 As shown, the first image obtained contains a row of multiple barcodes affixed to multiple wires. These multiple barcodes are all located on a non-planar surface. Then, by inputting the first image into the trained target detection model, the recognition result of at least one row of graphic codes located on a non-planar surface can be obtained.

[0086] Optionally, in order to enable the object detection model to have a better recognition ability for multiple graphic codes arranged in a complex plane, the third sample image and the fourth sample image used to train the object detection model both contain multiple graphic codes arranged in a regular or irregular plane.

[0087] Optionally, the apparatus 400 may further include a preprocessing module, used to preprocess the sample images (such as at least one of a first sample image, a second sample image, a third sample image, and a fourth sample image) in the training data before inputting them into the object detection model during the training process. The preprocessing may include at least one of format conversion, compression, sharpening, and enhancement. For example, preprocessing the sample images may include converting the sample images to a bitmap format and compressing the converted bitmap images to prevent excessively large images from causing mobile terminal crashes.

[0088] Accordingly, during the prediction process using the object detection model, the preprocessing module can also be used to: preprocess the first image before inputting it into the object detection model, wherein the preprocessing may include at least one of format conversion, compression, sharpening, and enhancement. For example, the preprocessing of the first image may include: converting the first image into a bitmap format; and compressing the first image converted to a bitmap format to prevent the first image from being too large and causing the mobile terminal to freeze, for example, the first image may be compressed to within 100k.

[0089] For example, preprocessing of sample images may include: converting the sample image into a bitmap format; compressing the converted bitmap format sample image; sharpening and enhancing the compressed sample image to improve image resolution; then converting the sharpened and enhanced bitmap format sample image into an image format, and then having the object detection model detect the graphic code in it.

[0090] Accordingly, the preprocessing of the first image may include: converting the first image into a bitmap format; compressing the first image converted into a bitmap format; sharpening and enhancing the compressed first image to improve the image resolution; then converting the sharpened and enhanced bitmap format first image into an image format, and then having the object detection model detect the graphic code in it.

[0091] Furthermore, based on this, the target detection model can be specifically used for: The first image in image format is cropped; The first image after being cut is subjected to graphic code detection to obtain the detection result of at least one graphic code in the first image.

[0092] For example, suppose Figure 1 The method shown is executed by a specified SDK. After capturing the first image, the first image is converted into a bitmap format and then subjected to certain technical processing, such as compression, to prevent the image from becoming too large and causing the mobile terminal to freeze. The first image can be compressed to within 100k. Then, the first image is uploaded to the specified SDK. After receiving the first image, the specified SDK uses an improved YOLO V7 to perform sharpening and data enhancement operations on the bitmap format first image to improve its resolution. Then, the processed bitmap format first image is converted back into an image format. The improved YOLO V7 then segments the processed first image and calculates the coordinates of each graphic code to generate Rect coordinates. This allows for more accurate detection of the coordinates of the graphic codes in the first image.

[0093] The result output module 403 is used to output the detection results.

[0094] Specifically, the result output module 403 can use drawing technology to draw and display each graphic code at a specified position.

[0095] The graphic code detection device proposed in this application uses a lightweight network structure in its target detection model when detecting at least one graphic code on a first image. This requires less computing resources, is friendly to recognition devices, and will not cause the recognition device to lag or fail to recognize the graphic code during the recognition process. Therefore, it can quickly and effectively identify multiple graphic codes at the same time.

[0096] Furthermore, some embodiments of this application propose a graphic code detection device that can use a first sample image containing graphic codes located on a non-planar surface as training data to train a target detection model that can detect graphic codes located on a non-planar surface. Then, by applying the target detection model, at least one graphic code located on a non-planar surface in the input first image can be detected, thereby achieving the purpose of effectively identifying graphic codes located on a non-planar surface.

[0097] Therefore, it can be seen that the graphic code detection device proposed in some embodiments of this application can quickly provide correct recognition results for a large number of graphic codes (such as more than 10), especially when these graphic codes are located on a non-planar surface.

[0098] Optionally, the graphic code detection device proposed in this application embodiment may further include: The second image acquisition module is used to re-acquire a second image containing the first graphic code when there is a first graphic code with an incorrect recognition in the detection result. The second input module is used to input the second image into the target detection model to obtain the detection result of the first graphic code in the second image.

[0099] In other words, if an incorrect graphic code is encountered, the operator can perform an incremental scan on the graphic code separately to identify the correct graphic code.

[0100] Optionally, the first image acquisition module 401 can be specifically used for: In response to an operation on a target button in the image capture interface, the capture light source is adjusted, wherein the target button includes one of a first button and a second button, the first button being a trigger button for a light source adjustment strategy set for an outdoor shooting environment, and the second button being a trigger button for a light source adjustment strategy set for an indoor shooting environment; In response to an operation of the capture button on the image capture interface, a first image containing at least one graphic code is captured.

[0101] Optionally, the graphic code detection device proposed in this application may further include: an activation module, used to activate the graphic code detection device proposed in the embodiments of this application in response to an operation of the activation button 21 in the image capture interface.

[0102] For example, such as Figure 2 As shown, the mobile terminal's camera interface includes a start button 21 for a graphic code detection device proposed in this application embodiment. When an operation on the first button 21 is detected, the graphic code detection device proposed in this application embodiment is activated. Optionally, if no operation on the first button 21 is detected, existing graphic code recognition technologies (graphic code recognition technologies using non-lightweight network structure target detection models or other existing graphic code recognition technologies) can be used to recognize the graphic codes, allowing for flexible control of the graphic code recognition technology used by the mobile terminal and improving the recognition efficiency. For example, when encountering a construction worker using a passive optical transmission device with poor device performance, this button can be activated. In this case, a super-resolution algorithm (the improved YOLOv7 in this application embodiment) can be enabled to improve the recognition performance of the mobile terminal, so that all graphic codes and QR codes to be recognized can be quickly identified.

[0103] Furthermore, such as Figure 2As shown, the mobile terminal's shooting interface includes a first button 22 and a second button 23. Upon detecting an operation on the first button 22, the shooting light source of the mobile terminal is adjusted. The second button 22 is a trigger button for a light source adjustment strategy set for outdoor shooting environments. For example, in outdoor construction scenarios with passive optical transmission equipment, if construction workers encounter adverse environments, such as low light or strong light, they can activate this button to supplement or reduce the light, facilitating the capture of clear images containing graphic codes for quick identification. Alternatively, as... Figure 2 As shown, the mobile terminal's shooting interface includes a second button 23. Upon detecting an operation on the second button 23, the shooting light source of the mobile terminal is adjusted. The second button 23 is a trigger button for a light source adjustment strategy set for indoor shooting environments. For example, in an indoor construction scenario with a passive optical transmission device, if construction workers encounter dim lighting, they can turn on this button to increase brightness, making it easier for them to capture clear images of graphic codes for quick identification. Subsequently, in response to an operation on the shooting button 24 in the image shooting interface, a first image containing at least one graphic code is captured.

[0104] The graphic code detection device 400 provided in this application embodiment can also perform... Figure 1 The method, and implementation Figure 1 The embodiments shown in this application have the same functions and achieve the same technical effects, and will not be described in detail here.

[0105] In summary, the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0106] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0107] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0108] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0109] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

Claims

1. A method for detecting graphic codes, characterized in that, The method includes: Obtain a first image containing at least one graphic code; The first image is input into the target detection model to obtain the detection result of at least one graphic code in the first image. The target detection model adopts a lightweight network structure, and the detection result includes the coordinates and decoding result of at least one graphic code in the first image. Output the detection results.

2. The method according to claim 1, characterized in that, The target detection model is YOLO V7, and the backbone network used by YOLO V7 is the lightweight ShuffleNet.

3. The method according to claim 1, characterized in that, The training data for the target detection model includes multiple first sample images, each containing at least one graphic code located on a non-planar surface.

4. The method according to claim 3, characterized in that, The training data also includes multiple second sample images, each containing at least two graphic codes located on a non-planar surface and satisfying a first condition, wherein the first condition includes at least one of the following: At least one pair of the at least two graphic codes overlaps; At least one pair of the at least two graphic codes is repeated.

5. The method according to claim 4, characterized in that, Both the first sample image and the second sample image contain multiple graphic codes arranged in a regular pattern on a non-planar surface.

6. The method according to any one of claims 1, characterized in that, Before inputting the first image into the object detection model, the method further includes: The first image is preprocessed, wherein the preprocessing includes at least one of format conversion, compression, sharpening, and enhancement.

7. The method according to any one of claims 1-6, characterized in that, The step of inputting the first image into the target detection model to obtain the detection result of at least one graphic code in the first image includes: The first image is segmented using the target detection model to obtain the coordinates of each graphic code in the first image; The target detection model is used to detect graphic codes in the segmented first image to obtain the decoding result of at least one graphic code in the first image.

8. The method according to any one of claims 1-6, characterized in that, The method further includes: If the detection result contains a first graphic code with an incorrect recognition, a second image containing the first graphic code is re-acquired. The second image is input into the target detection model to obtain the detection result of the first graphic code in the second image.

9. The method according to any one of claims 1-6, characterized in that, The step of obtaining a first image containing at least one graphic code includes: In response to an operation on a target button in the image capture interface, the capture light source is adjusted, wherein the target button includes one of a first button and a second button, the first button being a trigger button for a light source adjustment strategy set for an outdoor shooting environment, and the second button being a trigger button for a light source adjustment strategy set for an indoor shooting environment; In response to an operation of the capture button on the image capture interface, a first image containing at least one graphic code is captured.

10. An electronic device, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the method as described in any one of claims 1 to 9.

12. A computer program product comprising instructions, wherein when a computer executes the instructions of the computer program product, the computer performs the method as described in any one of claims 1 to 9.

Images