Bar code recognition method, bar code recognition device, electronic equipment and storage medium

Abstract

The invention discloses a bar code recognition method, a bar code recognition device, electronic equipment and a storage medium, and the bar code recognition method comprises the steps: obtaining a bar code image; a bar code image is input to a bar code corner coordinate detection model, corresponding bar code corner coordinates are obtained, and the bar code corner coordinate detection model comprises a target detection model and a feature point detection model; determining a target bar code area corresponding to the bar code image according to the bar code corner coordinates; and decoding the target bar code area to obtain corresponding bar code information. According to the method, image recognition is carried out on the target bar code area corresponding to the bar code image in the bar code recognition process, the corresponding bar code information is obtained, bar code recognition errors caused by recognition of background information of a non-bar code area are avoided, and the recognition accuracy of bar code recognition is improved.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and in particular relates to a barcode recognition method, a barcode recognition device, an electronic device and a storage medium. Background Art

[0002] With the development of mobile Internet, barcodes are used more and more widely in people's lives. Barcode recognition has spread to every corner of daily life. More and more people are accustomed to using mobile phone applications to scan code to pay, scan code to read information, scan code to register, and other functions.

[0003] At present, in the process of barcode recognition, the corresponding barcode information is obtained by performing image recognition on the collected barcode image. However, the collected barcode image usually contains a barcode area and a non-barcode area. When there is background information in the non-barcode area, the barcode information obtained by performing image recognition on the barcode image consists of the real barcode information corresponding to the barcode area and the non-real barcode information corresponding to the non-barcode area. The barcode recognition error leads to low recognition accuracy of the barcode. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a barcode recognition method, a barcode recognition device, an electronic device and a storage medium to overcome or at least partially solve the above problems of the prior art.

[0005] In a first aspect, an embodiment of the present application provides a barcode recognition method, comprising: acquiring a barcode image; inputting the barcode image into a barcode corner point coordinate detection model to obtain corresponding barcode corner point coordinates, the barcode corner point coordinate detection model comprising a target detection model and a feature point detection model; determining a target barcode area corresponding to the barcode image according to the barcode corner point coordinates; performing a decoding operation on the target barcode area to obtain corresponding barcode information.

[0006] In the second aspect, the embodiment of the present application provides a barcode recognition device, including an acquisition module, an input module, an area determination module and a decoding module. The acquisition module is used to acquire a barcode image; the input module is used to input the barcode image into a barcode corner point coordinate detection model to obtain the corresponding barcode corner point coordinates, and the barcode corner point coordinate detection model includes a target detection model and a feature point detection model; the area determination module is used to determine the target barcode area corresponding to the barcode image according to the barcode corner point coordinates; the decoding module is used to perform a decoding operation on the target barcode area to obtain the corresponding barcode information.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory; one or more processors coupled to the memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the barcode recognition method provided in the first aspect above.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the barcode recognition method provided in the first aspect above.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is executed on a computer device, the computer device executes the barcode recognition method provided in the first aspect above.

[0010] The solution provided by the present application obtains the corresponding barcode corner coordinates by acquiring a barcode image and inputting the barcode image into a barcode corner point coordinate detection model. The barcode corner point coordinate detection model includes a target detection model and a feature point detection model. According to the barcode corner point coordinates, the target barcode area corresponding to the barcode image is determined, and a decoding operation is performed on the target barcode area to obtain the corresponding barcode information. In the barcode recognition process, image recognition is performed on the target barcode area corresponding to the barcode image to obtain the corresponding barcode information, thereby avoiding the barcode recognition error caused by the recognition of background information in the non-barcode area, and improving the recognition accuracy of the barcode. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 A schematic diagram of a scenario of a barcode recognition system provided in an embodiment of the present application is shown.

[0013] Figure 2 A schematic flow chart of a barcode recognition method provided in an embodiment of the present application is shown.

[0014] Figure 3 A structural schematic diagram of a detection head structure of a barcode corner point coordinate detection model in a barcode recognition method provided in an embodiment of the present application is shown.

[0015] Figure 4 A schematic diagram of the structure of a filter core of the filtering process in the barcode recognition method provided in an embodiment of the present application is shown.

[0016] Figure 5 Another schematic flow chart of the barcode recognition method provided in an embodiment of the present application is shown.

[0017] Figure 6 A structural block diagram of a barcode recognition device provided in an embodiment of the present application is shown.

[0018] Figure 7 A functional block diagram of an electronic device provided in an embodiment of the present application is shown.

[0019] Figure 8 A computer-readable storage medium provided in an embodiment of the present application for storing or carrying a program code for implementing a barcode recognition method provided in an embodiment of the present application is shown.

[0020] Fig. 9 A computer program product provided in an embodiment of the present application and used to store or carry program codes for implementing the barcode recognition method provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0023] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates other working conditions.

[0024] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0025] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0026] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0027] With the development of mobile Internet, barcodes are used more and more widely in people's lives. Barcode recognition has spread to every corner of daily life. More and more people are accustomed to using mobile phone applications to scan code to pay, scan code to read information, scan code to register, and other functions.

[0028] At present, in the process of barcode recognition, the corresponding barcode information is obtained by performing image recognition on the collected barcode image. However, the collected barcode image usually contains a barcode area and a non-barcode area. When there is background information in the non-barcode area, the barcode information obtained by performing image recognition on the barcode image consists of the real barcode information corresponding to the barcode area and the non-real barcode information corresponding to the non-barcode area. The barcode recognition error leads to low recognition accuracy of the barcode.

[0029] In view of the above problems, the inventors have conducted a long period of research and proposed a barcode recognition method, a barcode recognition device, an electronic device and a storage medium provided in the embodiments of the present application. The method obtains a barcode image and inputs the barcode image into a barcode corner point coordinate detection model to obtain the corresponding barcode corner point coordinates. The barcode corner point coordinate detection model includes a target detection model and a feature point detection model. According to the barcode corner point coordinates, the target barcode area corresponding to the barcode image is determined, and a decoding operation is performed on the target barcode area to obtain the corresponding barcode information. In the barcode recognition process, image recognition is performed on the target barcode area corresponding to the barcode image to obtain the corresponding barcode information, thereby avoiding the barcode recognition error caused by the recognition of background information in the non-barcode area and improving the recognition accuracy of the barcode.

[0030] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0031] See also Figure 1 , which shows a schematic diagram of an application scenario of the barcode recognition system provided in an embodiment of the present application, and may include a terminal device 100, the terminal device 100 includes a camera 110 and a processor 120, the camera 110 is communicatively connected to the processor 120, and exchanges data with the processor 120.

[0032] Among them, the terminal device 100 can be a mobile terminal device (for example, any one of a mobile phone, a personal digital assistant (PDA), a tablet personal computer (Tablet PC), a laptop computer, a smart watch, a smart bracelet or a wearable device, etc.), or a fixed terminal device (desktop computer, smart panel, etc.). The type of the terminal device 100 is not limited here, and can be set according to actual needs.

[0033] The camera 110 can be used to capture the barcode of the object, obtain the corresponding barcode image, and send the barcode image to the processor 120. The camera 110 can be any one of a white light camera, an infrared camera, a laser camera, etc. The type of the camera 110 is not limited here and can be set according to actual needs.

[0034] The processor 120 may be configured to receive the barcode image sent by the camera 110 and perform a decoding operation on the barcode region of the barcode image to obtain corresponding barcode information.

[0035] See also Figure 2 , which shows a flow chart of a barcode recognition method provided by an embodiment of the present application. In a specific embodiment, the barcode recognition method can be applied to Figure 1 The terminal device 100 in the barcode recognition system shown in FIG. 1 is taken as an example below. Figure 2 The process shown is described in detail, and the barcode recognition method may include the following steps S110 to S140.

[0036] Step S110: Acquire a barcode image.

[0037] In an embodiment of the present application, when a user needs to identify the barcode of an object, a recognition instruction can be sent to the processor. The processor receives and responds to the recognition instruction and sends a corresponding image acquisition instruction to the camera. The camera receives and responds to the image acquisition instruction, performs image acquisition on the barcode of the object, obtains the corresponding barcode image, and sends the barcode image to the processor. The processor receives the barcode image returned by the camera.

[0038] In some embodiments, the terminal device may further include an input panel, which is communicatively connected to the processor and performs data exchange with the processor. When a user needs to recognize a barcode of an object, he or she may input a recognition instruction into the input panel, the input panel receives and responds to the recognition instruction, and forwards the recognition instruction to the processor, and the processor receives the recognition instruction forwarded by the input panel.

[0039] In some embodiments, the terminal device may further include a voice recognition module, which is communicatively connected to the processor and performs data exchange with the processor. When a user needs to recognize a barcode of an object, a voice message carrying a recognition instruction may be sent within the voice collection range of the voice recognition module. The voice recognition module collects the voice message, performs voice recognition on the voice message, obtains a recognition instruction, and sends the recognition instruction to the processor. The processor receives the recognition instruction sent by the voice recognition module.

[0040] In some embodiments, the barcode recognition system may further include a client, which is connected to the processor via a network and exchanges data with the processor via the network. When a user needs to recognize the barcode of an object, a recognition instruction may be sent to the client, the client receives and responds to the recognition instruction, forwards the recognition instruction to the processor via the network, and the processor receives the recognition instruction forwarded by the client.

[0041] Among them, the client can be any one of a mobile client (for example, a mobile phone client, a personal digital assistant (PDA) client, a tablet computer (Tablet Personal Computer, Tablet PC) client, a laptop computer client, a smart watch client, a smart bracelet client or a wearable client, etc.) or a fixed client (for example, a desktop computer client, a smart panel client, etc.). The type of client is not limited here and can be set according to actual needs.

[0042] The network can be any one of a ZigBee network, a Bluetooth (BT) network, a Wireless Fidelity (Wi-Fi) network, a Thread network, a Long Range Radio (LoRa) network, a Low-Power Wide-Area Network (LPWAN), an infrared network, a Narrow Band Internet of Things (NB-IoT), a Controller Area Network (CAN), a Digital Living Network Alliance (DLNA) network, a Wide Area Network (WAN), a Local Area Network (LAN), a Metropolitan Area Network (MAN) or a Wireless Personal Area Network (WPAN), etc. The type of network is not limited here and can be set according to actual needs.

[0043] Step S120: input the barcode image into the barcode corner point coordinate detection model to obtain the corresponding barcode corner point coordinates.

[0044] In an embodiment of the present application, the processor stores a pre-trained barcode corner point coordinate detection model. After acquiring the barcode image, the processor can input the barcode image into the barcode corner point coordinate detection model. The barcode corner point coordinate detection model receives and responds to the barcode image, detects the barcode corner point coordinates corresponding to the barcode image, and outputs the barcode corner point coordinates to the processor. The processor receives the barcode corner point coordinates output by the barcode corner point coordinate detection model.

[0045] The barcode corner point coordinate detection model may include a target detection model and a landmark detection model. The target detection model may be any one of a YOLO model, an SSD model, a RetinaNet model, or a Faster R-CNN model, etc. The type of the target detection model is not limited here and may be set according to actual needs.

[0046] In one application scenario, such as Figure 3As shown, the barcode corner point coordinate detection model may include a first branch, a second branch, and a third branch. Among them, the target detection model includes the first branch and the second branch, and the landmark detection model includes the third branch. The first branch, the second branch, and the third branch of the target detection model may be fused to obtain the barcode corner point coordinate detection model.

[0047] The first branch can be used to detect the barcode type of the barcode image to obtain corresponding barcode type information (classification, cls), cls can be any one of a one-dimensional barcode, a two-dimensional barcode or a three-dimensional barcode, etc., which is not limited here.

[0048] The second branch can be used to detect the barcode frame information in the barcode image to obtain the corresponding barcode frame information. The barcode frame information may include the barcode frame value (rectangle, reg) and the probability that the barcode frame is an object (object, obj). reg includes the coordinates of the center point of the barcode frame (x0, y0) and the height of the barcode frame (w, h), etc. The value range of obj is [0, 1], which is not limited here.

[0049] The third branch can be used to detect the barcode corner point coordinates in the barcode image to obtain the corresponding barcode angle coordinate information (lmk). LMK may include the four corner point coordinates of the barcode in the barcode image, for example, the barcode upper left corner point coordinates (x1, y1), the barcode lower left corner point coordinates (x2, y2), the barcode upper right corner point coordinates (x3, y3) and the barcode lower right corner point coordinates (x4, y4).

[0050] Step S130: determining the target barcode area corresponding to the barcode image according to the barcode corner point coordinates.

[0051] In the embodiment of the present application, after inputting the barcode image into the barcode corner point coordinate detection model and obtaining the corresponding barcode corner point coordinates, the processor can determine the target barcode area corresponding to the barcode image according to the barcode corner point coordinates. The target barcode area can be used to characterize the image area defined by the barcode corner point coordinates in the barcode image.

[0052] Specifically, after inputting the barcode image into the barcode corner point coordinate detection model and obtaining the corresponding barcode corner point coordinates, the processor can sequentially connect the barcode corner point coordinates to obtain the area surrounded by the barcode corner point coordinates, which is the target barcode area.

[0053] In some embodiments, due to image distortion during the image acquisition process, after the processor inputs the barcode image into the barcode corner point coordinate detection model and obtains the corresponding barcode corner point coordinates, it can determine the initial barcode area corresponding to the barcode image according to the barcode corner point coordinates, and perform perspective transformation processing on the initial barcode area to obtain the corresponding target barcode area, thereby achieving image correction of the barcode area corresponding to the barcode image, ensuring that the target barcode area is closer to the real barcode area, reducing image distortion of the target barcode area, and facilitating improving the recognition accuracy of the target barcode area when performing barcode recognition on the target barcode area.

[0054] The processor may correct and crop the initial barcode area according to a preset barcode aspect ratio using perspective transformation technology to obtain a corresponding target barcode area.

[0055] The preset barcode aspect ratio may be an aspect ratio value preset by the user, or an aspect ratio value automatically generated by the processor according to a recognition process of multiple barcode recognitions, etc. For example, the preset barcode aspect ratio may be 3:2, the preset barcode aspect ratio may be 2:1, the preset barcode aspect ratio may be 3:1, etc., which is not limited here.

[0056] In some embodiments, after inputting the barcode image into the barcode corner point coordinate detection model and obtaining the corresponding barcode corner point coordinates, the processor can determine the initial barcode area corresponding to the barcode image based on the barcode corner point coordinates, perform perspective transformation processing on the initial barcode area to obtain the corresponding perspective-processed barcode area, and perform sharpening processing on the perspective-processed barcode area to obtain the corresponding target barcode area, which can eliminate blurred afterimages of the barcode image and further help improve the recognition accuracy when performing barcode recognition on the target barcode area.

[0057] Among them, the sharpening processing can be a filtering processing, and the processor can perform filtering processing on the perspective processed barcode area to obtain the corresponding target barcode area, the boundary filter kernel value corresponding to the filtering processing is less than zero, the intermediate filter kernel value corresponding to the filtering processing is greater than zero, and the absolute value of the intermediate filter kernel value is greater than the absolute value of the boundary filter kernel value.

[0058] The barcode is characterized by vertical stripes. The filter kernel can be designed to reduce the vertical filtering effect and eliminate the horizontal blur afterimage effect, making the boundary of the barcode more obvious.

[0059] In one application scenario, the filter kernel of the filtering process may be 1*3, such as Figure 4As shown, the filter kernel includes a first boundary filter kernel (border1), a second boundary filter kernel (border1) and an intermediate filter kernel (middle), and border1 and border2 are connected through middle.

[0060] The first boundary filter kernel value corresponding to border1 (value border1 ), the second boundary filter kernel value corresponding to border2 (value border2 ) and the intermediate filter kernel value (value middle ) satisfies the following relationship:

[0061] value border1 <0, value middle >0,value border2 <0, value border1 =value border2 ;

[0062] |value middle |>|value border1 |.

[0063] Step S140: performing a decoding operation on the target barcode area to obtain corresponding barcode information.

[0064] In the embodiment of the present application, the terminal device may further include a preset application program interface (API), the preset API is connected to the processor, and the processor may call a corresponding preset decoding tool through the preset API, and the preset decoding tool is used to perform a decoding operation on the barcode to obtain the barcode information. The preset decoding tool may be any one of zbar or zxing, etc., which is not limited here.

[0065] After the processor determines the target barcode area corresponding to the barcode image based on the coordinates of the barcode corner points, it can call the corresponding preset decoding tool through the preset API. The preset decoding tool performs a decoding operation on the target barcode area to obtain the corresponding barcode information. In the barcode recognition process, image recognition is performed on the target barcode area corresponding to the barcode image to obtain the corresponding barcode information, thereby avoiding barcode recognition errors caused by recognizing background information in non-barcode areas and improving the recognition accuracy of barcodes.

[0066] The barcode information may include the barcode type and digital code, etc., which are not limited here.

[0067] In some embodiments, factors such as the acquisition angle and / or environment of the barcode image may cause shadows to be blocked or the image to be blurred in the barcode area. When a decoding operation is performed on the barcode area with shadows or blurred images, the barcode decoding may fail.

[0068] After the processor determines the target barcode area corresponding to the barcode image according to the coordinates of the barcode corner points, it can perform three-dimensional affine transformation processing on the target barcode area to obtain the corresponding three-dimensional affine barcode area, and call the corresponding preset decoding tool through the preset API. The preset decoding tool performs a decoding operation on the three-dimensional affine barcode area to obtain the corresponding barcode information. In the barcode recognition process, image recognition is performed on the three-dimensional affine barcode area after the three-dimensional affine processing to obtain the corresponding barcode information, thereby avoiding the failure of barcode decoding due to shadow occlusion or image blur in the barcode area, and improving the recognition success rate of the barcode.

[0069] In some embodiments, after the processor determines the target barcode area corresponding to the barcode image based on the coordinates of the barcode corner points, the processor can perform three-dimensional affine transformation processing on the target barcode area in multiple affine angle directions to obtain multiple corresponding three-dimensional affine barcode areas, each three-dimensional affine barcode area corresponds to an affine angle direction, and call the corresponding preset decoding tool through a preset API. The preset decoding tool performs a decoding operation on each three-dimensional affine barcode area in turn until the corresponding barcode information is obtained, thereby realizing multi-angle decoding of the barcode, avoiding the failure of barcode decoding due to the mismatch between the affine angle and the shooting angle, and further improving the recognition success rate of the barcode.

[0070] Among them, the multiple affine angles can be N angles in the interval of [0, α°]. The process of the processor performing three-dimensional affine transformation on the target barcode area in multiple three-dimensional affine angle directions can be that the processor performs N angle affine transformations in four viewing angles of up, down, left and right. During the affine transformation and decoding process, if the decoding is successful, the three-dimensional affine transformation and decoding operations are stopped.

[0071] The solution provided in this embodiment obtains the corresponding barcode corner coordinates by acquiring a barcode image and inputting the barcode image into a barcode corner coordinate detection model. The barcode corner coordinate detection model includes a target detection model and a feature point detection model. According to the barcode corner coordinates, the target barcode area corresponding to the barcode image is determined, and a decoding operation is performed on the target barcode area to obtain the corresponding barcode information. In the barcode recognition process, image recognition is performed on the target barcode area corresponding to the barcode image to obtain the corresponding barcode information, thereby avoiding the barcode recognition error caused by the recognition of background information in the non-barcode area, and improving the recognition accuracy of the barcode.

[0072] See also Figure 5 , which shows a flow chart of a barcode recognition method provided by another embodiment of the present application. In a specific embodiment, the barcode recognition method can be applied to Figure 1 The barcode recognition system shown in FIG. 1 is a terminal device 100. The terminal device 100 is taken as an example below. Figure 5 The process shown is described in detail, and the barcode recognition method may include the following steps S210 to S260.

[0073] Step S210: Acquire a barcode image.

[0074] In this embodiment, step S210 may refer to the contents of the corresponding steps in the aforementioned embodiments, which will not be described in detail here.

[0075] Step S220: construct and train a barcode corner point coordinate detection model.

[0076] In this embodiment, the processor can construct a corresponding barcode corner point coordinate detection model based on the target detection model and the landmark detection model, obtain a historical barcode image, and mark the corner point coordinates of the historical barcode image to obtain the corresponding historical marked barcode image, and divide the historical marked barcode image according to a preset division rule to obtain a corresponding training set and test, and input the training set to the barcode corner point coordinate detection model for training.

[0077] The preset division rule is an artificial division rule, for example, the preset division rule may be an artificial division rule of training set: test set = 8:7, when the number of historically annotated barcode images is 15,000, the training set is 8,000, and the test set is 7,000; the preset division rule may be an artificial division rule of training set: test set = 1:1, when the number of historically annotated barcode images is 20,000, the training set is 10,000, and the test set is 10,000. The division method of the preset division rule is not limited here, and can be set according to actual needs.

[0078] Specifically, the processor can fuse the target detection model and the landmark detection model to obtain the corresponding barcode corner point coordinate detection model, obtain the historical barcode image, and mark the corner point coordinates of the historical barcode image to obtain the corresponding historical marked barcode image, and divide the historical marked barcode image according to the preset division rules to obtain the corresponding training set and test, and input the training set to the barcode corner point coordinate detection model for training.

[0079] In some embodiments, the target detection model is a YOLO model, and the processor can fuse the YOLO model and the landmark detection model to obtain a corresponding barcode corner coordinate detection model, obtain a historical barcode image, and mark the corner coordinates of the historical barcode image to obtain the corresponding historical marked barcode image, and divide the historical marked barcode image according to a preset division rule to obtain a corresponding training set and test, and input the training set to the barcode corner coordinate detection model for training.

[0080] In some embodiments, the target detection model is an SSD model, and the processor can fuse the SSD model and the landmark detection model to obtain a corresponding barcode corner point coordinate detection model, obtain a historical barcode image, and mark the corner point coordinates of the historical barcode image to obtain the corresponding historical marked barcode image, and divide the historical marked barcode image according to a preset division rule to obtain a corresponding training set and test, and input the training set to the barcode corner point coordinate detection model for training.

[0081] In some embodiments, the target detection model is a RetinaNet model, and the processor can fuse the RetinaNet model and the Landmark detection model to obtain a corresponding barcode corner coordinate detection model, obtain a historical barcode image, and annotate the corner coordinates of the historical barcode image to obtain the corresponding historical annotated barcode image, and divide the historical annotated barcode image according to a preset division rule to obtain a corresponding training set and test, and input the training set to the barcode corner coordinate detection model for training.

[0082] In some embodiments, the target detection model is a Faster R-CNN model, and the processor can fuse the Faster R-CNN model and the Landmark detection model to obtain a corresponding barcode corner coordinate detection model, obtain a historical barcode image, and annotate the corner coordinates of the historical barcode image to obtain the corresponding historical annotated barcode image, and divide the historical annotated barcode image according to a preset division rule to obtain a corresponding training set and test, and input the training set to the barcode corner coordinate detection model for training.

[0083] Step S230: Determine whether the barcode corner point coordinate detection model converges.

[0084] In this embodiment, after constructing and training the barcode corner point coordinate detection model, the processor may determine whether the barcode corner point coordinate detection model has converged.

[0085] Specifically, after building and training the barcode corner point coordinate detection model, the processor can input a test set into the barcode corner point coordinate detection model. The barcode corner point coordinate detection model receives and responds to the test set, tests the test set, obtains the corresponding test results, and determines the test accuracy corresponding to the test results based on the test results and the test set, and determines whether the barcode corner point coordinate detection model has converged based on the test accuracy.

[0086] When the test accuracy is greater than or equal to the preset accuracy threshold, it is determined that the barcode corner point coordinate detection model has converged; when the test accuracy is less than the preset accuracy threshold, it is determined that the barcode corner point coordinate detection model has not converged.

[0087] Step S240: When it is determined that the barcode corner point coordinate detection model has converged, the barcode image is input into the barcode corner point coordinate detection model to obtain the corresponding barcode corner point coordinates.

[0088] In this embodiment, when the processor determines that the barcode corner point coordinate detection model has converged, the processor can input the barcode image into the barcode corner point coordinate detection model. The barcode corner point coordinate detection model receives and responds to the barcode image, detects the barcode corner point coordinates corresponding to the barcode image, and outputs the barcode corner point coordinates to the processor. The processor receives the barcode corner point coordinates output by the barcode corner point coordinate detection model, and realizes the detection of the barcode corner point coordinates corresponding to the barcode image when the barcode corner point coordinate detection model has converged according to the test set, thereby ensuring the stability of the barcode corner point coordinate detection model and improving the credibility of the barcode corner point coordinates.

[0089] Step S250: Determine the target barcode area corresponding to the barcode image according to the barcode corner point coordinates.

[0090] Step S260: performing a decoding operation on the target barcode area to obtain corresponding barcode information.

[0091] In this embodiment, step S250 and step S260 may refer to the contents of the corresponding steps in the aforementioned embodiment, which will not be described in detail here.

[0092] The solution provided in this embodiment obtains a barcode image, constructs and trains a barcode corner point coordinate detection model, determines whether the barcode corner point coordinate detection model converges, and when it is determined that the barcode corner point coordinate detection model converges, inputs the barcode image into the barcode corner point coordinate detection model to obtain the corresponding barcode corner point coordinates, and determines the target barcode area corresponding to the barcode image according to the barcode corner point coordinates, and performs a decoding operation on the target barcode area to obtain the corresponding barcode information. In the barcode recognition process, image recognition is performed on the target barcode area corresponding to the barcode image to obtain the corresponding barcode information, thereby avoiding the barcode recognition error caused by the recognition of background information in the non-barcode area, and improving the recognition accuracy of the barcode.

[0093] Furthermore, when the barcode corner point coordinate detection model is determined to converge according to the test set, the barcode corner point coordinates corresponding to the barcode image are detected, which ensures the stability of the barcode corner point coordinate detection model and improves the credibility of the barcode corner point coordinates.

[0094] See also Figure 6 , which shows a barcode recognition device 300 provided by an embodiment of the present application. The barcode recognition device 300 can be applied to Figure 1 The terminal device 100 in the barcode recognition system shown in FIG. 1 is taken as an example below. Figure 6 The barcode recognition device 300 shown is described in detail. The barcode recognition device 300 may include an acquisition module 310 , an input module 320 , an area determination module 330 and a decoding module 340 .

[0095] The acquisition module 310 can be used to acquire a barcode image; the input module 320 can be used to input the barcode image into a barcode corner point coordinate detection model to obtain corresponding barcode corner point coordinates, wherein the barcode corner point coordinate detection model includes a target detection model and a feature point detection model; the area determination module 330 can be used to determine the target barcode area corresponding to the barcode image according to the barcode corner point coordinates; the decoding module 340 can be used to perform a decoding operation on the target barcode area to obtain corresponding barcode information.

[0096] In some implementations, the region determination module 330 may include a determination unit and a transformation processing unit.

[0097] The determination unit may be used to determine the initial barcode region corresponding to the barcode image according to the barcode corner point coordinates; the transformation processing unit may be used to perform perspective transformation processing on the initial barcode region to obtain the corresponding target barcode region.

[0098] In some embodiments, the transform processing unit may include a transform processing subunit and a sharpening processing subunit.

[0099] The transformation processing subunit can be used to perform perspective transformation processing on the initial barcode area to obtain a corresponding perspective processed barcode area; the sharpening processing subunit can be used to perform sharpening processing on the perspective processed barcode area to obtain a corresponding target barcode area.

[0100] In some embodiments, the sharpening process may be a filtering process, and the sharpening process sub-unit may include a filtering process sub-sub-unit.

[0101] The filtering processing sub-unit can be used to filter the perspective processed barcode area to obtain the corresponding target barcode area, the boundary filter kernel value corresponding to the filtering processing is less than zero, the intermediate filter kernel value corresponding to the filtering processing is greater than zero, and the absolute value of the intermediate filter kernel value is greater than the absolute value of the boundary filter kernel value.

[0102] In some implementations, the decoding module 340 may include an affine processing unit and a decoding unit.

[0103] The affine processing unit can be used to perform three-dimensional affine transformation processing on the target barcode area to obtain the corresponding three-dimensional affine barcode area; the decoding unit can be used to perform decoding operation on the three-dimensional affine barcode area to obtain the corresponding barcode information.

[0104] In some implementations, the affine processing unit may include an affine processing sub-unit.

[0105] The affine processing subunit can be used to perform three-dimensional affine transformation processing on the target barcode area in multiple affine angle directions to obtain multiple corresponding three-dimensional affine barcode areas, each of which can correspond to an affine angle direction.

[0106] In some embodiments, the decoding unit may include a decoding sub-unit.

[0107] The decoding subunit may be used to perform a decoding operation on each three-dimensional affine barcode region in turn until the corresponding barcode information is obtained.

[0108] In some embodiments, the target detection model includes a first branch and a second branch, the first branch is used to detect the barcode type of the barcode image, and the second branch is used to detect the barcode frame information in the barcode image.

[0109] In some implementations, the feature point detection model includes a third branch, and the third branch is used to detect the coordinates of the barcode corner points in the barcode image.

[0110] In some implementations, the barcode recognition device 300 may further include a construction module and a convergence determination module.

[0111] The construction module can be used for inputting the barcode image into the barcode corner coordinate detection model through the input module 320, and constructing and training the barcode corner coordinate detection model before obtaining the corresponding barcode corner coordinates; the convergence determination module can be used for determining whether the barcode corner coordinate detection model converges.

[0112] In some implementations, the input module 320 may include an input unit.

[0113] The input unit can be used to input the barcode image to the barcode corner point coordinate detection model to obtain the corresponding barcode corner point coordinates when the convergence determination module determines that the barcode corner point coordinate detection model has converged.

[0114] The solution provided in this embodiment obtains the corresponding barcode corner coordinates by acquiring a barcode image and inputting the barcode image into a barcode corner coordinate detection model. The barcode corner coordinate detection model includes a target detection model and a feature point detection model. According to the barcode corner coordinates, the target barcode area corresponding to the barcode image is determined, and a decoding operation is performed on the target barcode area to obtain the corresponding barcode information. In the barcode recognition process, image recognition is performed on the target barcode area corresponding to the barcode image to obtain the corresponding barcode information, thereby avoiding the barcode recognition error caused by the recognition of background information in the non-barcode area, and improving the recognition accuracy of the barcode.

[0115] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Any processing method described in the method embodiment can be implemented by the corresponding processing module in the device embodiment, and will not be repeated one by one in the device embodiment.

[0116] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0117] See also Figure 7, which shows a functional block diagram of an electronic device 400 provided by an embodiment of the present application, and the electronic device 400 may include one or more of the following components: a memory 410, a processor 420, and one or more applications, wherein the one or more processors 420 are coupled to the memory 410, the one or more applications may be stored in the memory 410 and configured to be executed by the one or more processors 420, and the one or more applications are configured to execute the method described in the aforementioned method embodiment.

[0118] The memory 410 may include a random access memory (RAM) or a read-only memory (ROM). The memory 410 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 410 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as obtaining a barcode image, inputting a barcode image, obtaining barcode corner coordinates, determining a target barcode area, performing a decoding operation, obtaining barcode information, determining an initial barcode area, perspective transformation processing, obtaining a target barcode area, obtaining a perspective processed barcode area, sharpening processing, filtering processing, three-dimensional affine transformation processing, obtaining a three-dimensional affine barcode area, constructing a barcode corner coordinate detection model, training a barcode corner coordinate detection model, determining whether the barcode corner coordinate detection model converges, and determining whether the barcode corner coordinate detection model converges, etc.), instructions for implementing the following various method embodiments, etc. The storage data area can also store data created by the electronic device 400 during use (such as barcode images, barcode corner point coordinate detection models, barcode corner point coordinates, target detection models, feature point detection models, target barcode areas, barcode information, initial barcode areas, perspective processed barcode areas, boundary filter kernel values, intermediate filter kernel values, three-dimensional affine barcode areas, multiple affine angle directions, multiple three-dimensional affine barcode areas, first branch, second branch, barcode type, barcode frame information, third branch), etc.

[0119] The processor 420 may include one or more processing cores. The processor 420 uses various interfaces and lines to connect various parts of the entire electronic device 400, and executes various functions and processes data of the electronic device 400 by running or executing instructions, programs, code sets or instruction sets stored in the memory 410, and calling data stored in the memory 410. Optionally, the processor 420 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), programmable logic array (PLA), etc. The processor 420 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 420, but may be implemented separately through a communication chip.

[0120] Please refer to Figure 8 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 500 stores a program code 510, which can be called by a processor to execute the method described in the above method embodiment.

[0121] The computer-readable storage medium 500 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer-readable storage medium 500 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 500 has storage space for program code 510 that performs any method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code 510 can be compressed, for example, in an appropriate form.

[0122] Please refer to Fig. 9, which shows a structural block diagram of a computer program product 600 provided in an embodiment of the present application. The computer program product 600 includes a computer program / instruction 610, and the computer program / instruction 610 is stored in a computer-readable storage medium of a computer device. When the computer program product 600 is run on a computer device, a processor of the computer device reads the computer program / instruction 610 from the computer-readable storage medium, and the processor executes the computer program / instruction 610, so that the computer device performs the method described in the above method embodiment.

[0123] The solution provided in this embodiment obtains the corresponding barcode corner coordinates by acquiring a barcode image and inputting the barcode image into a barcode corner coordinate detection model. The barcode corner coordinate detection model includes a target detection model and a feature point detection model. According to the barcode corner coordinates, the target barcode area corresponding to the barcode image is determined, and a decoding operation is performed on the target barcode area to obtain the corresponding barcode information. In the barcode recognition process, image recognition is performed on the target barcode area corresponding to the barcode image to obtain the corresponding barcode information, thereby avoiding the barcode recognition error caused by the recognition of background information in the non-barcode area, and improving the recognition accuracy of the barcode.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A barcode recognition method, characterized in that: include: Get the barcode image; Inputting the barcode image into a barcode corner point coordinate detection model to obtain corresponding barcode corner point coordinates, wherein the barcode corner point coordinate detection model includes a target detection model and a feature point detection model; Determining a target barcode area corresponding to the barcode image according to the barcode corner point coordinates; A decoding operation is performed on the target barcode area to obtain corresponding barcode information.

2. The barcode recognition method according to claim 1, characterized in that: The step of determining the target barcode area corresponding to the barcode image according to the barcode corner point coordinates includes: Determining an initial barcode area corresponding to the barcode image according to the barcode corner point coordinates; The initial barcode region is subjected to perspective transformation processing to obtain a corresponding target barcode region.

3. The barcode recognition method according to claim 2, characterized in that: The performing perspective transformation processing on the initial barcode area to obtain a corresponding target barcode area includes: Performing perspective transformation processing on the initial barcode area to obtain a corresponding perspective processed barcode area; The perspective processed barcode area is sharpened to obtain a corresponding target barcode area.

4. The barcode recognition method according to claim 3, characterized in that: The sharpening process is a filtering process, and the sharpening process is performed on the perspective processed barcode area to obtain the corresponding target barcode area, including: The filtering process is performed on the perspective processed barcode area to obtain a corresponding target barcode area, wherein a boundary filter kernel value corresponding to the filtering process is less than zero, an intermediate filter kernel value corresponding to the filtering process is greater than zero, and an absolute value of the intermediate filter kernel value is greater than an absolute value of the boundary filter kernel value.

5. The barcode recognition method according to claim 1, characterized in that: The performing a decoding operation on the target barcode area to obtain corresponding barcode information includes: Performing a three-dimensional affine transformation on the target barcode region to obtain a corresponding three-dimensional affine barcode region; A decoding operation is performed on the three-dimensional affine barcode area to obtain corresponding barcode information.

6. The barcode recognition method according to claim 5, characterized in that: The performing three-dimensional affine transformation processing on the target barcode area to obtain a corresponding three-dimensional affine barcode area includes: Performing a three-dimensional affine transformation process on the target barcode region in multiple affine angle directions to obtain multiple corresponding three-dimensional affine barcode regions, each of which corresponds to an affine angle direction; The performing a decoding operation on the three-dimensional affine barcode area to obtain corresponding barcode information includes: The decoding operation is performed on each three-dimensional affine barcode area in turn until the corresponding barcode information is obtained.

7. The barcode recognition method according to claim 1, characterized in that: The target detection model includes a first branch and a second branch, the first branch is used to detect the barcode type of the barcode image, and the second branch is used to detect the barcode frame information in the barcode image.

8. The barcode recognition method according to claim 1, characterized in that: The feature point detection model includes a third branch, and the third branch is used to detect the coordinates of the barcode corner points in the barcode image.

9. The barcode recognition method according to any one of claims 1 to 8, characterized in that: Before inputting the barcode image into the barcode corner point coordinate detection model to obtain the corresponding barcode corner point coordinates, the barcode recognition method further includes: Build and train a barcode corner coordinate detection model; Determining whether the barcode corner point coordinate detection model converges; The step of inputting the barcode image into a barcode corner point coordinate detection model to obtain corresponding barcode corner point coordinates includes: When it is determined that the barcode corner point coordinate detection model has converged, the barcode image is input into the barcode corner point coordinate detection model to obtain corresponding barcode corner point coordinates.

10. A barcode recognition device, characterized in that: include: An acquisition module, used for acquiring a barcode image; An input module, used to input the barcode image into a barcode corner point coordinate detection model to obtain corresponding barcode corner point coordinates, wherein the barcode corner point coordinate detection model includes a target detection model and a feature point detection model; An area determination module, used to determine the target barcode area corresponding to the barcode image according to the coordinates of the barcode corner points; The decoding module is used to perform a decoding operation on the target barcode area to obtain corresponding barcode information.

11. An electronic device, characterized in that: include: Memory; One or more processors coupled to the memory; One or more application programs, wherein the one or more application programs are stored in the memory and are configured to be executed by one or more processors, and the one or more application programs are configured to execute the barcode recognition method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the barcode recognition method according to any one of claims 1 to 9.

Images