Method, device and storage medium for automatically identifying the box label number of a consignment

Abstract

Disclosed are a method and device for automatically identifying the box label number of a consignment. A method for automatically identifying the box label number of a consignment, wherein the box label number uniquely identifies the consignment, comprises: receiving a picture of a box label of the consignment; detecting text blocks in the picture of the box label by using a first neural network model; identifying text in each text block by using a second model; searching for at least one character string of a predetermined length and containing only numbers and spaces in the identified text by using a sliding window of a predetermined length; locating at least one barcode in the picture of the box label; and outputting one of the found character strings as the box label number based on the relative positional relationship between the found character string and the barcode.

Description

Technical Field

[0001] The present disclosure relates to a method and device and a storage medium for automatically identifying a box label number of a consignment. Background Art

[0002] The consignment includes paper boxes, wooden boxes, etc. Usually, each consignment is affixed with a box label, on which the shipping address, destination address, shipping time, weight, etc. of the consignment are marked. In addition, the box label is usually marked with a box label number, which uniquely identifies the consignment.

[0003] When managing the consignment, the length, width and height of the consignment are usually measured, and the measurement results are stored in association with the box label number. At this time, the manager needs to manually input the box label number, which is troublesome for the manager. Summary of the invention

[0004] An object of the present invention is to provide a new method and device for automatically identifying box label numbers.

[0005] The present disclosure proposes a method for automatically identifying a box label number of a consignment, where the box label number uniquely identifies the consignment, and the method comprises: receiving a picture of a box label of the consignment; detecting text blocks in the picture of the box label by using a first neural network model; identifying text in each text block by using a second model; searching for at least one character string of the predetermined length and containing only numbers and spaces in the identified text by using a sliding window of the predetermined length; locating at least one barcode in the picture of the box label; and outputting one of the found character strings as the box label number based on the relative positional relationship between the found character string and the barcode.

[0006] Other features and advantages of the present disclosure will become apparent from the following description with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure without limitation. In the various figures, like reference numerals are used to represent like items.

[0008] Figure 1 is a block diagram of an exemplary automatic box label number recognition device according to some embodiments of the present disclosure.

[0009] Figure 2 is a flowchart illustrating an exemplary method for automatically identifying box label numbers according to some embodiments of the present disclosure.

[0010] Figure 3A picture of an exemplary shipper's box label is shown.

[0011] Figure 4 The diagram illustrates a general hardware environment in which the present disclosure may be applied according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0012] In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the described exemplary embodiments. However, it is clear to those skilled in the art that the described embodiments may be practiced without some or all of these specific details. In the described exemplary embodiments, well-known structures or processing steps are not described in detail in order to avoid unnecessarily obscuring the concepts of the present disclosure.

[0013] The blocks in each block diagram shown below can be implemented by hardware, software, firmware or any combination thereof to implement the principles of the present disclosure. It should be understood by those skilled in the art that the blocks described in each block diagram can be combined or divided into sub-blocks to implement the principles of the present disclosure.

[0014] The steps of the method presented in this disclosure are intended to be illustrative. In certain embodiments, the method can be completed with one or more additional steps not described and / or completed without the one or more steps discussed. In addition, the order in which the steps of the method are illustrated and described is not intended to be restrictive.

[0015] In the present disclosure, text may include words, numbers, letters, symbols, etc. Characters may include numbers, letters, symbols, etc.

[0016] Figure 1 is a block diagram of an exemplary automatic box label number recognition device 100 according to some embodiments of the present disclosure.

[0017] like Figure 1 As shown, the device 100 may include: a receiving component 110, configured to receive a picture of a box label of a consignment; a text block detection component 120, configured to detect text blocks in the picture of the box label by using a CTPN (Connectionist Text Candidate Network) model; a text recognition component 130, configured to recognize text in each text block by using a Tesseract model; a sliding window search component 140, configured to search for at least one character string of a predetermined length and containing only numbers and spaces in the recognized text by using a sliding window of a predetermined length; a barcode positioning component 150, configured to locate at least one barcode in the picture of the box label; and an output component 160, configured to output one of the found character strings as a box label number based on the relative position relationship between the found character string and the barcode.

[0018] The following will further describe in detail Figure 1 The operation of each component is shown.

[0019] Figure 2 FIG. 2 is a flowchart illustrating an exemplary box label number automatic identification method 200 according to some embodiments of the present disclosure.

[0020] The method 200 starts from step S210 . In step S210 , the receiving component 110 receives a picture of a box label of a consignment.

[0021] The image of the box label of the consignment can be obtained by taking a picture with a camera. For example, a smart device with a camera (a smart phone, a smart tablet, etc.) can be used to take a picture of the box label of the consignment, thereby obtaining the image of the box label of the consignment. The receiving component 110 can receive or obtain the image of the box label of the consignment from the camera.

[0022] By photographing the box sticker of the consignment item while making the box sticker of the consignment item fall within the viewfinder of the camera, a picture of the box sticker of the consignment item is obtained, wherein the image of the box sticker occupies more than half of the area of ​​the picture. In some embodiments, the image of the box sticker may occupy more than 70%, 80% or 90% of the area of ​​the picture. In some embodiments, the box sticker is photographed while being in the middle part of the viewfinder. In this way, it is ensured that the complete box sticker is photographed. In some embodiments, the camera is photographed while facing the box sticker. Alternatively, the camera may be photographed while the imaging surface of the camera forms a small angle (such as an angle of less than 30°) with the plane where the box sticker is located. In this way, excessive distortion and deformation of the box sticker image can be avoided. In some embodiments, the viewfinder may have any appropriate shape, such as a rectangle, a square, etc. Alternatively, the viewfinder may have a shape similar to that of an actual box sticker, such as a viewfinder may have an aspect ratio similar to that of an actual box sticker, so as to assist in better photographing the box sticker. The box sticker may have a uniform size and shape. Here, similar aspect ratios mean that the difference between the aspect ratios of the two is within the range of ±5% of the aspect ratio of the box label.

[0023] Figure 3The figure shows a picture of an exemplary consignment box label. It can be seen that a plurality of information items are printed on the box label, including: the box label number "190 206 168 1124", the consignment weight "62.000" (unit: kg), the consignment's shipping address "RDC BBA Chengdu, 800, Logistic Avenue, Xihanggang, 610207, Chengdu, People's Republic of China", the consignment's destination address "Final Destination: 66913, BMW Brilliance Automotive Ltd., RDC BBA Shanghai, CN", the box label generation date and time (i.e., shipping time) "19.06.2019 12:00", several bar codes, and the like. In the present disclosure, the box label number only contains numbers and spaces and has a predetermined length. Here, in the case where one number is regarded as one digit, the box label number has a length of 16 digits.

[0024] It should be understood that Figure 3 The box stickers shown are only exemplary, and the size, shape, and layout of information items on the box stickers may vary. The information items included in the box stickers are not limited to the information items shown, but may be increased or decreased according to actual needs. Figure 3 Only the main part of the box sticker is shown. It should be understood that the picture received in step S210 may include the image of the box sticker and the background image, as long as the image of the box sticker occupies more than half of the area of ​​the received picture.

[0025] In some embodiments, the received box sticker picture can be pre-processed. The size, brightness, contrast, etc. of the picture can be adjusted. The picture can also be rotated, cropped, etc.

[0026] Next, the method 200 proceeds to step S220, where the text block detection component 120 detects the text block in the box sticker picture by using the CTPN model. The CTPN model is a known neural network model. Here, the box sticker picture received at step S210 is input into the CTPN model, and the CTPN model outputs the detected text block. It should be noted that the CTPN model used here is a pre-trained model, and how to train the CTPN model will be described later.

[0027] by Figure 3 Take the box sticker picture shown in the figure as an example. Figure 3When the box sticker picture shown is input into the CTPN model, the CTPN model outputs all the text blocks in the box sticker picture. For example, the CTPN model can detect the box sticker number "190 206168 1124" as one or more text blocks, can detect each line of the address content as a text block, and can detect the generation date and time of the box sticker as one or more text blocks.

[0028] Next, the method 200 proceeds to step S230, where the text recognition component 130 recognizes the text in each text block by using the Tesseract model. The Tesseract model is a known deep learning model. Here, the text block detected at step S220 is input into the Tesseract model, and the text in each identified text block is output by the Tesseract model. The Tesseract model is excellent in recognizing letters, numbers, symbols, etc., and there is no need to pre-train this model when using this model to recognize letters, numbers, symbols, etc.

[0029] Specifically, the Tesseract model will output numbers, letters, symbols, and so on in each text block.

[0030] Next, the method 200 proceeds to step S240, where the sliding window search component 140 searches for at least one character string of a predetermined length and containing only numbers and spaces in the recognized text by using a sliding window of a predetermined length. More specifically, the sliding window may have a length of 16 bits, and the sliding window is used to search for a 16-bit character string containing only numbers and spaces, i.e., a box label number.

[0031] Still Figure 3 Taking the box sticker picture shown as an example, a sliding window with a length of 16 bits is used to traverse all the texts identified in step S230. When the content within the sliding window is only numbers and spaces, the found content is determined as a candidate box sticker number. In some embodiments, a string with a predetermined length, only containing numbers and spaces, and with numbers in the first and last digits, can be found. It should be understood that one or more strings may be found.

[0032] The position of the box sticker number in the actual box sticker may be fixed. Therefore, a sliding window may be used to search for a target string based on the position of the box sticker number in the box sticker. In some embodiments, a sliding window may be used to traverse the recognized text that is only in the upper half of the box sticker image to search for the target string, because the box sticker number is located in the upper half of the box sticker, and therefore the image of the box sticker number is unlikely to be located in the lower half of the box sticker image. In other embodiments, when the box sticker image in the box sticker image can be located, a sliding window may be used to traverse the text at a predetermined position (or predetermined area) of the box sticker image to search for the box sticker number.

[0033] Next, the method 200 proceeds to step S250, where the barcode positioning component 150 locates at least one barcode in the box sticker picture. Specifically, the barcode positioning component 150 performs expansion and corrosion processing on the box sticker picture received at step S210, performs connected domain detection on the box sticker picture after the expansion and corrosion processing, and determines the connected domain with an area greater than a preset value as the located barcode. The expansion and corrosion processing and the connected domain detection processing are known image processing methods. After the expansion and corrosion processing and the connected domain detection processing, the barcode in the box sticker picture will be detected as a connected domain, and some character strings (such as "190") may also be detected as a connected domain. Therefore, only the connected domain with an area greater than the preset value is determined as a barcode. For example, only the connected domain with an area greater than half (or 60%, etc.) of the area of ​​the largest connected domain can be determined as a barcode area.

[0034] As mentioned later, step S205 is intended to locate the barcodes adjacent to the box label number, rather than to accurately locate all the barcodes. Therefore, step S205 only needs to locate a portion of the barcodes with a larger area. Here, the preset value for screening barcodes can be a fixed value set according to experience (or a fixed ratio relative to the area of ​​the largest connected domain).

[0035] Next, the method 200 proceeds to step S260, where the output component 160 outputs one of the found character strings as the box label number based on the relative position relationship between the found character strings and the barcode. In addition, the output component 160 also outputs a digital string located at a predetermined position in the picture of the box label and recognized by the Tesseract model as the weight of the consignment.

[0036] The output component 160 can compare one or more first positions of at least one string found in step S240 in the box label image with one or more second positions of at least one barcode located in step S250 in the box label image, and when the relative position relationship between a first position and one or more second positions meets a preset condition, output the string located at this first position as the box label number.

[0037] by Figure 3 For example, the output component 160 can output a string of characters that is located adjacent to a barcode as the box label number. More specifically, the output component 160 can determine that a first position and a second position are close to each other when the distance between the two positions is less than a preset value, and further determine that the two positions are adjacent. Here, the preset value can be an empirical value. It should be understood that the layout of the information items in the box label can change. When the layout changes and is known, the box label number can be determined based on the relative position relationship between the box label number and one or more barcodes.

[0038] It should be understood that even if only one character string is found in step S240, the character string can be confirmed to be the box label number by determining the relative position relationship between the character string and one or more bar codes. By verifying the relative position relationship between the character string and the bar code, the box label number can be identified more reliably.

[0039] In addition, Figure 3 Taking the box sticker image of the container as an example, the output component 160 can output the numeric string located at the bottom and leftmost of the box sticker image in the text recognized by the Tesseract model as the weight of the consignment. This numeric string representing the weight only contains numbers and symbols (such as decimal points) and can have any length or precision.

[0040] It should be understood that the present disclosure is not limited to this. When the digital string representing weight is located at other positions in the box sticker image, since this string has a fixed position in the box sticker image, the method according to the present disclosure can still find it based on position.

[0041] In some embodiments, the output component 160 can output the found box label number and the found weight digital string to a predefined interface. The predefined interface can be, for example, an output interface of a software application for consignment management. For example, the length, width and height data of the consignment, the box label number of the consignment, and the weight data of the consignment can be output via the output interface. This greatly facilitates the management of the consignment. In addition, by using the box label number that uniquely identifies the consignment, the user of the method 200 (such as the consignment manager) can easily retrieve the relevant information of the consignment, such as the consignment's shipping address, destination address, logistics information, what kind of parts are consigned, the purpose of the parts, etc.

[0042] The following describes how to train the CTPN model. The training of the CTPN model can be performed before the method 200 starts.

[0043] A large number of images of consignment box stickers (e.g., 2000-4000 images) are used to train the CTPN model. These images can be images of box stickers taken on the spot. Marks (or labels) can be added manually to each image. The added marks can be marks that mark the text blocks in the box sticker images, such as a rectangular box that marks the position and size of the text block and a mark that indicates that the rectangular box represents the text, such as a mark "1" that indicates text and a mark "0" that indicates non-text. The CTPN model can be trained using consignment box sticker images with marks so that the text blocks detected by the CTPN model are as consistent as possible with the marked text blocks.

[0044] It should be understood that the above introduces the use of the CTPN model as a text block detection model, and the Tesseract model as a text recognition model. However, the present disclosure is not limited to this. For text block detection, in addition to the CTPN model, other neural network models can also be used, such as RRPN (rotation region candidate box network) model, DMPNet (deep matching prior network) model, SegLink (slice link) model, TextBoxes (text box) model, etc. For text recognition, in addition to the Tesseract model, other deep learning models or neural network models can also be used, such as CRNN (convolutional recurrent neural network) model, FOTS (fast directional text location) model, STN-OCR (spatial transform network-optical character recognition) model, etc.

[0045] Reference above Figure 1 and Figure 2An exemplary method and device for automatically identifying box label numbers according to the present disclosure are introduced. It can be understood that the present disclosure realizes automatic identification of box label numbers. By applying the method of the present disclosure to software applications such as those used for consignment management, the burden of managers needing to manually input box label numbers is eliminated. In addition, by displaying and / or storing the identification number (box label number) of the consignment, the weight of the consignment, the length, width, and height data of the consignment, etc. in an associated manner, more complete management of the consignment is achieved.

[0046] Hardware Implementation

[0047] Figure 4 A general hardware environment 400 is shown in which the present disclosure may be applied according to an exemplary embodiment of the present disclosure.

[0048] refer to Figure 4 , a computing device 400 will now be described as an example of a hardware device applicable to various aspects of the present disclosure. The computing device 400 may be any machine configured to perform processing and / or computing, and may be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal digital assistant, a smart phone, a portable camera, or any combination thereof. The above-mentioned apparatus 100 may be implemented in whole or at least in part by the computing device 400 or a similar device or system.

[0049] The computing device 400 may include an element that can be connected to or communicate with the bus 402 via one or more interfaces. For example, the computing device 400 may include a bus 402, one or more processors 404, one or more input devices 406, and one or more output devices 408. The one or more processors 404 may be any type of processor, and may include, but are not limited to, one or more general-purpose processors and / or one or more special-purpose processors (such as special-purpose processing chips). The input device 406 may be any type of device that can input information to the computing device, and may include, but are not limited to, a mouse, a keyboard, a touch screen, a microphone, and / or a remote controller. The output device 408 may be any type of device that can present information, and may include, but are not limited to, a display, a speaker, a video / audio output terminal, and / or a printer. The computing device 400 may also include or be connected to a non-transient storage device 410, which may be any storage device that is non-transient and can implement a data repository, and may include but is not limited to a disk drive, an optical storage device, a solid-state storage device, a floppy disk, a flexible disk, a hard disk, a magnetic tape or any other magnetic medium, a compact disk or any other optical medium, a ROM (read-only memory), a RAM (random access memory), a cache memory and / or any other memory chip or cartridge, and / or any other medium from which a computer can read data, instructions and / or code. The non-transient storage device 410 may be detachable from the interface. The non-transient storage device 410 may have data / instructions / code for implementing the above methods and steps. The computing device 400 may also include a communication device 412. The communication device 412 may be any type of device or system capable of communicating with an external device and / or with a network, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication equipment and / or a communication device such as Bluetooth. TM Chipsets for devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication facilities, etc.

[0050] The bus 402 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0051] Computing device 400 may also include working memory 414, which may be any type of working memory that can store instructions and / or data useful for operation of processor 404, and may include, but is not limited to, random access memory and / or read-only memory devices.

[0052] Software elements may be located in the working memory 414, including but not limited to an operating system 416, one or more application programs 418, a driver and / or other data and code. Instructions for executing the above methods and steps may be included in one or more application programs 418, and the components of the above device 100 may be implemented by the processor 404 reading and executing the instructions of one or more application programs 418. More specifically, the text block detection component 120 may be implemented, for example, by the processor 404 when executing the application program 418 having an instruction to execute step S220. The text recognition component 130 may be implemented, for example, by the processor 404 when executing the application program 418 having an instruction to execute step S230. The sliding window search component 140 may be implemented, for example, by the processor 404 when executing the application program 418 having an instruction to execute step S240. The barcode positioning component 150 may be implemented, for example, by the processor 404 when executing the application program 418 having an instruction to execute step S250. Similarly, the receiving component 110 and the output component 160 can be implemented, for example, by the processor 404 when executing the application program 418 having instructions for performing steps S210 and S260, respectively. The executable code or source code of the instructions of the software element can be stored in a non-transitory computer-readable storage medium (such as the above-mentioned (one or more) storage devices 410) and can be read into the working memory 414 in the case of possible compilation and / or installation. The executable code or source code of the instructions of the software element can also be downloaded from a remote location.

[0053] From the above embodiments, it will be clear to those skilled in the art that the present disclosure can be implemented by software and necessary hardware, or can be implemented by hardware, firmware, etc. Based on this understanding, the embodiments of the present disclosure can be partially implemented in the form of software. Computer software can be stored in a computer-readable storage medium, such as a floppy disk, a hard disk, an optical disk, or a flash memory. Computer software includes a series of instructions that enable a computer (such as a personal computer, a service station, or a network terminal) to run a method according to each embodiment of the present disclosure or a part thereof.

[0054] The disclosure having thus been described, it will be apparent that the same may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the disclosure, and all such modifications as would be obvious to one skilled in the art are intended to be included within the scope of the following claims.

Claims

1. A method for automatically identifying a consignment's box label number, the box label number uniquely identifying the consignment, characterized in that: include: Picture of the box sticker for receiving the consignment, By using the first neural network model to detect text blocks in the box sticker picture, By using the second model to identify the text in each text block, By traversing the recognized text using a sliding window with a predetermined length, at least one character string with the predetermined length and containing only numbers and spaces in the recognized text is searched, Locate at least one barcode in the image of the box sticker, and Based on the relative position relationship between the found character string and the barcode, one of the found character strings is output as the box label number. Wherein, at least one barcode in the image of the positioning box label includes: Perform expansion and corrosion processing on the box sticker picture. Perform connected domain detection on the box sticker image after expansion and corrosion processing. A connected domain whose area is larger than a preset value is determined as the located barcode.

2. The method according to claim 1, further comprising: The numerical string located at a predetermined position in the image of the box label and identified by the second model is output as the weight of the consignment.

3. The method according to claim 1, wherein: By photographing the box label of the consignment while making the box label fall within the viewfinder of the camera, a picture of the box label of the consignment is obtained, wherein the image of the box label occupies more than half of the area of ​​the picture.

4. The method according to claim 2, wherein: The output further includes: outputting the character string as a box label number and the digital string as the weight of the consignment to a predefined interface.

5. The method according to claim 1, further comprising: The first neural network is trained using pictures of box labels of marked consignments to minimize the difference between the detected text blocks and the markings that mark the text blocks in the pictures of the box labels.

6. The method according to claim 1, wherein: The first neural network model includes a CTPN model, and the second model includes a Tesseract model.

7. A device for automatically identifying the box label number of a consignment, characterized in that: include: Means for performing the method according to any one of claims 1-6.

8. A device for automatically identifying the box label number of a consignment, characterized in that: include: at least one processor; as well as At least one storage device stores instructions, and when the instructions are executed by the at least one processor, the at least one processor performs the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium, characterized in that: Instructions are stored which, when executed by a processor, cause the method according to any one of claims 1 to 6 to be performed.

Images