Method, mobile device, and analysis and processing computer for generating shipping information

Abstract

A method for generating shipping information is proposed, in which at least one photo of a transport carrier (1) is taken by a mobile device (2). In the mobile device (2), the camera information is analyzed and processed by a first artificial intelligence system in terms of the recognition of transport information (4), and in the case where the transport information (4) is recognized, at least one photo is taken by the mobile device (2). The at least one photo is transmitted to an analysis and processing computer (3), and in the analysis and processing computer (3), the at least one photo is analyzed and processed by a second artificial intelligence system in terms of the recognition of optical transport information (4). The recognized transport information (4) is analyzed and processed in terms of content, and the content of the transport information (4) is used together with other inputs on the mobile device (2) to generate shipping information.

Description

Field of the Invention

[0001] The present invention starts from a method for generating shipping information (Versandinformation) or a mobile device and an analysis computer for executing the method. Background Art

[0002] It is already known to generate shipping information by analyzing photos of transport carriers. Summary of the Invention

[0003] In contrast, the method for generating shipping information according to the present invention, or the mobile device and the analysis computer for executing the method, have the following advantages: The individual steps of the method can be assigned to the mobile device or the analysis computer particularly efficiently. A particularly simple mobile device is proposed in which the amount of information transmitted from the mobile device to the analysis computer can be kept as low as possible. Here, the method according to the present invention is configured by an artificial intelligence system present in the mobile device such that important relevant information is identified and these information are detected and transmitted only by means of photos. The analysis computer is configured such that it optimally analyzes and processes these limited information in order to identify and analyze important relevant transport information from the transmitted photos. Then, the transport information thus obtained is used to generate shipping information. By the optimized distribution between the mobile device and the analysis computer of the method, resources are optimally utilized in order to implement the method efficiently and cost-effectively.

[0004] Other improvement solutions and advantages are obtained from the following description. The training of the first and / or second artificial intelligence system is particularly easily carried out by using corresponding optically implemented transport information. The transport information is particularly easily synthetically generated, matched in terms of perspective, and then the transport information is fused with an image of the transport carrier. Here, in order to produce an optimal training effect, errors can in particular also be added. The extraction of important relevant information is in particular carried out by optical mark recognition. Thus, information implemented as letters, numbers or barcodes can in particular be analyzed and processed. In order to improve the data quality, the information thus obtained is compared with a database containing order information. This makes it possible to reliably obtain important relevant information. Brief Description of the Drawings

[0005] Embodiments of the present invention are shown in the drawings and further described in the specification.

[0006] In Figure 1 it, the method according to the present invention is illustrated with the aid of a transport carrier, a mobile device and an analysis computer;

[0007] Figure 2Shows the various steps of the method. Detailed implementation

[0008] In Figure 1 the transport carrier 1 is schematically shown, which is in particular configured as a pallet 6 here. A large number of packaging units - in particular cardboard boxes 5 - are arranged on the pallet 6, and each of these packaging units is provided with the following transport information 4: The transport information is optically readable, for example, by means of a label, a sticker or a printed matter. In the following, the term "label" is used to represent various forms of optically readable transport information 4. A large number of goods consignments for a determined recipient can be pooled together by means of the transport carrier 1 or the pallet 6 and jointly transmitted to the recipient. Here, it is usually necessary to provide the recipient with the following information: Which goods are being sent to him now by means of the transport carrier. These preliminary information enable the recipient of the goods in the cardboard box 5 to plan in advance, because he knows when he will receive these goods. In particular, when the goods sent in this way are for commercial purposes - for example, for supplying parts for production - such advance shipping information is important in order to be able to plan production.

[0009] In order to generate the shipping information sent to the recipient, in Figure 1 a mobile device 2 and an analysis and processing computer 3 are shown, which are connected to each other by means of an interface 7 (in particular a radio interface 7). The mobile device 2 is used to read the optically readable transport information 4 installed on the packaging unit 5 and transmit this transport information to the analysis and processing computer 3 via the interface 7. Here, only limited processing power is provided on the mobile device 2 and the main part of the data processing is carried out in the analysis and processing computer 3. However, in order to be able to achieve a meaningful detection of the transport information 4, limited data processing power is provided in the mobile device 2.

[0010] In Figure 1 the example, the transport carrier 1 relates, for example, to a pallet 6 on which cardboard boxes 5 are arranged as packaging units. However, alternatively, other transport carriers (such as containers, grid boxes or larger cardboard boxes with a plurality of packaging units 5) are suitable. If the transport carrier is configured such that the packaging unit 5 is not visible from the outside, a detection should be carried out when filling the transport carrier 1 with the packaging unit 5.

[0011] The mobile device 2 can in particular consist of a correspondingly programmed mobile phone or smartphone, which has a camera. However, alternatively, all mobile devices 2 that are provided only for this purpose can also be used. With regard to the mobile device 2, it is in particular envisaged that it is used by an operator, who is for example a warehouse employee who arranges the corresponding packaging unit 5 on the transport carrier 1. Alternatively, the mobile device 2 can also be part of an automated system for equipping the transport carrier 1 with the packaging unit 5, and the automated assembly device reads the transport information 4 at the end, for example. With regard to the interface 7, it is in particular envisaged that it is either a radio interface via WLAN or via a mobile radio standard. Alternatively, the mobile device 2 can also be placed in a master station after each use, in which a wired data transmission takes place between the mobile device 2 and the analysis computer 3.

[0012] The analysis computer 3 is a common data processing device that has sufficient capabilities to carry out the necessary steps of the method. Here, the analysis computer 3 does not necessarily have to be directly controlled by those who supply the packaging carrier 1 with the packaging unit 5. The analysis computer 3 can for example be controlled by a larger plant that is supplied by a large number of smaller transport companies. Each of these smaller transport companies then has a mobile device 2, which is usually just a correspondingly programmed mobile phone with a camera. The individual smaller transport companies then transmit the information transmitted via the mobile phone into the computing system of the larger plant, and the computing system then derives the shipping information from the information thus transmitted.

[0013] Data is transferred from the mobile device 2 to the analysis computer 3, especially by transmitting photos. When transmitting photos, it is not necessary to perform content analysis on the shipping information installed on the packaging unit 5 in the mobile device 2. However, because the data volume of photos is relatively large, it is desirable to transmit only those photos that contain actually important shipping information through the interface 7. For this purpose, the information recorded by the mobile device 2 can be preprocessed. For example, the operator of the mobile device 2 can start detecting important shipping information 4 after gathering the goods carried by the shipping carrier. Then, the mobile device 1 continuously analyzes the camera information in terms of whether the shipping information 4 (i.e., the optically readable shipping information 4 installed on the packaging unit 5) is recognized in the form of a label, sticker, or print. Such recognition of the label or shipping information 4 can be particularly easily carried out by a first artificial intelligence system that is only configured to recognize optically readable shipping information (such as a label). For example, a particularly simple recognition of a label can be to recognize a rectangular area that is different in color from the rest of the packaging unit 5 or from the shipping carrier 1. In addition, simple recognition of shipping information can be carried out by recognizing barcodes or QR codes. Whenever such a barcode or QR code is recognized, the simply configured artificial intelligence system in the mobile device 2 can recognize the shipping information. Then, whenever the shipping information 4 (i.e., the label) is recognized by analyzing the camera information, the mobile device 2 generates a photo of the optically readable shipping information 4 (i.e., the label). Other criteria - such as sufficient clarity or sufficient photo size - can also be ensured here.

[0014] In one configuration, it can also be indicated to the operator of the mobile device 2 which areas of the shipping carrier 1 are important. First, the shipping carrier 1 is photographed from a certain distance so that all the shipping information 4 visible from one side of the shipping carrier 1 can be seen. Based on the artificial intelligence system, rectangular labels 4 are identified and the operator is required to take a close-up of each of these labels. In this way, it can be ensured that the photos transmitted to the analysis computer 3 contain all the important information of the shipping carrier 1. Correspondingly, the operator of the mobile device 2 can also be required to take images of other sides (usually 4 sides in the case of a rectangular pallet) in order to obtain other shipping information on other sides of the shipping carrier 1.

[0015] The first artificial intelligence system present in the mobile device 2 does not have to be particularly powerful in performance and does not have to have a large data processing capacity here, but only has to identify the label on which the optically readable shipping information 4 is arranged. If an area of the shipping carrier 1 is misinterpreted as shipping information 4 here, this is not a problem because post-processing will also be carried out by the analysis computer 3. That is, content analysis is only carried out in the analysis computer 3.

[0016] By additional input on the mobile device 2, the packaging unit 5 can in particular be assigned to the transport carrier 1. This can be simply achieved, for example, by starting the detection of the transport carrier 1 by a first input on the mobile device 2 and ending the detection of the transport carrier 1 by another input when all the labels containing the transport information 4 have been photographed. Thus, by means of the said additional input (which can simply involve the start and end of the detection of the transport carrier 1), the assignment of the packaging unit 5 to the transport carrier 1 can be achieved.

[0017] After at least one photo (but usually several photos) has been transmitted and additional input has been made on the mobile device 2, analysis and processing are carried out by the analysis and processing computer 3. Then, the photos detected by the mobile device 2 photos are analyzed and processed in the analysis and processing computer 3. First of all, the identification of various transport information is carried out here. The transport information 4 is usually arranged on the label in different standardized arrangements. For example, the text information is located at a predefined distance from the barcode, and this text information repeats the information of the barcode or contains other information (such as a further description of the goods or quantity or sender or recipient contained). The type of label can be identified according to the arrangement of this information. An artificial intelligence system is also used in the analysis and processing unit 3, and this artificial intelligence system has been trained with information on the optical implementation of the transport information 4. However, such a system is significantly more powerful and can, for example, not only identify the outline of the label, but also identify what type of label it is and in which area of the label which information can be found.

[0018] The first artificial intelligence system in the mobile device 2, for example, cannot distinguish between two randomly arranged labels that are close together, and also cannot identify whether one label or two labels are involved. However, the artificial intelligence system of the analysis and processing computer 3 can still separate such labels arranged close together because the second artificial intelligence system is more powerful and can analyze and process additional information on the label structure. Therefore, in the second artificial intelligence system of the analysis and processing computer 3, the transport information 4 is distinguished in terms of the type of optical implementation in the label and content analysis and processing is carried out. For example, content analysis and processing can be achieved by the artificial intelligence system identifying the area containing the text information and then reading the text information generated therein by optical signal analysis and processing. Correspondingly, the area containing the barcode or QR code can be identified and the information contained therein (usually numbers) can be read.

[0019] Despite being processed by an artificial intelligence system, the following errors may still occur during analysis and processing: These errors are caused by the damage or contamination of the tags containing the transport information 4. Therefore, for example, it may not be possible to read individual regions of the barcode or individual letters of the text information. Then, such errors can be compensated by comparing with other information in the analysis and processing computer 3. On the one hand, the text information or the information encoded in the barcode contains sufficient redundancy to reconstruct the actually expected transport information. Alternatively, it can also be compared with other data (such as order information) in order to obtain the actually expected transport information again.

[0020] Both the first artificial intelligence system in the mobile device 2 and the second artificial intelligence system in the analysis and processing computer 3 must be trained with training images. Here, images of different transport carriers 1 must be provided as training data for training the first and second artificial intelligence systems, and these transport carriers have different packaging units as well as different tags, stickers or markings with transport information 4 arranged thereon. For this purpose, for example, different transport carriers 1 with different packaging units 5 and different transport information 4 can be photographed and provided to these two artificial intelligence systems together with the correct transport information 4. If subsequent changes occur (such as due to other packaging units 5 or other tags, stickers or markings with transport information 4), then images or photos must be re-taken to train the artificial intelligence system. This method is relatively complex because the transport carrier 1, the packaging unit 5 and the optical implementation of the transport information 4 must be constructed and photographed in all possible combinations separately. Alternatively, such images can be synthetically generated. For this purpose, the optical implementation of the transport information 4 (which usually exists in the computer anyway) is used to generate the tags, stickers or printed matter of the transport information 4. Then, the images so generated of the tags, stickers or printed matter of the transport information 4 are post-processed in such a way that these images visually correspond to the appearance images of such transport information 4 on the real transport carrier 1 or the packaging unit 5. For this purpose, these images are changed in terms of the viewing angle because a clearly vertical direction is not always given when taking a photo with the mobile device 2. The mobile device 2 can also be tilted or slightly tipped over, or rotated at an angle to align with the transport information 4. For example, a rectangular transport label will be distorted in a trapezoidal shape, whereby the size of the letters or barcode bars on the label changes accordingly. Correspondingly, curved surfaces, creases, contamination or damage can also be simulated, which may change the recognizability of the transport information 4 on the tag, sticker or marking. Then, the transport information 4 so changed is synthetically combined with the image of the packaging unit 5 and the image of the transport carrier 1 to form an image of the transport carrier 1 that has the packaging unit 5 and the transport information 4 arranged thereon. Then, these synthetic images are used to train the artificial intelligence system in the mobile device 2 or the analysis and processing computer 3.

[0021] It should also be noted here that an artificial intelligence system can generally be constructed significantly more simply in the mobile device 2, because the artificial intelligence system only has to recognize whether a photo is meaningful or meaningless. Here, if an unimportant relevant area of the transport carrier 1 is photographed by mistake, it is not a problem, because such a wrong photo is reliably recognized by the reprocessing of the analysis and processing computer 3.

[0022] In Figure 2 FIG. shows the respective method steps of the method according to the invention. In a first step 201, the method is started by an input on the mobile device 2. Such an input can, for example, be manually input by an operator of the mobile device 2. Alternatively, in the case of an automatic assembly of the transport carrier 1, the completion of the assembly of the transport carrier 1 can be determined and the mobile device 2 can be automatically started. In a subsequent step 202, the mobile device 2 continuously analyzes and processes camera information (i.e., a continuous image data stream), which is taken by the camera of the mobile device 2. Here, whenever the transport information 4 is recognized, a photo of the transport information 4 is generated. As already mentioned, the mobile unit 2 only has to use a low-performance artificial intelligence system for this purpose, because, for example, only significantly different labels or other markings on the packaging unit 5 have to be recognized. If all the transport information 4 of the transport carrier 1 has been detected, then in a subsequent step 203, an input is made by an input on the mobile device 2, which input indicates that the detection of the transport information has been completed for the processed transport carrier 1. This can again be achieved by an operator of the mobile device 2. The photos generated by the mobile device 2 in step 202 are either transmitted continuously or only when the detection is completed (i.e., in step 203) via the interface 7 to the analysis and processing computer 3. Then, in step 204, the photos are processed in the analysis and processing computer. Here, the photos of the transport information 4 are further checked by a second artificial intelligence system, and the type of transport information or the specific implementation of the transport information 4 on a label, sticker or printed matter is recognized here. Then, the transport information 4 is determined, where necessary in comparison with a database in order to assign other information to the transport information. Here, in particular, a comparison can be made between the order information and the transport information, from which it can be determined that the goods ordered with a certain order are now on the transport carrier and that the ordered goods are now being transported. Accordingly, then in a subsequent step 205, a shipping information is generated, from which it can be concluded which goods are now on the way to the addressee based on which orders. Then, the method ends with step 205.

Claims

1. A method for generating shipping information, in which at least one photograph of a transport carrier (1) is taken by a mobile device (2). Characterized in that in the mobile device (2), camera information is analyzed and processed by a first artificial intelligence system in terms of the recognition of transport information (4). In the case where transport information (4) is recognized, at least one photograph is taken by the mobile device (2) and the at least one photograph is transmitted to an analysis and processing computer (3). Among them, only the following photographs are transmitted from the mobile device (2) to the analysis and processing computer (3): these photographs contain the recognized transport information. In the analysis and processing computer (3), the at least one photograph is analyzed and processed by a second artificial intelligence system in terms of the recognition of optical transport information (4), the recognized transport information (4) is analyzed and processed in terms of content, and the content of the transport information (4) is used together with other inputs on the mobile device (2) to generate the shipping information. Among them, the first artificial intelligence system and the second artificial intelligence system are trained by training images of various optical transport information (4). Among them, the second artificial intelligence system has significantly stronger performance than the first artificial intelligence system. The content of the transport information (4) installed on the packaging unit (5) is not analyzed and processed in the mobile device (2). The mobile device (2) continuously analyzes and processes camera information in terms of whether the transport information (4) is recognized in the form of a label, sticker or printed matter. Among them, resources are optimally used through the optimized allocation between the mobile device (2) and the analysis and processing computer (3) in the method, and the amount of information transmitted from the mobile device (2) to the analysis and processing computer (3) is kept as low as possible.

2. The method according to claim 1, Characterized in that the training images have been synthetically generated in such a way that error-free transport information (4) is changed in terms of perspective and added to an image of the transport carrier (1).

3. The method according to claim 2, Characterized in that the training images have been synthetically generated in such a way that errors are added to the error-free transport information (4).

4. The method according to any one of claims 1 to 3, Characterized in that the optical transport information (4) is processed by optical character recognition.

5. The method according to any one of claims 1 to 3, Characterized in that the content of the transport information (4) is compared with a database containing order information.

6. The method according to claim 5, Characterized in that in the comparison, order information is assigned to the content of the transport information (4) taking into account typical errors.

7. A mobile device (2) for performing the method according to any one of claims 1 to 6, the mobile device having a first artificial intelligence system for identifying transport information (4), and the mobile device having means for detecting at least one photograph of a transport carrier (1), sending the at least one photograph to an analysis processing computer (3), and inputting other inputs.

8. An analysis processing computer for performing the method according to any one of claims 1 to 6, the analysis processing computer having a second artificial intelligence system for identifying optical transport information (4) and means for analyzing and processing the identified transport information in terms of content, generating shipping information based on the content of the transport information (4), and making other inputs on the mobile device (2).

Images