Information processing device, information processing method, and program

Abstract

To provide an information processing device that can improve the accuracy of identifying the destination of a package. [Solution] The information processing device includes an image acquisition unit that acquires image data obtained from an imaging device that images a pile of cargo including at least one pallet and at least one load loaded on the pallet, and an identification unit that detects the position of the pallet and the position of a label attached to the cargo based on the acquired image data, identifies a skid which is an area including the area where the cargo is located for each pallet based on the detected pallet position and the position of the label or the position of the cargo, reads label information which is information shown on at least one label in the image data, and identifies destination information regarding the destination of the cargo for each skid based on the label information.

Description

[[Technical Field]]

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program. [[Background Art]]

[0002] Conventionally, there has been known an information processing apparatus that identifies a destination of a luggage using an image captured by an imaging device. This information processing apparatus is an information processing apparatus communicable with an imaging device that images a luggage to be moved and an image display terminal, and includes: specifying means for specifying the destination of the luggage shown in image data using the image data acquired from the imaging device; generating means for generating a screen capable of identifying the destination of the luggage specified by the specifying means; and transmitting means for transmitting the generated screen to the image display terminal, wherein the specifying means specifies a platform shown in the image data from the image data acquired from the imaging device, specifies the destination of one of the luggages placed on the platform, and thereby specifies the destinations of other luggages placed on the platform as the same destination (see Patent Document 1). [[Prior Art Literature]] [[Patent Literature]]

[0003] [[Patent Document 1]] Japanese Patent No. 7323757 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0004] In the technique of Patent Document 1, the accuracy of specifying the destination of a luggage may not be sufficient, and there is room for improvement in specifying the destination of the luggage.

[0005] The present disclosure provides an information processing apparatus, an information processing method, and a program capable of improving the accuracy of specifying the destination of a luggage. [[Means for Solving the Problem]]

[0006] One aspect of the present disclosure is an information processing device comprising: an image acquisition unit that acquires image data obtained from an imaging device that images a pile of cargo including at least one pallet and at least one load mounted on the pallet; and an identification unit that detects the position of the pallet and the position of a label attached to the cargo based on the acquired image data, identifies a skid which is an area including the area where the cargo is located for each pallet based on the detected position of the pallet and the position of the label or the position of the cargo, reads label information which is information shown on at least one of the labels in the image data, and identifies destination information relating to the destination of the cargo for each skid based on the label information.

[0007] One aspect of the present disclosure is an information processing method performed collaboratively by a processor and a memory, comprising: acquiring image data obtained by imaging a load including at least one pallet and at least one load placed on the pallet; detecting the position of the pallet and the position of a label attached to the load based on the acquired image data; identifying a skid, which is an area including the area where the load is located, for each pallet based on the detected pallet position and the position of the label or the position of the load; reading label information, which is information shown on at least one of the labels in the image data, and identifying destination information relating to the destination of the load for each skid based on the label information.

[0008] One aspect of this disclosure is a program that causes a computer to execute the above-described information processing method. [Effects of the Invention]

[0009] According to this disclosure, the accuracy of identifying the destination of packages can be improved. [Brief explanation of the drawing]

[0010] [Figure 1]This figure shows an example of the configuration of an information processing system in the first embodiment of this disclosure. [Figure 2A] A diagram illustrating an example of the relationship between pallets, cargo, labels, skids, and cargo piles. [Figure 2B] A diagram showing an example of the relationship between pallets, cargo, labels, and cargo piles in a comparative example. [Figure 3] A diagram showing an example of the recognition range between the pallet, label, and skid. [Figure 4] This figure shows examples of label position recognition and label image correction. [Figure 5] An example of the operation of the information processing system in the first embodiment is shown in the sequence diagram. [Figure 6] A flowchart showing an example of skid label position detection processing. [Figure 7] A flowchart showing an example of label specification detection processing. [Figure 8] Block diagram showing an example configuration of the information processing system of this embodiment in the second embodiment. [Figure 9] There is a diagram illustrating an example of the characteristics of a pan / tilt camera and a fixed camera. [Figure 10] Flowchart showing an example of the operation of the information processing system in the second embodiment. [Figure 11] Block diagram showing an example configuration of the information processing system of this embodiment in the third embodiment. [Figure 12] A diagram showing the brightness of a predetermined placement location during imaging in the comparative example. [Figure 13] This figure shows an example of the brightness of a predetermined placement location during imaging in the third embodiment. [Figure 14] This figure shows an example of combining the detection results of label parameter detection based on images captured with different exposure patterns. [Figure 15] Sequence diagram showing an example of the operation of the information processing system in the third embodiment. [Figure 16] Image diagram showing an example configuration of the information processing system in the fourth embodiment. [Figure 17] Block diagram showing an example configuration of the information processing system in the fourth embodiment. [Figure 18] Sequence diagram showing an operation example of the information processing system in the fourth embodiment [Figure 19] Sequence diagram showing an operation example of the information processing system in the fifth embodiment Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments will be described in detail with appropriate reference to the drawings. However, overly detailed description may be omitted in some cases. For example, detailed description of already well-known matters and repeated description for substantially the same configuration may be omitted. This is to avoid unnecessary redundancy in the following description and facilitate understanding for those skilled in the art. The accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims thereby.

[0012] (Background Leading to Embodiments of the Present Disclosure) In the technology of Patent Document 1, the following cases can be considered as cases where the accuracy of specifying the destination of cargo may be insufficient.

[0013] For example, in the technology of Patent Document 1, no consideration is given to what range of cargo placed on one platform should be treated as having the same destination. For example, no consideration is given to whether all cargo above one platform should be treated as having the same destination, or only part of the cargo above one platform should be treated as having the same destination. Therefore, for example, when platforms are arranged side by side in the horizontal direction or stacked along the vertical direction, it is difficult to distinguish whether all cargo stacked above a platform have the same destination or include different destinations. In such a case, the accuracy of specifying the destination of the cargo may be insufficient.

[0014] Furthermore, for example, Patent Document 1 describes detecting the position of a label in the XY region on the platform where the luggage is placed after detection of the platform itself, and if the finder pattern of the QR code (registered trademark) containing the luggage's destination information on that label cannot be read, then reading the string is performed. However, the case where neither the finder pattern nor the string can be read is not considered. In such cases, it may not be possible to recognize the luggage's destination, and the accuracy of identifying the luggage's destination may be insufficient.

[0015] The following embodiments describe an information processing device, an information processing method, and a program that can improve the accuracy of identifying the destination of a package.

[0016] (First Embodiment) Figure 1 shows an example configuration of an information processing system 5 in a first embodiment of the present disclosure. The information processing system 5 includes an information processing device 100, a host system 200, and one or more imaging devices 300. The information processing device 100, the host system 200, and the one or more imaging devices 300 are communicated together, for example, via a network NT. The information processing system 5 is installed, for example, in a factory or warehouse and used for logistics.

[0017] The information processing device 100 and the higher-level device 200 are, for example, computer devices, and may be PCs (Personal Computers), terminals, smartphones, tablet devices, or server devices. If the information processing device 100 and the higher-level device 200 are server devices, they may be configured as on-premise server devices or as cloud-based servers on a network. The server device may consist of a single computer or be configured as a distributed system using multiple computers. In this embodiment, the information processing device 100 is exemplified as an image analysis server.

[0018] The information processing device 100 comprises a processor 110, a memory 120, and a communication device 130.

[0019] The processor 110 may be configured using, for example, a Central Processing Unit (CPU), a Digital Signal Processor (DSP), or a Graphical Processing Unit (GPU). The processor 110 may also be configured using various integrated circuits (for example, a Large Scale Integration (LSI) or a Field Programmable Gate Array (FPGA)). The processor 110 implements various functions by executing programs held in the memory 120. These various functions are, for example, the functions of each component shown in Figure 1. The processor 110 comprehensively controls each part of the information processing device 100 and performs various processes.

[0020] Memory 120 includes, for example, Random Access Memory (RAM) or Read Only Memory (ROM). Memory 120 may include volatile memory or non-volatile memory. Memory 120 may include, for example, a Hard Disk Drive (HDD), Solid State Drive (SSD), optical disc, SD card, etc. Memory 120 may also be an external storage medium and may be detachable from the information processing device 100. Memory 120 stores various data, information, or programs.

[0021] The communication device 130 communicates various data or information according to a wired or wireless communication method. The communication method used by the communication device 130 may include, for example, a Local Area Network (LAN), a Wide Area Network (WAN), a mobile phone network, or power line communication.

[0022] The communication device 130 is capable of communicating with external communication devices. The communication device 130 communicates with the host device 200, the imaging device 300, etc.

[0023] The higher-level device 200 controls, processes, analyzes, etc., each device, equipment, and item (e.g., luggage) provided in the information processing system 5. The higher-level device 200 also has a luggage information database 210 that holds information about luggage (also referred to as luggage information). Luggage information may include luggage identification information to identify the luggage, information on the number of luggage (i.e., the number of luggage items), information on the contents contained in the luggage, information on the destination (destination) of the luggage, etc. Luggage is, for example, a storage container for various items, such as a container, a bottle, or a cardboard box. Also, one or more labels (slips) are attached to each luggage, for example, by being affixed. If luggage has multiple sides, labels are affixed to at least one of the multiple sides of the luggage. The luggage information database 210 may also be composed of a device different from the higher-level device 200 (e.g., a database server).

[0024] The imaging device 300 captures an image of a subject to obtain an image. The imaging device 300 includes a swivel camera 300A. The swivel camera 300A is capable of panning, tilting, and zooming, and its imaging range is changeable, and the imaging range can be moved and resized. The imaging device 300 may also include cameras other than the swivel camera 300A.

[0025] The subject imaged by the imaging device 300 includes, for example, a pile of goods. The pile of goods includes at least one pallet and at least one load loaded on at least one pallet. The at least one pallet may include multiple pallets stacked vertically, multiple pallets arranged horizontally, or both.

[0026] Furthermore, the pan-tilt camera 300A may have a controllable zoom magnification, and may enlarge or reduce the subject to capture an image of the subject. For example, the pan-tilt camera 300A may capture an image of all or part of the cargo pile, or capture an image of a label as part of the cargo pile.

[0027] The processor 110 of the information processing device has an image acquisition unit 111 and a specific unit 112 as components having various functions.

[0028] The image acquisition unit 111 acquires image data (also simply referred to as images) obtained by the imaging device 300. The image data may include, for example, piles of cargo, packages, labels, etc. However, it is possible that the piles of cargo may not be in the predetermined position and therefore may not be visible in the image data.

[0029] The identification unit 112 identifies various pieces of information. For example, the identification unit 112 may detect the position of the pallet, the position of the label attached to the cargo, and the position of the cargo based on the acquired image data. In this case, the identification unit 112 may identify the label position, pallet position, and cargo position based on an object detection model for detecting various objects as AI (Artificial Intelligence). The acquired image data may be an image (also referred to as a whole image) that captures the entire predetermined location where the pallet and the piles of cargo stacked on the pallet may be placed. The whole image is an image in which a subject including one or more pallets and each pile of cargo stacked on each pallet is captured.

[0030] For example, the identification unit 112 may use a large number of overall images and the label positions within the overall images as training data, and train an object detection model to derive the label positions within the overall images from the overall images. For example, the identification unit 112 may use a large number of overall images and the pallet positions within the overall images as training data, and train an object detection model to derive the pallet positions within the overall images from the overall images. For example, the identification unit 112 may use a large number of overall images and the luggage positions within the overall images as training data, and train an object detection model to derive the luggage positions within the overall images from the overall images. In this way, the identification unit 112 can detect label positions, pallet positions, and luggage positions even when there are multiple types of pallets, labels, and luggage.

[0031] The identification unit 112 may identify a skid for each pallet, which is an area containing the pallet and its contents, based on the detected pallet position and the label position or the contents position. A skid is a single unit that is transported by a forklift or the like. For example, one skid contains one pallet and one or more pieces of cargo. Since skids are transported as a single unit, the destination of the one or more pieces of cargo included in a skid is basically the same destination.

[0032] As a specific example of how to identify skids, the identification unit 112 may identify the space between a predetermined pallet stacked vertically (vertically stacked) and another pallet stacked above it as a skid. Alternatively, if there are no other pallets stacked above the predetermined pallet, the space between the predetermined pallet and the top of the pile of goods may be identified as a skid. In other words, the identification unit 112 can define skids even when the pallets are not stacked vertically.

[0033] The identification unit 112 reads label information (for example, label specification information described later), which is information indicated on at least one label in the overall image. The label information includes, for example, information indicated by the characters written on each item on the label (text information). The label information also includes information indicated by the code written on the label (identification code). The content written on each item on the label and the content corresponding to the code may be the same or different. If they are the same, multiple identical pieces of content are written on the label, and the identification unit 112 can understand the content by reading any of them, thereby improving the reliability of the read content.

[0034] The identification unit 112 reads the labels of one or more packages loaded on each skid. In this case, the labels of the packages may be read one by one. That is, it reads the label of a predetermined package, and if the reading is successful, the reading is complete. This is because it is sufficient to read at least one label. On the other hand, if the reading fails, the unit identifies the location of the labels of other packages (other labels on the skid) based on the label locations within the same skid, and reads these labels. It is also possible to read at least one label of at least one package on the same pallet simultaneously.

[0035] The identification unit 112 may identify destination information for each skid based on the label information. If a single skid contains multiple packages, the identification unit 112 may identify the destination information of the packages based on the labels and then identify that other packages in the same skid have the same destination. In other words, if the identification unit identifies the first destination information of a first package contained in a predetermined skid among multiple packages, it may then identify that the second destination information of a second package contained in the same skid is the same as the first destination information.

[0036] The communication device 130 transmits the destination information identified by the identification unit 112 to the host device 200.

[0037] Next, we will explain the relationship between pallets, cargo, labels, skids, and cargo piles.

[0038] Figure 2A shows an example of the relationship between a pallet, cargo, label, skid, and pile. Figure 2A shows a pallet 11, cargo 12, label 13, skid 14, and pile 15.

[0039] One or more pallets 11 may be placed in a predetermined location. One or more items 12 may be stacked on each pallet 11. The items 12 may be arranged horizontally or stacked vertically. One or more labels 13 may be attached to each item 12, for example, by being affixed or displayed by a simple display for each item 12. Each pile 15 is formed by the one or more items 12 stacked on each pallet 11. Each skid 14 is formed by each pallet 11 and the one or more items 12 stacked on that pallet 11. In other words, one skid 14 is formed by one pallet 11 and one pile 15. Note that if one pallet 11 has no items 12 stacked on it, and another pallet 11 is placed directly on top of that pallet 11, then the skid 14 is formed by only that one pallet 11. The skids 14 make it possible to identify the space (skid space) where items 12 are stacked on each pallet 11, even if multiple pallets 11 are stacked vertically.

[0040] In Figure 2A, two pallets, pallet 11A and pallet 11B, are shown as pallet 11. On pallet 11A, a load 15A is formed by two loads 12. On pallet 11B, a load 15B is formed by four loads 12.

[0041] Furthermore, the predetermined placement location mentioned above is, for example, the position where the cargo 12 is unloaded from the truck, the position where the cargo 12 is loaded onto the truck, or any other location. The predetermined placement location can be included in the imaging range CR of the imaging device 300 and can be imaged by the imaging device 300.

[0042] Figure 2B shows another example of the relationship between pallet 11, cargo 12, label 13, skid 14, and cargo pile 15.

[0043] In Figure 2B, the pallets 11 can be placed in predetermined locations. These predetermined locations can be included in the imaging range CR of the imaging device 300 and can be imaged by the imaging device 300. In Figure 2B, the pallets 11 cannot be stacked vertically, so a wide horizontal space is required. Although Figure 2B illustrates the arrangement of pallets 11 horizontally, skids 14 can be formed based on each pallet 11, and the range of the skids 14 can be defined.

[0044] Figure 3 shows an example of the recognition range between the pallet 11, label 13, and skid 14.

[0045] The imaging device 300 captures an image including one or more pallets 11 and one or more loads 12 (i.e., a pile of loads 15) stacked on each pallet 11, and obtains an overall image GN as the captured image. The identification unit 112 of the processor 110 of the information processing device 100 acquires the overall image GN and performs image recognition on the overall image GN to detect (recognize) the position of the pallets 11 (also referred to as the pallet position PA), the position of the labels 13 (also referred to as the label position LA), the position of the loads 12 (also referred to as the load position CA), and the range of the skids 14 (skid range SA) in the overall image GN. The label position LA indicates the location of the area where the labels 13 are located in the overall image GN. The pallet position PA indicates the location of the area where the pallets 11 are located in the overall image GN. The load position CA indicates the location of the area where the loads 12 are located in the overall image GN. The skid range SA indicates the location of the area where the skids 14 are located in the overall image GN. These detection results may be stored in the memory 120.

[0046] The pallet position PA, label position LA, luggage position CA, and skid range SA are shown as rectangular frames, but are not limited to this. Depending on the positional relationship between the pallet 11, label 13, skid 14, and imaging device 300, the positional relationship of each range in the image changes, and may become a quadrilateral shape other than a rectangle (for example, a parallelogram or trapezoid), or a shape other than a quadrilateral.

[0047] Figure 4 shows an example of label position LA recognition and an example of label image correction. Here, we assume that the image of the label position LA (also referred to as label image GL) is labeled image GL1, which is the recognized label position before correction, and labeled image GL2, which is the corrected label position LA. Label image GL is an image extracted from the overall image GN where the label position LA is located, or a zoomed image captured by zooming in on the label position LA.

[0048] The identification unit 112 first recognizes the label position LA1 based on the overall image GN. The identification unit 112 corrects the label image GL1 of the label position LA in the overall image GN to obtain the label image GL2. In this case, the identification unit 112 estimates the positional relationship between the label 13 and the imaging device 300 based on the label image GL1, generates an image where the label 13 is in front of the imaging device 300 (the surface directly facing the imaging direction by the imaging device 300), and sets this as the label image GL2.

[0049] In Figure 4, before correction, the label image GL1 is captured from a position not directly in front of the imaging device 300, and has a slightly deformed rectangular shape. In contrast, the specific unit 112 corrects the label image GL1 to obtain the label image GL2, which has a rectangular shape.

[0050] The identification unit 112 recognizes the information of each item 13a described on the label 13 based on the corrected label image GL2 and reads its contents. In this case, as shown in Figure 4, the identification unit 112 may recognize the range of each item 13a in the image at label position LA2 and separate the read information for each item 13a, or it may read the information for each item 13a. In Figure 4, as an example, the range of each item 13a is shown by a rectangular frame.

[0051] In this way, the information processing device 100 extracts the label position LA from the acquired image and converts the label image GL at label position LA into a rectangle, thereby enabling matching with the label format. This allows the recognition information to be classified into each item 13a according to its position within the label 13. Furthermore, the information processing device 100 can improve the accuracy and reliability of the reading results by using in combination the reading information of the code 13b within the label 13 and the reading information of the characters of each item 13a within the label.

[0052] Next, we will explain the operation of the information processing system 5. Figure 5 is a sequence diagram illustrating an example of the operation of the information processing system 5.

[0053] First, in the information processing device 100, the processor 110 requests the imaging device 300 to capture an overall image GN that provides an overview of a predetermined location (step S101). In this case, the processor 110 may transmit an imaging request signal to the imaging device 300 via the communication device 130.

[0054] The imaging device 300 receives an imaging request from the information processing device 100, for example, by receiving an imaging request signal (step S201). The imaging device 300 captures an overall image GN in response to the imaging request and transmits the captured overall image GN to the information processing device 100 (step S201). The overall image GN may be a still image, a video, or a series of still images.

[0055] In the information processing device 100, the image acquisition unit 111 of the processor 110 acquires the overall image GN from the imaging device 300 via the communication device 130. Based on the overall image GN, the identification unit 112 detects the pallet position PA, the label position LA, and the luggage position according to the object detection model (step S102).

[0056] The identification unit 112 detects the skid range SA based on the detected pallet position PA and label position LA or cargo position, and detects the label position LA for the detected skid range SA (step S103). The information of the detected label position LA is stored in the memory 120.

[0057] The identification unit 112 reads information about one of the one or more detected label locations LA from the memory 120 (step S104).

[0058] The identification unit 112 requests the imaging device 300 to zoom to the read label position LA (step S105). In this case, the identification unit 112 may transmit a zoom request signal to the imaging device 300 via the communication device 130.

[0059] The imaging device 300 receives a zoom request from the information processing device 100, for example, by receiving a zoom request signal (step S202). In response to the zoom request, the imaging device 300 zooms in on the label position LA in the overall image GN (i.e., increases the zoom magnification) and captures a zoomed image of the label position LA. The imaging device 300 transmits the captured zoomed image to the information processing device 100 (step S202).

[0060] In the information processing device 100, the image acquisition unit 111 acquires a zoomed image from the imaging device 300 via the communication device 130 (step S106). The identification unit 112 performs image recognition of the zoomed image and performs label specification detection processing to detect (read) the label specifications (step S107). The label specifications are the contents indicated by each item 13a and the contents indicated by the code 13b written on the label 13.

[0061] The identification unit 112 determines whether or not the label specifications were successfully detected as a result of the label specifications detection process (step S108).

[0062] If the detection of label specifications is unsuccessful (No. in step S108), the identification unit 112 proceeds to step S104. In other words, if the detection of label specifications is unsuccessful, the identification unit 112 attempts to detect label specifications corresponding to other label positions LA within the same skid 14.

[0063] If the detection of the label specifications is successful (Yes in step S108), the identification unit 112 transmits the label specification information, including the detection result of the label specifications, to the host device 200 via the communication device 130 (step S109).

[0064] The identification unit 112 determines whether there are other skids 14 (also referred to as target skids) for which label specifications are to be detected (step S110). In other words, since the detection of the label specifications of the current target skid is complete, the identification unit 112 determines whether there are any other target skids.

[0065] If there is no next target skid (No. in step S110), the identification unit 112 terminates the process shown in Figure 5, as it has completed the detection of the label specifications for all labels 13.

[0066] If there is a next target skid (Yes in step S110), the identification unit 112 updates the target skid to skid 14 as the next target skid (step S111). Then, the identification unit 112 proceeds to step S105. That is, the identification unit 112 attempts to detect the label specifications corresponding to the label position LA in the next target skid.

[0067] According to the example of operation shown in Figure 5, the information processing system 5 detects the skid range SA and label position LA from the overall image GN, and for each skid 14, it detects the label specifications of the labels 13 present within the skid 14. In this process, for each skid 14, the information processing system 5 acquires a zoomed image taken by zooming in on at least one label position LA within the skid range SA, and detects the label specifications based on the zoomed image. If at least one label specification is detected within the skid 14, the destination of the cargo 12 loaded on the skid 14 can be identified. The destination information of the cargo 12 is included in the label specification information.

[0068] Figure 6 is a flowchart showing an example of the skid label position detection process.

[0069] The identification unit 112 of the processor 110 of the information processing device 100 updates the pallet 11 (also referred to as the target pallet) for which the skid range SA is to be detected (step S121). In the initial state, the identification unit 112 determines a predetermined pallet 11 from among one or more pallets 11 that are reflected in the overall image GN to be the target pallet.

[0070] The identification unit 112 determines whether or not there is a pallet adjacent to the target pallet in the vertical direction (also referred to as the next pallet) (step S122).

[0071] If a next pallet exists (Yes in step S122), the identification unit 112 determines the space between the target pallet and the next pallet in three-dimensional space as the skid range SA (step S123).

[0072] When the processing in step S123 is completed, the identification unit 112 proceeds to step S121 and updates the target pallet. In this case, the identification unit 112 may determine one of the pallets 11 for which the skid range SA has not yet been set as the target pallet, for example, the pallet 11 that was designated as the next pallet in steps S122 and S123 may be determined as the target pallet.

[0073] If there is no next pallet in step S122 (No. of step S122), the identification unit 112 determines the space between the target pallet and the top of the cargo 12 stacked on the target pallet as the skid range SA (step S124).

[0074] The identification unit 112 stores information of the determined skid range SA and the label positions LA included in that skid range SA in the memory 120 (step S125). In other words, information of the label positions LA is stored for each skid.

[0075] According to this skid label detection process, the information processing system 5 can easily determine the skid 14 and the skid range SA based on the pallet 11.

[0076] Figure 7 is a flowchart showing an example of the label specification detection process.

[0077] The identification unit 112 of the processor 110 of the information processing device 100 performs image recognition on the acquired zoom image, extracts the contour of the label 13 in the zoom image to detect the label position LA, and extracts the image corresponding to the label position LA (label image GL1) (step S131).

[0078] The identification unit 112 corrects the acquired label image GL1 to label image GL2 by converting it into a rectangle using projection transformation (step S132). This correction is the one illustrated in Figure 4.

[0079] The identification unit 112 compares the label image GL (label image GL2) with the label format (step S133). The label format information is stored in the memory 120. The label format includes information indicating the position of each item 13a on the label 13. The label format also includes information indicating the position of the code 13b on the label 13 (e.g., a one-dimensional code (barcode) or a two-dimensional code (e.g., QR code®, Data Matrix®)).

[0080] The identification unit 112 performs OCR (optical character recognition) on the range of each item in the label image GL and reads the content of each item 13a in the label image GL (step S134).

[0081] The identification unit 112 performs image recognition on the range of code 13b in the label image GL and reads the content of code 13b (step S135).

[0082] According to this label specification detection process, the information processing system 5 can suitably read the label specifications of the label 13 even if the package 12 is stacked on the pallet 11 or other packages 12 at various angles during the movement of the package 12.

[0083] As described above, the information processing system 5 of this embodiment detects the label specifications of the label 13 based on the image captured by the imaging device 300, making it easier and faster to read the label specifications for each item 13a of the label 13 than if a forklift operator or the like were to manually read them using a handheld scanner. Manual reading may be difficult when the pile of goods 15 is high, but the information processing system 5 improves safety and work efficiency.

[0084] Furthermore, the information processing device 100 can track the packages 12 on a skid-by-skid basis, enabling efficient logistics. For example, since the information processing device 100 often moves each package 12 to the same destination on each skid, it is sufficient to read at least one label 13, and it is not necessary to correctly read all of the labels 13. In this case, the information processing device 100 can reduce the reading time and perform efficient reading.

[0085] Furthermore, even if the surface of the package 12 is not directly facing the imaging device 300 but is tilted, the image captured by the imaging device 300 can be shaped into, for example, a rectangle, to recognize the label content. Therefore, the information processing device 100 can easily read labels even without precisely aligning the orientation of the package 12, and the arrangement of the package 12 can be sped up. Also, for example, if the information processing system 5 is used in a factory or warehouse, the information of the package 12 can be quickly and automatically read using fixed equipment, improving safety and work efficiency.

[0086] (Second embodiment) In the second embodiment, the imaging device 300 includes both a pan-tilt camera 300A and a fixed camera 300B, and the label specification information is derived based on the image captured by the fixed camera 300B and the image captured by the pan-tilt camera 300A.

[0087] Figure 8 is a block diagram showing an example configuration of the information processing system 5 of this embodiment. In Figure 8, components similar to those in the information processing system 5 of Figure 1 are denoted by the same reference numerals, and their descriptions are omitted or simplified.

[0088] In this embodiment, the imaging device 300 includes a swivel camera 300A and a non-swivel fixed camera 300B. The fixed camera 300B has an immovable imaging range, and its imaging range cannot be moved or resized. The number of swivel cameras 300A and the number of fixed cameras 300B can be arbitrary.

[0089] The identification unit 112 of this embodiment has the following functions in addition to the functions of the identification unit 112 in the previous embodiment. Specifically, the identification unit 112 detects label specifications based on a zoom image acquired from the pan-type camera 300A and obtains first label specification information. The first label specification information includes first destination information for the luggage 12. The identification unit 112 detects label specifications based on an overall image GN acquired from the fixed camera 300B and obtains second label specification information. The second label specification information includes second destination information I2 for the luggage 12.

[0090] The identification unit 112 may ultimately determine the label specifications for each skid 14 based on the first label specifications information and the second label specifications information. In this case, the identification unit 112 may ultimately determine the destination of each skid 14 based on the first destination information and the second destination information.

[0091] Figure 9 shows an example of the features of the swivel camera 300A and the fixed camera 300B.

[0092] Figure 9 shows the image quality, label imaging method, label quality, and overall evaluation for the swivel camera 300A and the fixed camera 300B. The image quality of the swivel camera 300A is moderate, while the image quality of the fixed camera 300B is high. The label imaging method for the swivel camera 300A involves zooming in on the label 13 by rotating (panning, tilting) and zooming the swivel camera 300A, resulting in a long cycle time. This is because image processing takes time. The label imaging method for the fixed camera 300B involves extracting the label position LA from a high-resolution image through image processing to derive the label image GL, resulting in a short cycle time. The label quality of the swivel camera 300A is high because it is possible to zoom in on the label position LA. On the other hand, the label quality of the fixed camera 300B is moderate, and under certain conditions, it is at a level between the lowest and low levels.

[0093] In short, the swivel camera 300A has a longer cycle time because it performs pan, tilt, and zoom operations, but it has superior label image quality and makes it easy to read label specifications. On the other hand, the fixed camera 300B has slightly inferior image quality compared to the swivel camera 300A, but it has a shorter cycle time because it does not perform pan, tilt, or zoom operations. Furthermore, the fixed camera 300B can detect the label specifications of the target label 13 at once, making it easier to supplement the information.

[0094] Figure 10 is a flowchart illustrating an example of the operation of the information processing system 5 in this embodiment. Figure 10 shows an example of the operation of the information processing device 100. In Figure 10, the processing by the pan-type camera 300A and the processing by the fixed camera 300B are shown in parallel, but both are processed by the information processing device 100. Note that in Figure 10, the explanations of the processing shown in Figures 6 to 8 may be omitted or simplified.

[0095] First, the image acquisition unit 111 and the identification unit 112 of the information processing device 100 perform the same processing as steps S101 to S103 shown in Figure 5 regarding the images captured by the swivel camera 300A and the captured images. Specifically, the image acquisition unit 111 receives the overall image GN1 captured by the swivel camera 300A (step S141A). Based on the overall image GN1, the identification unit 112 detects the pallet position PA1, the label position LA1, and the cargo position CA1 (step S142A). The identification unit 112 performs skid label position detection processing and detects the skid range SA1 and the label position LA1 for each skid 14 (step S143A).

[0096] Similarly, the image acquisition unit 111 and the identification unit 112 perform the same processing as steps S101 to S103 shown in Figure 5 with respect to the images captured by the fixed camera 300B and the captured images. Specifically, the image acquisition unit 111 receives the overall image GN2 captured by the fixed camera 300B (step S141B). The identification unit 112 detects the pallet position PA2, the label position LA2, and the cargo position CA2 based on the overall image GN2 (step S142B). The identification unit 112 performs skid label position detection processing and detects the skid range SA2 and the label position LA2 for each skid 14 (step S143B).

[0097] The identification unit 112 determines the skid range SA in the overall image GN based on the detected skid range SA1 and skid range SA2 (step S144). In this case, the identification unit 112 may calculate the skid range SA as the average position of skid range SA1 and skid range SA2, or it may adopt either skid range SA1 or skid range SA2 as the skid range SA, or it may derive the skid range SA by other methods.

[0098] The identification unit 112 determines the label position LA in the overall image GN based on the derived label positions LA1 and LA2 (step S144). In this case, the identification unit 112 may calculate the average position of label positions LA1 and LA2 as the label position LA, or it may adopt either label position LA1 or label position LA2 as the label position LA, or it may derive the label position LA by other methods.

[0099] Next, the identification unit 112 performs label specification detection based on the overall image GN1 captured by the pan-type camera 300A (also referred to as the first label specification detection) and label specification detection based on the overall image GN1 captured by the fixed camera 300B (also referred to as the second label specification detection). The first label specification detection and the second label specification detection may be performed simultaneously and in parallel (parallel processing), or they may be performed with a time difference (serial processing).

[0100] In the first label specification detection, the same processing as a part of the skid-specific label specification detection in the first embodiment (steps S104 to S108) is performed.

[0101] Specifically, the identification unit 112 reads information about the label position LA from the memory 120 (step S145A). The identification unit 112 requests a zoomed image of each label position LA from the pan-tilt camera 300A (step S146A) and obtains the zoomed image from the pan-tilt camera 300A (step S147A). The identification unit 112 performs label specification detection processing (step S148A). It determines whether the detection of the label specifications was successful or not (step S149). If the detection of the label specifications fails (No in step S148A), the process proceeds to step S145A; if the detection of the label specifications is successful (Yes in step S148A), the process proceeds to step S150. The label specification information obtained through this label specification detection is also referred to as the first label specification information.

[0102] On the other hand, the second label specification detection process performs the following steps.

[0103] Specifically, the identification unit 112 reads the label position LA information from the memory 120 (step S145B). Based on the overall image GN2 and the label position LA, the identification unit 112 electronically extracts an image of each label position LA (label image GL) from the overall image GN2 and generates an extracted image. Based on each extracted image, the identification unit 112 performs label specification detection processing (step S146B). Here, in step S131 of the label specification detection processing, the label image is extracted from the extracted image rather than the zoomed image. Then, the identification unit 112 matches the label specification information obtained by the label specification detection processing for each label 13 and detects the label specification information (also referred to as the second label specification information) for all labels 13 (step S146B). Then, the identification unit 112 proceeds to step S150. Note that for some labels 13, the detection of label specifications may fail, and the label specification information for at least one label 13 may not be derived.

[0104] The identification unit 112 combines the first label specification information obtained based on the rotating camera 300A and the second label specification information obtained based on the fixed camera 300B (step S150). The information processing device 100 attempts to acquire label specification information for all labels 13 that are subject to label specification detection, but information included in at least one of the first label specification information and the second label specification information has already been acquired for all labels 13.

[0105] For example, the label specifications of label 13 that were successfully read as the second label specifications are considered acquired. If the first label specifications have been obtained for label 13 that failed to be read as the second label specifications, the label specifications for this label 13 are considered acquired. On the other hand, if the first label specifications have not yet been obtained for label 13 that failed to be read as the second label specifications, the label specifications for this label 13 are considered unacquired.

[0106] The identification unit 112 determines whether the detection of label specifications for all labels 13 has been completed (step S151). In this case, the identification unit 112 determines for each skid 14 whether the detection of label specifications for at least one label 13 has been successful, and whether the label specification information for at least one label 13 has been acquired.

[0107] If the detection of label specifications for at least one label 13 has not been completed for each skid 14 (No. in step S151), the identification unit 112 updates the target skid to the next skid 14 for which label specifications have not yet been obtained (step S152). Then, the identification unit 112 proceeds to step S145A. This is because there are still label specifications that have not yet been obtained, and the unit needs to acquire information on these label specifications.

[0108] For example, in the detection of label specifications based on the fixed camera 300B, the overall image GN including the label position LA of each label 13 can be obtained quickly, but the image quality of the cropped image of the label position LA is low, resulting in low accuracy in reading the label 13 and a high likelihood of failing to acquire label specification information. In contrast, in the detection of label specifications based on the pan-tilt camera 300A, it takes time to acquire each zoom image including the label position LA of each label 13, but the label quality is high, resulting in high reading accuracy and a high likelihood of successfully acquiring label specification information. The identification unit 112 can acquire the label specification information of each label 13 quickly by using label specification detection based on the fixed camera 300B, and can also acquire the label specification information of labels 13 that failed to be detected by using label specification detection based on the pan-tilt camera 300A with high accuracy.

[0109] On the other hand, if the detection of label specifications for at least one label 13 for each skid 14 is completed (Yes in step S151), the identification unit 112 transmits the label specifications information, including the detected label specifications, to the host device 200 via the communication device 130 (step S153).

[0110] Note that the timing for combining the label specification detection information based on the swivel camera 300A and the label specification detection information based on the fixed camera 300B, as shown in step S150, is just one example and is not limited to the example in Figure 10. Also, the timing for transmitting the combined label specification information of each label 13 to the host device 200, as shown in step S153, is just one example and is not limited to the example in Figure 9.

[0111] Thus, the information processing system 5 of this embodiment can utilize the differences in characteristics between the pan-tilt camera 300A and the fixed camera 300B to complement (integrate) the label specification information read from the images captured by each camera. Therefore, the information processing system 5 can obtain label specification information for at least one label 13 for each skid 14, and can quickly read accurate information.

[0112] (Third embodiment) In the third embodiment, label specification detection is described in which the brightness of a predetermined placement location where the label specifications are detected is taken into consideration.

[0113] Figure 11 is a block diagram showing an example configuration of the information processing system 5 of this embodiment. In Figure 11, components similar to those in the information processing system 5 of Figure 1 or Figure 8 are denoted by the same reference numerals, and their descriptions are omitted or simplified.

[0114] The processor 110 of the information processing device 100 has, in addition to at least some of the functions of each component shown in the first or second embodiment, the function of an imaging condition control unit 113. The imaging condition control unit 113 controls the imaging conditions of the imaging device 300. The imaging conditions include, for example, the imaging range, the imaging timing, the exposure pattern when imaging by the imaging device 300, etc. The exposure pattern includes, for example, the amount of exposure and other exposure-related information. The imaging range is determined based on the imaging direction, angle of view, zoom magnification, etc.

[0115] The imaging condition control unit 113 may instruct the imaging device 300 to image a predetermined location using at least two different exposure patterns. In this case, the imaging condition control unit 113 may perform image recognition on the overall image GN of the image taken at the predetermined location and detect the illuminance difference corresponding to the difference in pixel values ​​at each pixel in the overall image. If the illuminance difference is greater than or equal to a threshold th, the imaging condition control unit 113 may instruct the imaging device 300 to image the pile 15 using at least two different exposure patterns. In this case, the imaging condition control unit 113 may transmit an exposure control signal to the imaging device 300 via the communication device 130 to control the exposure pattern of the imaging device 300.

[0116] Furthermore, the difference in illuminance at a predetermined location may be detected by methods other than image recognition of the image. For example, an illuminance sensor may be placed at a location where information about the location can be detected, and the difference in illuminance may be detected by the difference in illuminance detected at each location of the location by the illuminance sensor.

[0117] The imaging device 300 receives an exposure control signal from the information processing device 100 and controls the exposure pattern according to the exposure control signal. As a result, the brightness of the image captured by the imaging device 300 is adjusted. The imaging device 300 captures the subject while the exposure pattern is controlled.

[0118] Figure 12 shows the brightness at a predetermined placement location during imaging in the comparative example.

[0119] In environments with extreme differences in brightness, such as when the designated placement location is semi-outdoors, the dynamic range of the imaging device 300 may be exceeded during imaging. In Figure 12, the overall image guide (GN) of the designated placement location includes areas that are too bright (overexposed), areas with appropriate brightness (correct brightness, properly exposed), and areas that are too dark (underexposed). In such conditions, it is difficult for the imaging device 300 to image the entire package 12 or pile 15 targeted for label specification detection with proper exposure, making it difficult to obtain an image with appropriate brightness. Furthermore, the bracketing function and HDR (High Dynamic Range) of the imaging device 300 are generally limited to image quality and exposure adjustments within the range that constitutes a single image, making it difficult to obtain an image with appropriate brightness.

[0120] Figure 13 shows an example of the brightness at a predetermined placement location during imaging in this embodiment.

[0121] In this embodiment, when imaging is performed by the imaging device 300, the imaging condition control unit 113 instructs the imaging device 300 to control the exposure pattern. Figure 13 illustrates three exposure patterns: exposure pattern 1, exposure pattern 2, and exposure pattern 3. Exposure patterns 1, 2, and 3 are predetermined, for example, and information for each exposure pattern is stored in the memory 120.

[0122] The imaging device 300 obtains an overall image GN11 by capturing an image with exposure pattern 1. The overall image GN11 has a region L0 with appropriate brightness, a region L2 with excessively dark brightness, and a region L3 with even more excessively dark brightness (darker than region L2). The imaging device 300 detects the label specifications of label 13 in the region L0 with appropriate brightness by performing label specification detection processing based on a zoomed image or cropped image including the label position LA.

[0123] The imaging device 300 obtains an overall image GN12 by capturing an image with exposure pattern 2. The overall image GN12 has a region L1 that is too bright, a region L0 that is at the appropriate brightness, and a region L2 that is too dark. The imaging device 300 detects the label specifications of label 13 in the region L0 that is at the appropriate brightness by performing label specification detection processing based on a zoomed image or cropped image that includes the label position LA.

[0124] The imaging device 300 obtains an overall image GN13 by capturing an image with exposure pattern 3. The overall image GN13 has a region L1 with excessively bright brightness and a region L0 with appropriate brightness. The imaging device 300 detects the label specifications of label 13 in the region L0 with appropriate brightness by performing label specification detection processing based on a zoomed image or cropped image including the label position LA.

[0125] Figure 14 shows an example of a composite of detection results for label parameter detection based on images captured with different exposure patterns.

[0126] By imaging the cargo pile 15 with each of the exposure patterns 1 to 3, the position of the region L0 with appropriate brightness differs in the overall image GN11 to GN13. By combining the detection results (reading results, label specification information) of each label 13 obtained in each exposure pattern 1 to 3, label specification information for all labels 13 captured in the overall image GN (GN11 to GN13) can be obtained. Here, by combining the label specification information obtained in each exposure pattern 1 to 3, the processing load can be reduced compared to detecting the label specifications from the combined image after combining the label images obtained in each exposure pattern 1 to 3.

[0127] Next, the operation of the information processing system 5 of this embodiment will be described.

[0128] Figure 15 is a sequence diagram showing an example of the operation of the information processing system 5. Note that in Figure 15, explanations of the processes shown in Figures 6-8 or Figure 10 may be omitted or simplified.

[0129] The imaging condition control unit 113 of the information processing device 100 initializes the exposure pattern when imaging is performed by the imaging device 300 (step S161). Here, the imaging condition control unit 113 requests that the imaging device 300 be initialized to a predetermined exposure pattern (exposure control request) and transmits, for example, an exposure control signal to the imaging device 300.

[0130] The imaging device 300 receives a request from the information processing device 100 for initial exposure pattern settings (exposure control request) and, for example, receives an exposure control signal (step S211). The imaging device 300 sets the exposure pattern to initial settings in response to the exposure control request (step S211).

[0131] The identification unit 112 transmits an imaging request to the imaging device 300 (step S162).

[0132] The imaging device 300 receives an imaging request from the information processing device 100, for example, by receiving an imaging request signal (step S212). The imaging device 300 captures the overall image GN in response to the imaging request and transmits the captured overall image GN to the information processing device 100 (step S212).

[0133] In the information processing device 100, the image acquisition unit 111 of the processor 110 acquires the overall image GN from the imaging device 300 via the communication device 130 (step S163). Based on the overall image GN, the identification unit 112 detects the pallet position PA, the label position LA, and the luggage position CA according to the object detection model (step S164).

[0134] The identification unit 112 performs skid label position detection processing (step S165). The identification unit 112 then derives label specification information for at least one label 13 for each skid 14 by label specification detection based on the swivel camera 300A or label specification detection based on the fixed camera 300B. The processor 110 stores the derived at least one label specification information for each skid 14 in the memory 120 (step S166).

[0135] The identification unit 112 determines whether all exposure patterns have been set among the settable exposure patterns (for example, exposure patterns held in memory 120), that is, whether there are any unset exposure patterns (step S167).

[0136] If not all exposure patterns have been set, that is, if there are unset exposure patterns (No. in step S167), the identification unit 112 requests that the exposure pattern be updated (changed) to one of the unset exposure patterns (step S168). In this case, the identification unit 112 may send an exposure update request signal to the imaging device 300 via the communication device 130.

[0137] The imaging device 300 receives a request from the information processing device 100 to update the exposure pattern, for example, by receiving an exposure update request signal (step S213). The imaging device 300 updates (changes) the exposure pattern in response to the exposure update request (step S213).

[0138] In other words, the information processing device 100 and the imaging device 300 repeatedly acquire label specification information for each detectable label based on an image captured with a predetermined exposure pattern, while changing the exposure pattern.

[0139] If all exposure patterns have already been set, that is, if there are no unset exposure patterns (Yes in step S167), the identification unit 112 synthesizes the label specification information obtained for each exposure pattern (step S169). As a result, the identification unit 112 obtains the label specification information for all the labels 13 of the skids 14 placed in predetermined locations.

[0140] Furthermore, the higher-level device 200 transmits the package information stored in the package information database 210 to the information processing device 100 (step S301).

[0141] In the information processing device 100, the identification unit 112 acquires package information from the host device 200 via the communication device 130 (step S170).

[0142] The identification unit 112 compares the package information with the label specification information to determine whether each piece of information included in the label specification information is also included in the package information (step S171). In other words, the information processing system 5 can improve the reliability of the detected label specification information by using information that has been previously recognized as package information when detecting the label specifications. The information that has been previously recognized includes, for example, the destination of the package 12, i.e., the destination written on the label 13, and the number of packages 12, i.e., the number of labels 13 attached to the packages 12.

[0143] If each piece of information included in the label specifications is also included in the package information (Yes in step S171), the identification unit 112 determines that the matching is successful and terminates the process shown in Figure 15.

[0144] If any of the information included in the label specification detection information is not included in the package information (No. in step S171), the identification unit 112 determines that the matching has failed and proceeds to step S161. In other words, the information processing device 100 restarts the initialization of the exposure pattern, sets initial values ​​for an exposure pattern different from the original exposure pattern, and repeats the process shown in Figure 15.

[0145] In Figure 15, the information processing system 5 is shown as executing each process without considering the illuminance difference, but this is not the only example. The information processing system 5 may execute each process in Figure 15 only if the illuminance difference at the predetermined location is greater than or equal to the threshold th.

[0146] As shown in the example of operation in Figure 15, the information processing system 5 can perform label specification detection while taking into account the brightness at the location where the cargo 12, which is unloaded or lifted by a forklift or the like, is placed. Therefore, even when the brightness and darkness differ greatly, by performing label specification detection based on multiple images captured with multiple exposure patterns and synthesizing the label specification information, it is possible to easily supplement the label specification information that could not be obtained from each image, making it easier to obtain the label specification information for each label 13. In addition, the information processing device 100 may synthesize the multiple images captured with different exposure patterns themselves, but by synthesizing the detection results, the time required to perform label specification detection can be shortened.

[0147] Thus, the image acquisition unit 111 acquires multiple image data captured with multiple different exposure patterns from the imaging device 300, and the identification unit 112 may identify destination information based on the multiple image data. Furthermore, the information processing system 5 can shorten the time until label reading is completed by capturing images with different exposure patterns only when the illuminance difference is greater than or equal to a threshold th.

[0148] (Fourth embodiment) In the fourth embodiment, we will describe a case in which the pile of goods 15 and the label 13, which should be visible in the overall image GN captured by the imaging device 300, are not included in the image.

[0149] Figure 16 is an illustrative diagram showing an example configuration of the information processing system 5 of this embodiment. In Figure 16, components similar to those in the information processing system 5 of Figures 1, 8, or 11 are denoted by the same reference numerals, and their descriptions are omitted or simplified.

[0150] The information processing system 5 includes an information processing device 100, a host system 200, an imaging device 300, a mobile unit 400, and wireless communication equipment 500. The imaging device 300 includes a pan / tilt camera 300A and a fixed camera 300B. The information processing device 100, the host system 200, the imaging device 300, the mobile unit 400, and the wireless communication equipment 500 are communicated together via network NT.

[0151] The information processing device 100 communicates with the host device 200, the imaging device 300, the mobile device 400, etc., via the communication device 130.

[0152] The higher-level device 200 is a device that controls a controlled object that exists below the higher-level device 200. The controlled object is, for example, a mobile body 400. The mobile body 400 travels, for example, within a factory or warehouse. The mobile body 400 includes, for example, an Autonomous Mobile Robot (AMR). The higher-level device 200 includes, for example, an AMR control device that controls the AMR. The mobile body 400 can transport a pile of goods 15 in units of skids 14. In other words, the mobile body 400 can transport one or more loads 12 stacked on a single pallet 11.

[0153] The wireless communication equipment 500 is, for example, a local base station or access point, and enables communication between multiple communication devices wirelessly or via wired connections. For example, the host device 200 transmits various instructions (e.g., transport instructions) to the mobile unit 400 via the wireless communication equipment 500.

[0154] Figure 17 is a block diagram showing an example configuration of the information processing system 5 of this embodiment. In Figure 17, components similar to those in the information processing system 5 of Figures 1, 8, or 11 are denoted by the same reference numerals, and their descriptions are omitted or simplified.

[0155] The processor 110 of the information processing device 100 has, in addition to at least some of the functions of the components shown in the first to third embodiments, the function of a mobile body control unit 114. The mobile body control unit 114 controls the mobile body 400. The mobile body control unit 114 may determine whether or not the load 15 is visible in the acquired overall image GN. In other words, it may determine whether or not the load 15 is included in the imaging range CR, or whether or not the load 15 is present at a predetermined location. If the load 15 is not visible in the overall image GN, the mobile body control unit 114 may control the mobile body 400 or the host device 200 to move the load 15 so that it is included in the imaging range CR. In this case, the mobile body control unit 114 may transmit a movement control signal to move the load 15 via the communication device 130. An example of a case where the load 15 is not included in the overall image GN is when the forklift places the load 12 in a location other than the predetermined location.

[0156] Furthermore, the mobile unit control 114 may determine whether or not the label 13 is visible in the acquired overall image GN. For example, if the label 13 is not attached to the front of the luggage 12, or if the luggage 12 with the label 13 attached is rotated along the horizontal plane, the label 13 will not be visible in the overall image GN. The front of the luggage 12 is, for example, the surface facing (directly facing) the imaging direction of the imaging device 300. If the label 13 is not visible, the mobile unit control 114 may control the mobile unit 400 or the host device 200 to change the orientation of the luggage pile 15. In this case, the mobile unit control 114 may transmit an orientation change signal via the communication device 130. For example, the mobile unit control 114 may control the orientation of the luggage pile 15 so that the label 13 is included in the imaging range CR.

[0157] Next, the operation of the information processing system 5 of this embodiment will be described.

[0158] Figure 18 is a sequence diagram showing an example of the operation of the information processing system 5 of this embodiment. Note that in Figure 18, the explanations of the processes shown in Figures 6-8, 10, or 15 may be omitted or simplified.

[0159] The mobile body control unit 114 of the information processing device 100 initializes a mobile body control pattern (also referred to as an AMR pattern, for example) for controlling the mobile body 400 (step S181). The mobile body control pattern is information indicating how the mobile body 400 will move the load 12. For example, the mobile body control pattern may include rotating the load 15 by a predetermined angle (e.g., 90 degrees) via the pallet 11 along a horizontal plane, moving the load 15 a predetermined distance along the horizontal plane via the pallet 11, etc. In other words, the mobile body control pattern may include the rotation angle (amount of rotation), rotation speed, rotation direction, travel distance (amount of travel), travel speed, travel direction, etc. Information on configurable mobile body control patterns may be stored in advance in a memory 120, for example. Here, the mobile body control unit 114 requests (mobile body control request) that the mobile body control pattern be initialized to a predetermined mobile body control pattern and transmits a mobile body control signal to the host device 200, for example.

[0160] The higher-level device 200 receives a request from the information processing device 100 for initial setup of the mobile control pattern (mobile control request) and transmits the mobile control request to the mobile body 400 (step S311). In this case, the higher-level device 200 receives, for example, a mobile control signal and transmits the mobile control request to the mobile body 400. In response to the mobile control request, the mobile body 400 moves or initializes the orientation of the load 15 containing the load 12 via the pallet 11.

[0161] The identification unit 112 transmits an imaging request to the imaging device 300 (step S182).

[0162] The imaging device 300 receives an imaging request from the information processing device 100, for example, by receiving an imaging request signal (step S221). The imaging device 300 captures the overall image GN in response to the imaging request and transmits the captured overall image GN to the information processing device 100 (step S221).

[0163] In the information processing device 100, the image acquisition unit 111 receives the overall image GN from the imaging device 300 via the communication device 130 (step S183). Based on the overall image GN, the identification unit 112 detects the pallet position PA, the label position LA, and the luggage position CA according to the object detection model (step S184).

[0164] The specific unit 112 performs skid label position detection processing (step S185). Then, The identification unit 112 derives label specification information for at least one label 13 for each skid 14 by detecting label specifications based on the rotating camera 300A or the fixed camera 300B. The identification unit 112 stores the derived at least one label specification information for each skid 14 in the memory 120 (step S186).

[0165] The mobile body control unit 114 determines whether a mobile body control pattern has already been set from among the configurable mobile body control patterns (for example, mobile body patterns held in memory 120), that is, whether there are any unset mobile body control patterns (step S187). For example, suppose the mobile body control pattern is to rotate the load 15 by 90 degrees. In this case, by repeatedly performing control according to the mobile body control pattern, it may be possible to rotate the load 0 degrees, 90 degrees, 270 degrees, or 360 degrees (=0 degrees) relative to a predetermined placement location.

[0166] If not all mobile control patterns have been set, that is, if there are unset mobile control patterns (No. in step S187), the mobile control unit 114 requests (control update request) that one of the unset mobile control patterns be updated (changed) to one of the mobile control patterns (step S188). In this case, the mobile control unit 114 may send a control update request signal to the host device 200 via the communication device 130.

[0167] The higher-level device 200 receives a control update request from the information processing device 100, for example, by receiving a control update request signal. The higher-level device 200 transmits the control update request signal to the mobile body 400 (step S312). The mobile body 400 receives the control update request and, in response to the control update request, rotates or moves the load 15 via the pallet 11.

[0168] The processing of the information processing device 100 proceeds from step S188 to step S182. That is, the information processing device 100 repeatedly detects the label specifications of at least one label 13 for each skid 14, based on an overall image GN that includes the load 15 whose orientation and position have been changed according to a predetermined moving object control pattern.

[0169] If all mobile control patterns have already been set, that is, if there are no unset mobile control patterns (Yes in step S187), the identification unit 112 synthesizes the label specification information obtained for each mobile control pattern (step S189). As a result, the identification unit 112 obtains the label specification information for the labels 13 of all skids 14 placed in predetermined locations.

[0170] Furthermore, the higher-level device 200 transmits the package information stored in the package information database 210 to the information processing device 100 (step S313).

[0171] In the information processing device 100, the identification unit 112 acquires package information from the host device 200 via the communication device 130 (step S190).

[0172] The identification unit 112 compares the package information with the label specification information to determine whether each piece of information included in the label specification information is also included in the package information (step S191). In other words, the information processing system 5 can improve the reliability of the detected label specification information by using information that has been previously recognized as package information when detecting the label specifications. The information that has been previously recognized includes, for example, the shipping address of the package, i.e., the shipping address written on label 13, and the number of packages, i.e., the number of labels 13.

[0173] If each piece of information included in the label specification detection information is also included in the package information (Yes in step S190), the identification unit 112 determines that the matching is successful and terminates the process shown in Figure 18.

[0174] If any of the information included in the label specification detection information is not included in the package information, the identification unit 112 determines that the matching has failed and proceeds to step S181. In other words, the process is restarted from the initialization of the mobile body control pattern, and initial values ​​for a mobile body control pattern different from the original mobile body control pattern (for example, rotation at different angles, movement at different distances) are set, and the process in Figure 18 is repeated.

[0175] In this example, the mobile control unit 114 of the information processing device 100 transmits mobile control requests and control update requests to the mobile device 400 via the communication device 130 and the higher-level device 200, but it is not limited to this. The mobile control unit 114 may also transmit mobile control requests and control update requests directly to the mobile device 400 via the communication device 130.

[0176] Thus, the information processing system 5 of this embodiment can adjust so that the label 13 is included within the imaging range CR even if the cargo pile 15 or the label 13 is not visible in the initial image captured by the imaging device 300, thereby enabling proper label reading and acquisition of label specification information. Therefore, the information processing system 5 can easily supplement label specification information that could not be obtained in each image by performing label specification detection and synthesis based on multiple overall image GNs captured while the cargo 12 is controlled by multiple moving body control patterns. Thus, the information processing system 5 can easily obtain label specification information for each label 13 of the skids 14.

[0177] For example, by repeatedly rotating the load 12 by 90 degrees using a movement control pattern, if a label 13 is attached to any side of the rectangular load 12, the label specifications detection result for the label 13 can be obtained. Therefore, the information processing system 5 eliminates the need for the worker to rotate or move the load 12 themselves, reducing the burden on the worker and shortening the time required to detect the label specifications.

[0178] (Fifth embodiment) The fifth embodiment is a modification of the first embodiment. In this embodiment, the same matters as in the first embodiment will be omitted or simplified in their explanation.

[0179] The configuration of the information processing system 5 in the fifth embodiment is the same as that of the information processing system 5 in the first embodiment, so its description will be omitted.

[0180] Figure 19 is a sequence diagram showing an example of the operation of the information processing system 5 in this embodiment. In Figure 19, the same reference numerals are used for processes similar to those in Figure 5, and their explanations are omitted or simplified. In Figure 19, each label 13 within the same skid has the same destination information.

[0181] First, as in Figure 5, the information processing device 100 performs the processes in steps S101 to S106, and the pan-tilt camera 300A performs the processes in steps S201 and S202. As a result, the information processing device 100 acquires a zoomed image from the imaging device 300.

[0182] The identification unit 112 performs image recognition on the zoomed image, reads the code 13b, and determines whether or not it succeeded in reading the contents of the code 13b (code reading, code detection) (step S501).

[0183] If the code reading is successful (Yes in step S501), the identification unit 112 proceeds to step S109 and transmits the information of the read code 13b as label specification information to the host device 200 via the communication device 130.

[0184] On the other hand, if code reading fails (No. in step S501), the identification unit 112 performs OCR and reads the contents (text information) of each item 13a written on the label 13 (OCR reading, OCR detection). The identification unit 112 then determines whether it succeeded in reading the contents of at least some of the items 13a (step S502).

[0185] If the OCR reading fails to read the contents of all items 13a (No. in step S502), the identification unit 112 proceeds to step S104 and continues processing other labels 13 in the same skid as the label 13 that was the target of reading.

[0186] On the other hand, if the OCR reading successfully reads at least part of the contents of each item 13a (Yes in step S502), the identification unit 112 saves or updates the contents of the items 13a that were successfully read by OCR (OCR reading results) in the memory 120 (step S503).

[0187] The identification unit 112 determines whether or not it has succeeded in OCR reading all items 13a, that is, whether or not it has obtained the contents of all items 13a (OCR reading results) (step S504).

[0188] If the OCR reading results for at least some of the items 13a have not been obtained (No. in step S504), the identification unit 112 proceeds to step S104 and continues processing other labels 13 in the same skid as the label 13 that was to be read.

[0189] On the other hand, if the OCR reading results for all items 13a are obtained (Yes in step S504), the identification unit 112 proceeds to step S109 and transmits the information of the contents of each item 13a (for example, all items) that was successfully read as label specification information to the host device 200 via the communication device 130.

[0190] Then, the information processing device 100 performs the processing in steps S110 and S111.

[0191] Furthermore, if the OCR reading results for at least some of the items 13a have not been obtained for a given label 13 (No. in step S504), and the label information reading is repeatedly performed for other labels 13 within the same skid, the identification unit 112 may sequentially add and save the contents of the read items 13 each time it successfully performs OCR reading of any of the items 13a. In this case, the identification unit 112 may exclude from OCR reading items 13a whose reading results have already been successfully stored from the items 13a that have been previously read successfully. For example, if the information for items 2 and 3 has been read, the identification unit 112 may limit the reading to the information of other items excluding items 2 and 3, since the information for items 2 and 3 is already stored in the memory 120. Then, if the information of other items 13a is successfully read, the identification unit 112 may store at least some of the information of the other items that were successfully read in the memory 120. By repeatedly performing such processing, the identification unit 112 can increase the likelihood of ultimately obtaining information for all items 13a.

[0192] Furthermore, when repeatedly reading label information for other labels 13 within the same skid for a given label 13, the identification unit 112 may perform code reading again when reading the label information of other labels 13. In this case, the identification unit 112 may replace the previously held information with the information obtained from the successful re-execution of code reading (corresponding to the information for all items) without using the information of any of the items 13a already held in memory 120.

[0193] As described above, in this embodiment, the information processing system 5, the information processing device 100 reads a code (identification code) from the label 13 as label information by code reading. If the reading is successful, it obtains label specification information (in this case, the content corresponding to code 13b) based on the code 13b that was successfully read. If the information processing device 100 fails to read code 13b, it obtains text information from the label 13 as label information by OCR reading. In this case, if the information processing device 100 fails to obtain text information for some items 13a, it performs code reading and OCR reading again on other labels in the same skid, and based on the information obtained from the re-execution, it fills in the missing parts of the label information.

[0194] In code reading, if code 13b is soiled or damaged, it may become impossible to read the code, making it impossible to obtain label information. Also, OCR reading has lower reading accuracy and can take longer than code reading. In contrast, the information processing system 5 of this embodiment uses both code reading and OCR reading to make it easier to obtain label information and suppress the increase in processing time.

[0195] The information processing device 100 first attempts to read the code, and if successful, terminates processing using the acquired information. This enables the information processing device 100 to acquire information at high speed.

[0196] If code reading fails, the information processing device 100 performs OCR reading. Although OCR reading takes more time than code reading, it increases the likelihood of obtaining information even if code 13b cannot be read by code reading.

[0197] Furthermore, the information processing device 100 has a function to supplement information from other labels 13 within the same skid if some items 13a cannot be obtained during OCR reading. By ensuring that the labels 13 within the same skid contain the same information, the information processing device 100 can increase the likelihood of supplementing missing information by reading other labels 13. As a result, the information processing device 100 can significantly reduce the risk of information loss due to soiling or damage to the labels 13, and achieve more reliable information acquisition.

[0198] In this way, the information processing device 100 can acquire information quickly and reliably by using both code 13b reading and OCR-based text information reading. Furthermore, the information processing device 100 can acquire information using OCR even if code 13b cannot be read. In addition, the information processing device 100 can improve the resistance of label 13 to contamination and damage by supplementing information within the same skid. Furthermore, the information processing device 100 can improve the accuracy of reading the information contained in label 13 and reduce the risk of information loss.

[0199] (Summary of the embodiment) Based on the above, this disclosure contains at least the following information. The components and other elements in parentheses are examples of those corresponding to the embodiments described above, but are not limited to these.

[0200] (Item 1) An image acquisition unit (image acquisition unit 111) acquires image data (overall image GN) obtained from an imaging device (imaging device 300) that images a pile of cargo (pile 15) including at least one pallet (pallet 11) and at least one load (load 12) loaded on the pallet, Based on the acquired image data, the position of the pallet (pallet 11) (pallet position PA) and the position of the label (label 13) attached to the cargo (label position LA) are detected. Based on the detected position of the pallet and the position of the label or the position of the cargo, a skid (skid 14) is identified for each pallet, which is an area including the area where the cargo is located. A specific unit (specific unit 112) reads label information (label specification information), which is information shown on at least one of the labels in the image data, and identifies destination information regarding the destination of the cargo for each skid based on the label information, A transmission unit (communication device 130) transmits the destination information identified by the identification unit to a higher-level device (higher-level device 200), An information processing device (information processing device 100) having [a specific feature].

[0201] This allows the information processing device to manage a collection of goods stacked on a single pallet as a skid. Therefore, even if multiple pallets are stacked vertically, for example, the destination of the goods can be identified on a skid-by-skid basis, thereby accurately determining the destination of the goods on each pallet and improving the accuracy of identifying the destination of the goods.

[0202] (Item 2) The aforementioned at least one pallet includes a plurality of pallets stacked vertically, The information processing device described in item 1.

[0203] This allows the information processing device to improve the accuracy of identifying the destination of the goods loaded on each pallet, even when multiple pallets are stacked vertically.

[0204] (Item 3) The at least one pallet includes a first pallet and a second pallet stacked vertically above the first pallet. The identifying unit identifies the space between the first pallet and the second pallet as a skid. The information processing device described in item 2.

[0205] This allows the information processing device to identify skids as adjacent skids in the vertical direction.

[0206] (Item 4) The at least one pallet includes a third pallet that is not stacked with other pallets in the vertical direction. The specified unit identifies the space between the third pallet and the top of the pile of goods stacked on the third pallet as a skid. The information processing device described in item 2.

[0207] This allows the information processing device to identify skids even when multiple pallets are not stacked vertically.

[0208] (Item 5) When a skid contains multiple packages, the identifying unit identifies the first destination information of the first package among the multiple packages based on the label, and then identifies that the second destination information of the second package contained in the same skid is the same as the first destination information. An information processing device as described in any one of items 1 through 4.

[0209] As a result, the information processing device can identify the destination information of at least one piece of luggage within the same skid, thereby identifying that the destination information of other pieces of luggage is the same. Therefore, the information processing device can reduce the time required to identify the destination information of luggage within the skid and improve the efficiency of identifying destination information.

[0210] (Item 6) The imaging device includes a pan-type camera (pan-type camera 300A) and a fixed camera (fixed camera 300B), The aforementioned pan-type camera has a changeable imaging range (imaging range CR), The fixed camera has an image imaging range that cannot be changed. The identification unit identifies the destination of the cargo for each skid based on the third destination information identified using the first image data acquired from the rotating camera and the fourth destination information identified using the second image data acquired from the fixed camera. An information processing device as described in any one of items 1 through 5.

[0211] As a result, the information processing device can improve the accuracy of identifying the destination of a package based on high-resolution images of the label obtained by a rotating camera, and can also quickly identify the destination of a package based on images obtained at high speed by a fixed camera.

[0212] (Item 7) The system further includes an imaging condition control unit (imaging condition control unit 113) that controls the imaging conditions of the imaging device, The imaging condition control unit instructs the imaging device to image the cargo pile with a plurality of different exposure patterns. The image acquisition unit acquires a plurality of image data captured with the plurality of different exposure patterns from the imaging device. The identification unit identifies the destination information based on the plurality of image data. An information processing device as described in any one of items 1 through 6.

[0213] As a result, the information processing device can accurately identify the destination information of the luggage, as indicated on the label attached to the luggage, even when there is a large difference in illumination (difference in brightness) at the imaging location (predetermined placement location) where the luggage is imaged.

[0214] (Item 8) The imaging condition control unit, The illuminance difference within the aforementioned image data is detected, If the illuminance difference is greater than or equal to a threshold (th), the system instructs the system to image the cargo pile using the multiple different exposure patterns. The information processing device described in item 7.

[0215] As a result, the information processing device only captures images with multiple exposure patterns to identify the destination of the cargo when the illuminance difference within the image data is above a threshold, thereby reducing the processing load when identifying the destination of the cargo.

[0216] (Item 9) The system further includes a mobile control unit (mobile control unit 114) that controls the mobile unit (mobile unit 400), The aforementioned mobile body control unit is Determine whether the cargo pile is visible in the acquired image data. If the aforementioned cargo pile is not visible, the mobile body or the higher-level device controlling the mobile body is instructed to move the cargo pile. An information processing device as described in any one of items 1 through 8.

[0217] As a result, the information processing device can automatically transport packages to a designated location even if they are placed in a location different from their designated location, and can identify the destination of the packages.

[0218] (Item 10) It further comprises a mobile body control unit for controlling the mobile body, The aforementioned mobile body control unit is Determine whether the label is visible in the acquired image data. If the label is not visible, the mobile body or the higher-level device controlling the mobile body is instructed to change the orientation of the load. An information processing device as described in any one of items 1 through 8.

[0219] As a result, the information processing device can adjust the position and orientation of the cargo pile so that the labels are included in the imaging range of the imaging device, even if, for example, the side of the label attached to the cargo differs depending on the cargo, and the orientation of each cargo is unpredictable when multiple cargoes are arranged. Therefore, the information processing device can suitably identify the destination of the cargo based on the image in which the labels are captured.

[0220] (Item 11) The label information includes an identification code (code 13b) and text information (text information of item 13a), The specified part is, The identification code of the label information of the first label is read, If reading the identification code fails, the text information of the label information of the first label is read. If reading at least a portion of the text information of the label information of the first label fails, the label information of the second label in the skid containing the first label is read. An information processing device that meets any one of the following criteria: 1 through 10.

[0221] This allows the information processing device to identify destination information by combining reading using identification codes and reading using text information. Furthermore, the information processing device can efficiently read by reading the easily readable identification codes first, and can increase the likelihood of successful reading by using the reading of text information as a supplement. In addition, if the information processing device fails to read both the identification codes and the text information, it can increase the likelihood of identifying common destination information for multiple labels within the same skid by reading label information from other labels within the same skid.

[0222] (Item 12) An information processing method in which a processor (processor 110) and memory (memory 120) cooperate to perform, Image data obtained by imaging a pile of cargo including at least one pallet and at least one load loaded on the pallet, Based on the acquired image data, the position of the pallet and the position of the label attached to the package are detected. Based on the detected position of the pallet and the position of the label or the position of the cargo, for each pallet, a skid is identified which is an area including the area where the cargo is located. The process involves reading the label information, which is the information shown on at least one of the labels in the image data, and identifying destination information for each skid regarding the destination of the cargo based on the label information. The identified destination information is transmitted to the higher-level device. An information processing method having

[0223] As a result, the information processing method achieves the same effect as item 1.

[0224] (Item 13) A program that causes a computer to execute the information processing methods described in item 12.

[0225] This allows the program to achieve the same effect as item 1.

[0226] Although various embodiments have been described above with reference to the drawings, it goes without saying that this disclosure is not limited to such examples. It is clear to those skilled in the art that various modifications or alterations can be conceived within the scope of the claims, and these will naturally also fall within the technical scope of this disclosure. Furthermore, the components of the above embodiments may be combined in any way without departing from the spirit of the invention.

[0227] Furthermore, the above embodiment may also apply to a program that realizes the functions of an information processing method, which is supplied to a computer (e.g., an information processing device 100) via a network or various storage media, and which is read and executed by the processor of this computer, as well as a recording medium on which this program is stored. [Industrial applicability]

[0228] This disclosure is useful for information processing devices, information processing methods, and programs that can improve the accuracy of identifying the destination of cargo. [Explanation of Symbols]

[0229] 5. Information Processing Systems 11 Palettes 12 pieces of luggage 13 Labels 14 Skid 15 Mt. 100 Information Processing Devices 110 processors 111 Image acquisition unit 112 Specific part 113 Imaging Condition Control Unit 114 Mobile Unit Control Unit 120 memory 130 Communication Devices 200 Higher-level equipment 210 Package Information Database 300 Imaging devices 300A Swivel Camera 300B Fixed Camera 400 Mobile Units 500 Wireless communication equipment CA baggage location CR imaging range GN overall image GL Label Image LA label position PA Palette Position SA Skid Range

Claims

1. An image acquisition unit acquires image data obtained from an imaging device that images a pile of cargo including at least one pallet and at least one load placed on the pallet, Based on the acquired image data, the position of the pallet and the position of the label attached to the package are detected. Based on the detected position of the pallet and the position of the label or the position of the cargo, for each pallet, a skid is identified which is the area including the area where the cargo is located. A identifying unit reads label information, which is information indicated on at least one of the labels in the image data, and identifies destination information relating to the destination of the cargo for each skid based on the label information. An information processing device having

2. The aforementioned at least one pallet includes a plurality of pallets stacked vertically, The information processing apparatus according to claim 1.

3. The at least one pallet includes a first pallet and a second pallet stacked vertically above the first pallet. The specified unit identifies the space between the first pallet and the second pallet as a skid. The information processing apparatus according to claim 2.

4. The at least one pallet includes a third pallet that is not stacked with other pallets in the vertical direction. The specified unit identifies the space between the third pallet and the top of the pile of goods stacked on the third pallet as a skid. The information processing apparatus according to claim 2.

5. When a skid contains multiple packages, the identifying unit identifies the first destination information of the first package among the multiple packages based on the label, and then identifies that the second destination information of the second package contained in the same skid is the same as the first destination information. The information processing apparatus according to claim 1 or 2.

6. The imaging device includes a pan-type camera and a fixed camera. The aforementioned pan-rotating camera has a changeable imaging range. The fixed camera has an image imaging range that cannot be changed. The identification unit identifies the destination of the cargo for each skid based on the third destination information identified using the first image data acquired from the rotating camera and the fourth destination information identified using the second image data acquired from the fixed camera. The information processing apparatus according to claim 1 or 2.

7. The system further includes an imaging condition control unit that controls the imaging conditions of the imaging device, The imaging condition control unit instructs the imaging device to image the cargo pile with a plurality of different exposure patterns. The image acquisition unit acquires a plurality of image data captured with the plurality of different exposure patterns from the imaging device. The identification unit identifies the destination information based on the plurality of image data. The information processing apparatus according to claim 1 or 2.

8. The imaging condition control unit, The illuminance difference within the aforementioned image data is detected, If the illuminance difference is greater than or equal to a threshold, the system is instructed to image the cargo pile using the multiple different exposure patterns. The information processing apparatus according to claim 7.

9. It further comprises a mobile body control unit for controlling the mobile body, The aforementioned mobile body control unit is Determine whether the cargo pile is visible in the acquired image data. If the aforementioned load is not visible, the mobile body or the higher-level device controlling the mobile body is instructed to move the load. The information processing apparatus according to claim 1 or 2.

10. It further comprises a mobile body control unit for controlling the mobile body, The aforementioned mobile body control unit is Determine whether the label is visible in the acquired image data. If the label is not visible, the mobile body or the higher-level device controlling the mobile body is instructed to change the orientation of the load. The information processing apparatus according to claim 1 or 2.

11. The aforementioned label information includes an identification code and text information. The specified part is, The identification code of the label information of the first label is read, If reading the identification code fails, the text information of the label information of the first label is read. If reading at least a portion of the text information of the label information of the first label fails, the label information of the second label in the skid containing the first label is read. The information processing apparatus according to claim 1 or 2.

12. An information processing method in which a processor and memory work together, Image data obtained by imaging a pile of cargo including at least one pallet and at least one load loaded on the pallet, Based on the acquired image data, the position of the pallet and the position of the label attached to the package are detected. Based on the detected position of the pallet and the position of the label or the position of the cargo, for each pallet, a skid is identified which is an area including the area where the cargo is located. The process involves reading the label information, which is the information shown on at least one of the labels in the image data, and identifying destination information for each skid related to the destination of the cargo based on the label information. An information processing method having

13. A program that causes a computer to execute the information processing method described in claim 12.

Images