Information processing device, information processing method, and program

Abstract

To provide an information processing device, an information processing method and a program which can more properly grasp a relation between a movable body and an object at a position expressed by acquired position information on the movable body.SOLUTION: An information processing device comprises one or more processing units which acquire position information on a movable body, acquire information on a direction of the movable body, acquire placement information showing whether or not an object different from the movable body is placed on the movable body, and generate correspondence information in which the position information, the information on the direction of the movable body, and the placement information are made to correspond to one another in a time-series manner.SELECTED DRAWING: Figure 13

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, as a method for managing the position of an object carried by a moving body, identification marks are attached in advance to the object and the storage location of the object. After the moving body has carried the object to a certain storage location, the user operates a reading device to read the identification marks of the object and the storage location, and registers the position of the object after the movement (for example, Patent Document 1). Further, instead of a method for specifying a storage location, the position of the object can also be managed by grasping the relationship between the moving body represented by position information and the object carried by the moving body through user operation.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the method of managing by user operation, there is a problem that due to work mistakes or the like, the relationship between the moving body and the object at the position represented by the acquired position information of the moving body cannot be appropriately grasped.

[0005] An object of this invention is to more appropriately grasp the relationship between the moving body and the object at the position represented by the acquired position information of the moving body.

Means for Solving the Problems

[0006] To solve the above problems, the information processing apparatus according to the present invention acquires position information of a moving body, The orientation information of the moving body is acquired, Acquire placement information indicating whether or not an object different from the aforementioned moving body is placed on the moving body. Correspondence information is generated by chronologically associating the aforementioned position information, the orientation information of the moving body, and the previously described position information with each other. Equipped with one or more processing units 、 The position information is generated based on the positional relationship between a fixed reference object and the moving object. .

[0007] To solve the above problems, the information processing method according to the present invention is An information processing method performed by a computer in an information processing device, Obtain the location information of the moving object, The orientation information of the moving body is acquired, Acquire placement information indicating whether or not an object different from the aforementioned moving body is placed on the moving body. Correspondence information is generated by chronologically associating the aforementioned position information, the orientation information of the moving object, and the previously described position information with each other. death, The positional information is generated based on the positional relationship between a fixed reference object and the moving object.

[0008] To solve the above problems, the program according to the present invention In the computer of the information processing device, Process to acquire the location information of a moving object. A process for obtaining information on the orientation of the moving object. A process for acquiring placement information indicating whether or not an object different from the aforementioned moving body is placed on the moving body. A process for generating correspondence information by chronologically associating the aforementioned position information, the orientation information of the moving body, and the previously described position information with each other. Execute Let, The positional information is generated based on the positional relationship between a fixed reference object and the moving object. [Effects of the Invention]

[0009] According to the present invention, the relationship between the moving object and the object at the location represented by the acquired position information of the moving object can be understood more appropriately. [Brief explanation of the drawing]

[0010] [Figure 1] It is a diagram showing the configuration of the luggage management system. [Figure 2] It is a block diagram showing the functional configuration of the vehicle. [Figure 3] It is a block diagram showing the functional configuration of the server. [Figure 4] It is a diagram explaining the light emission pattern of the LED marker. [Figure 5] It is a schematic diagram showing the configuration of the warehouse. [Figure 6] It is a diagram explaining the storage mode of luggage in the storage area. [Figure 7] It is a diagram showing an example of the content of the status data. [Figure 8] It is a diagram showing an example of the content of the luggage management data. [Figure 9] It is a schematic diagram showing the warehouse at the second point in time. [Figure 10] It is a schematic diagram showing the warehouse at the fourth point in time. [Figure 11] It is a schematic diagram showing the warehouse at the fifth point in time. [Figure 12] It is a schematic diagram showing the warehouse at the sixth point in time. [Figure 13] It is a flowchart showing the control procedure of the correspondence information transmission process. [Figure 14] It is a flowchart showing the vehicle operation flow. [Figure 15] It is a flowchart showing the control procedure of the luggage management process. [Figure 16] It is a diagram showing the case where the position of the vehicle is different from the position of the luggage to be loaded and unloaded. [Figure 17] It is a diagram showing the configuration of the luggage management system of the second embodiment. [Figure 18] It is a block diagram showing the functional configuration of the vehicle of the second embodiment. [Figure 19] It is a block diagram showing the functional configuration of the server of the second embodiment. [Figure 20]This figure illustrates the light emission pattern of the LED marker according to the second embodiment. [Figure 21] This is a flowchart showing the control procedure for the LED marker light emission process in the second embodiment. [Figure 22] This flowchart shows the control procedure for the correspondence information generation process in the second embodiment. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments of the present invention will be described based on the drawings.

[0012] (First embodiment) <Overview of the luggage management system> Figure 1 shows the configuration of the luggage management system 1. The cargo management system 1 (information processing system) of this embodiment is installed in a warehouse 500 for storing cargo (objects). The cargo management system 1 comprises a vehicle 10 (mobile body), a server 20, and a plurality of LED markers 30 (reference objects, light-emitting devices), etc. The vehicle 10 is capable of sending and receiving data to and from the server 20 via a wireless communication path. The cargo management system 1 individually manages the storage location of a plurality of cargo L stored in the warehouse 500. In the following, the positions of the vehicle 10 and cargo L in the warehouse 500 are represented in the XYZ Cartesian coordinate system. The X and Y directions are parallel to the floor of the warehouse 500, and the Z direction is vertically upward. In Figure 1, the server 20 is depicted inside the warehouse 500, but it is not limited to this, and the server 20 may be installed outside the warehouse 500 or in a remote location.

[0013] <Vehicle Configuration> In this embodiment, the vehicle 10 is a forklift. The vehicle 10 includes forks 10a (loading section) on which the load L is placed, a mast 10b that raises and lowers the forks 10a in a substantially vertical direction relative to the ground, and two cameras 40 (imaging devices). The vehicle 10 can lift the load L with the forks 10a or lower the load L from the forks 10a and place it in a storage location, depending on the operation of the passenger (operator). The vehicle 10 can also move the load L within the warehouse 500 by moving the vehicle with the load L placed on the forks 10a. In detail, the cargo L is placed on a pallet P (base), and the pallet P is lifted together with the forks 10a. The pallet P is a flat rectangular parallelepiped with a roughly square shape in plan view, and has two through holes Pa that penetrate between a pair of sides. By inserting the forks 10a through each of these two through holes Pa and raising the forks 10a, the pallet P and cargo L can be lifted. Alternatively, two more through holes Pa may be provided in directions perpendicular to the through holes Pa shown in Figure 1, so that the forks 10a can be inserted into the through holes Pa from any side of the pallet P.

[0014] Figure 2 is a block diagram showing the functional configuration of vehicle 10. Vehicle 10 includes a terminal device 100 (information processing device), a luggage detection unit 16, a fork height detection unit 17, a direction detection unit 18, a camera 40, and the like. The terminal device 100 has a CPU 11 (Central Processing Unit), RAM 12 (Random Access Memory), a storage unit 13, an operation display unit 14, and a communication unit 15, etc. Each part of vehicle 10 is connected via a communication path 19 including a PoE hub, etc.

[0015] The terminal device 100 is, for example, a notebook PC, a tablet, or a smartphone, but is not limited to these. Furthermore, the terminal device 100 does not necessarily have to be provided separately from the vehicle 10, but may be incorporated as part of the vehicle 10 from the beginning.

[0016] The CPU 11 of the terminal device 100 is a processor that controls the operation of the terminal device 100 and the vehicle 10 by reading and executing the program 131 stored in the storage unit 13 and performing various arithmetic operations. In the first embodiment, the CPU 11 corresponds to "one or more processing units". The terminal device 100 may have multiple processors (multiple CPUs, etc.), and the multiple processes executed by the CPU 11 in this embodiment may be executed by these multiple processors. In this case, the multiple processors correspond to "one or more processing units". Furthermore, in this case, the multiple processors may be involved in common processing, or the multiple processors may independently execute different processes in parallel.

[0017] RAM12 provides CPU11 with a working memory space and stores temporary data.

[0018] The storage unit 13 is a non-temporary recording medium readable by the CPU 11 as a computer, and stores the program 131 and various data. The storage unit 13 includes non-volatile memory such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). The data stored in the storage unit 13 includes captured image data 132 related to images captured by the camera 40, and status data 133 representing the state of the vehicle 10. The specific contents of the status data 133 will be described later.

[0019] The operation display unit 14 includes a display device (e.g., a liquid crystal display) that displays information according to display control signals from the CPU 11, and an input device (e.g., a touch panel, mouse, and keyboard) that receives user input operations and outputs operation information corresponding to those input operations to the CPU 11.

[0020] The communication unit 15 is composed of a network card or a communication module, and transmits and receives data with the server 20 according to a predetermined communication standard. In this embodiment, the communication unit 15 transmits and receives data with the server 20 using wireless communication such as Wi-Fi (registered trademark).

[0021] The luggage detection unit 16 detects whether or not luggage L is placed on the fork 10a and outputs data related to the detection result to the CPU 11 of the terminal device 100. Based on the detection result by the luggage detection unit 16, placement information Ib (see Figure 7) indicating whether or not luggage L is placed is generated. The luggage detection unit 16 in this embodiment has a distance sensor that detects the distance to luggage L placed on the fork 10a, and detects the presence or absence of luggage L based on the difference in the detection result of the distance sensor when luggage L is placed on the fork 10a and when luggage L is not placed. However, the method of detecting luggage L is not limited to this, and for example, a method of detecting the weight of an object placed on the fork 10a, or a method of determining the presence or absence of luggage L based on an image of the fork 10a captured by the camera 40 (or a camera not shown) may be used.

[0022] The fork height detection unit 17 detects the height in the Z direction of the fork 10a, which is raised and lowered by the mast 10b, and outputs data related to the detection result to the CPU 11. Based on the detection result by the fork height detection unit 17, height information Id (see Figure 7) representing the height of the fork 10a is generated. The height in the Z direction of the fork 10a can also be determined by a geomagnetic sensor located inside the vehicle 10. Alternatively, the height in the Z direction of the fork 10a can also be determined by the positional relationship between the fork 10a and the LED marker 30, by setting the camera 40 to also image the fork 10a. By using the processing of images captured by the camera 40 as part of the function of the fork height detection unit 17, both the position information of the vehicle 10 and the height information Id can be obtained almost simultaneously, making the decision-making process of the CPU 11 more efficient.

[0023] The orientation detection unit 18 detects the orientation of the vehicle 10 and outputs data related to the detection result to the CPU 11. The orientation detection unit 18 has, for example, a geomagnetic sensor, and derives the orientation of the vehicle 10 based on the orientation of magnetic north derived by the geomagnetic sensor. Based on the detection result by the orientation detection unit 18, orientation information Ie (see FIG. 7) representing the orientation of the vehicle 10 is generated. Further, the orientation detection unit 18 may determine the orientation of the vehicle 10 based on information from the LED marker 30 imaged by the camera 40. By using the camera 40 as the orientation detection unit 18, both the position information and the orientation information Ie of the vehicle 10 can be obtained almost simultaneously, so that the determination process of the CPU 11 becomes efficient.

[0024] The camera 40 images the inside of the warehouse 500 at a predetermined frame rate and outputs the captured image data 132 to the terminal device 100. As shown in FIG. 1, two cameras 40 for imaging the upper front and upper rear of the vehicle 10 are provided on the vehicle 10 of the present embodiment.

[0025] <Configuration of LED Marker> Each of the plurality of LED markers 30 shown in FIG. 1 has three LEDs (Light Emitting Diodes) that emit red, green, and blue light, and can emit red, green, and blue light by switching at a predetermined switching cycle. The LED marker 30 is attached and fixed at a predetermined position on the ceiling of the warehouse 500. The plurality of LED markers 30 may be arranged in a regular array such as a matrix, or may be arranged at positions without regularity. Each LED marker 30 emits light in a light emission pattern 60 in which the emission colors of red, green, and blue change in time series. The light emission pattern 60 will be described later. The LED marker 30 may be communicatively connected to the server 20 and emit light according to the control by the server 20, or may be controlled by a device external to the luggage management system 1 to emit light.

[0026] <Configuration of Server> FIG. 3 is a block diagram showing the functional configuration of the server 20. The server 20 includes a CPU 21, RAM 22, storage unit 23, and communication unit 24. The various parts of the server 20 are connected via a bus 25. The server 20 may further include an operation unit for receiving user input and a display unit for displaying various information.

[0027] The CPU 21 is a processor that controls the operation of the server 20 by reading and executing the program 231 stored in the memory unit 23 and performing various arithmetic operations. The server 20 may have multiple processors (for example, multiple CPUs), and the multiple processes that the CPU 21 in this embodiment performs may be performed by these multiple processors. In this case, the multiple processors may be involved in common processing, or the multiple processors may independently execute different processes in parallel.

[0028] RAM22 provides CPU21 with a working memory space and stores temporary data.

[0029] The memory unit 23 is a non-temporary recording medium readable by the CPU 21 as a computer, and stores the program 231 and various data. The memory unit 23 includes non-volatile memory such as an HDD or SSD. The data stored in the memory unit 23 includes cargo management data 232 for managing cargo L stored in the warehouse 500. The specific contents of the cargo management data 232 will be described later.

[0030] The communication unit 24 is composed of a network card or communication module, and transmits and receives data with the terminal device 100 of the vehicle 10 in accordance with a predetermined communication standard.

[0031] <How the baggage management system works> The following is an overview of the operation for managing luggage L using luggage management system 1. The LED marker 30 emits light in a light emission pattern 60 that allows for identification of the LED marker 30. The terminal device 100 of the vehicle 10 acquires captured image data 132 of the image captured by the camera 40, identifies the LED marker 30 shown in the captured image, and derives the position of the vehicle 10 in the warehouse 500 based on the known position of the LED marker 30 in the warehouse 500 and the position of the LED marker 30 in the captured image. The terminal device 100 also generates correspondence information I (see Figure 7) which associates position information Ia (see Figure 7) related to the position of the vehicle 10, loading information Ib (see Figure 7) indicating whether or not the cargo L is loaded on the vehicle 10, height information Id (see Figure 7) related to the height of the fork 10a, and orientation information Ie related to the orientation of the vehicle 10, etc., in a time-series manner, and transmits it to the server 20. The server 20 determines the status of the package L based on the received correspondence information I, generates package status information J (see Figure 8) relating to the status of package L, and records it in the package management data 232. Here, "the status of package L" includes at least the position of package L, and may further include at least one of the following: the height of package L, the vehicle on which it is loaded, and the orientation of the insertion hole Pa of the pallet P. In this embodiment, an example in which "the status of package L" includes the position, height, vehicle on which it is loaded, and the orientation of the insertion hole Pa will be used for explanation. The operation of the package management system 1 will be described in detail below.

[0032] Figure 4 illustrates the light emission pattern 60 of the LED marker 30. The light emission pattern 60 of the LED marker 30 is a combination of predetermined units of time during which one of the following colors of light—red, green, and blue—is emitted. The light emission pattern 60 of this embodiment consists of 24 units of time S1 to S24. Such a light emission pattern 60 can transmit information represented by the sequence of 24 emitted colors. The number of units of time included in one light emission pattern 60 may be changed depending on the amount of information to be transmitted. The length of one unit of time is not particularly limited, but can be, for example, about 100 milliseconds. The LED marker 30 of this embodiment emits light with a unique light emission pattern 60 set for each LED marker 30. The light emission pattern 60 may represent, for example, a unique code corresponding to the LED marker 30, or it may directly represent the position of the LED marker 30 (for example, the X, Y, and Z coordinates).

[0033] The camera 40 installed on the vehicle 10 is set to an imaging frame rate (e.g., 20fps) such that the imaging period is shorter than the unit period of the light emission pattern 60. By detecting and analyzing the light emission pattern 60 of the LED markers 30 shown in the images from multiple images taken over a period that includes the entire light emission pattern 60, the LED markers 30 shown can be identified. Then, based on the known positions of each LED marker 30 within the warehouse 500 (the positions where the light-emitting devices are installed) and the positions of two or more LED markers 30 shown in the images within the images, the positional relationship between the LED markers 30 and the vehicle 10, and the position of the vehicle 10 within the warehouse 500 can be derived. Within a certain period of time after the position of the vehicle 10 has been identified (within a certain range of movement distance from the identified position), the position of the vehicle 10 can be derived with a certain degree of accuracy based on an image showing only a single LED marker 30. It is preferable to adjust the arrangement of the multiple LED markers 30 and the orientation of the cameras 40 so that at least one LED marker 30 is visible in the image captured by at least one of the cameras 40, regardless of the position and orientation of the vehicle 10. By providing two cameras 40 with different orientations, it is possible to ensure that the LED markers 30 are visible in the image captured by one of the cameras 40 even when the vehicle 10 is located at the edge of the warehouse 500 (near the wall).

[0034] The process of deriving the position of the vehicle 10 based on the image captured by the camera 40 is performed by the CPU 11 of the terminal device 100. The CPU 11 derives the X and Y coordinates of the vehicle 10 based on the captured image. The CPU 11 also identifies which of several predetermined areas in the warehouse 500 the derived coordinates belong to, and defines the identified area as the position information Ia of the vehicle 10.

[0035] Figure 5 is a schematic diagram showing the configuration of warehouse 500. The warehouse 500 of this embodiment has a plurality of storage areas 501 (predetermined storage areas) arranged in a 6x6 matrix, an receiving area 502 where goods L are received, and an outbound area 503 where goods L are dispatched. The plurality of storage areas 501, receiving area 502, and outbound area 503 constitute the aforementioned "multiple areas". Therefore, each of the "multiple areas" is one of the storage area 501, receiving area 502, and outbound area 503. The position of each storage area 501 is represented by a code that combines the code "A" to "F" representing the position in the Y direction and the code "1" to "6" representing the position in the X direction. For example, in Figure 5, the position of the storage area 501 where goods L004 is stored is "position C1". Also in Figure 5, goods L001 is in the receiving area 502, and the receiving operation is being carried out. In addition, a vehicle 10 is heading towards the receiving area 502 to move the received goods L001. In Figure 5, for convenience, vehicle 10 is represented by a rectangle with rounded corners. Also in Figure 5, dots are marked on storage areas 501 where one or more pieces of luggage L are stored. Note that the configuration of warehouse 500 shown in Figure 5 is illustrative and not limited to it. For example, the number and arrangement of storage areas 501 can be changed as appropriate. Also, a portion of the matrix of storage areas 501 may be the receiving area 502 and the shipping area 503.

[0036] In the receiving area 502, an receiving LED marker 31 with the same configuration as the LED marker 30 is provided. The receiving LED marker 31 emits light in a light emission pattern 60 that can identify the cargo L (cargo L001 in Figure 5) being received in the receiving area 502. The terminal device 100 of the vehicle 10 detects and analyzes the light emission pattern 60 of the receiving LED marker 31 captured by the camera 40 and obtains object information Ic (see Figure 7) (for example, cargo ID) that identifies the cargo L being received. The object information Ic may be associated with the light emission pattern 60 of the receiving LED marker 31 in table data or the like (not shown), or the object information Ic may be directly obtained from the light emission pattern 60.

[0037] Figure 6 is a diagram illustrating the storage configuration of cargo L in storage area 501. The storage area 501 of this embodiment is provided with a shelf 50 on which a vehicle 10 located in the storage area 501 can place luggage L. The shelf 50 has a lower storage area 51, a middle storage area 52, and an upper storage area 53, each with different heights in the Z direction (hereinafter referred to collectively as "storage areas 51-53"), and one piece of luggage L can be stored in each of these storage areas 51-53. Therefore, a maximum of three pieces of luggage L can be stored in one storage area 501. On the other hand, the height of the forks 10a of the vehicle 10 can be adjusted using the mast 10b to one of the following heights: H1 for the lower storage area 51, H2 for the middle storage area 52, or H3 for the upper storage area 53. By placing the cargo L on the shelf 50 with the height of the forks 10a adjusted to one of heights H1, H2, or H3, the cargo L can be placed in the storage area of ​​the corresponding height. Alternatively, instead of using the shelf 50, multiple packages L may be stacked from bottom to top (stacked or flat-stacked). In that case, for example, if there is already another package L (an object different from the one being placed) stacked at the bottom in the same location (a designated location), then when a package L is placed in that location, the CPU 11 will determine that the package L is placed as the second package from the bottom. Conversely, if there are two packages L stacked in the same location and only the top package L is to be placed on the vehicle 10, the CPU 11 will determine which package L will be placed on the vehicle 10. The same applies when packages are stacked in three or more layers. More specifically, when the CPU 11 places a package L placed on the fork 10a into the storage area, and when it lifts the package L from the storage area and places it on the fork 10a, it determines whether there is another package L different from this package L in the storage area, and obtains height information Id based on the determination result. For example, if there are other packages L in the storage area, the height of the second layer of the stack is used as the height information ID. Also, if the maximum number of stackable layers is three or more, the number of other packages L stacked in the storage area is identified, and the height information ID is obtained based on the identified number of other packages L. For example, if there are N layers of other packages L stacked in the storage area, the height of the (N+1)th layer of the stack is used as the height information ID.

[0038] The CPU 11 of the terminal device 100 associates the vehicle coordinates and position information Ia of the vehicle 10, derived from the image captured by the camera 40, with the aforementioned mounting information Ib, object information Ic, height information Id, orientation information Ie, etc., to generate correspondence information I. It also registers the generated correspondence information I in the status data 133.

[0039] Figure 7 shows an example of the contents of status data 133. Each row of data in status data 133 corresponds to one corresponding information I. Each corresponding information I has the following data items: "data number", "date and time", "vehicle ID", "location information Ia", "placement information Ib", "object information Ic", "height information Id", and "orientation information Ie".

[0040] The "data number" is a number or code assigned to multiple corresponding pieces of information I in chronological order. "Date and time" represents the date and time when correspondence information I was generated. Since correspondence information I is generated when location information Ia is acquired, it can also be said that "Date and time" represents the date and time when location information Ia was acquired. The "Vehicle ID" is a unique code assigned to the vehicle 10 on which the terminal device 100 is installed. The "vehicle coordinates" are the X and Y coordinates of the vehicle 10 derived from the image captured by the camera 40 using the method described above. "Location information Ia" is one of the storage area 501, entry area 502, or exit area 503 to which the "vehicle coordinates" belong, and represents the location of vehicle 10. "Loading Information Ib" indicates whether or not the load L is loaded onto the fork 10a. It is set to "1" if the load L is loaded onto the fork 10a, and to "0" if it is not loaded. "Object Information Ic" is the package ID of the placed package L when "Placement Information Ib" is "1". "Object Information Ic" is entered when the package ID has been identified based on the light emission pattern 60 of the receiving LED marker 31. Therefore, if the package L placed on the fork 10a was not placed in the receiving area 502 (i.e., the package L was placed in any of the storage areas 501), the package ID cannot be identified by the receiving LED marker 31, and "Object Information Ic" is set to "N / A". "Height Information Id" represents the height of the fork 10a detected by the fork height detection unit 17, and in this embodiment, it is one of "H1" to "H3". "Orientation information Ie" represents the orientation of the vehicle 10 detected by the orientation detection unit 18 (the forward direction in which the forks 10a are provided), and in this embodiment, it is one of "0" to "3". Of these, "0" represents the -X direction (upward direction in Figure 5), "1" represents the +Y direction (rightward direction in Figure 5), "2" represents the +X direction (downward direction in Figure 5), and "3" represents the +Y direction (leftward direction in Figure 5). However, the format of orientation information Ie is not limited to this, and the orientation may be represented by finer divisions such as 8 directions or 16 directions, or by numerical values ​​such as 0° to 360°. By referring to orientation information Ie in addition to position information Ia, even if the location where the cargo L was loaded and unloaded cannot be accurately determined by position information Ia alone (for example, when multiple loading locations in the storage area 501 are close together), the location where the cargo L was loaded and unloaded can be determined more accurately. This point will be explained later with specific examples in <Modification Example 1-3>.

[0041] The CPU 11 of the terminal device 100 repeatedly acquires location information Ia, placement information Ib, height information Id, orientation information Ie, etc., and generates corresponding information I at a predetermined frequency (i.e., at a predetermined transmission timing of corresponding information I), and registers each generated corresponding information I in the status data 133. In Figure 7, corresponding information I is generated at a frequency of once per minute, but this is for the sake of explanation, and corresponding information I may be generated at a higher frequency (for example, once every few seconds to tens of seconds). Furthermore, the CPU 11 of the terminal device 100 sends the generated correspondence information I to the server 20 each time it generates correspondence information I. Therefore, data identical in content to the status data 133 is stored in the server 20. Also, if the luggage management system 1 has multiple vehicles 10, the correspondence information I generated in each vehicle 10 is sent to and stored in the server 20. In the following, the 12 correspondence information I listed in Figure 7 will be referred to as "Correspondence Information I1" to "Correspondence Information I12" in chronological order of "Date and Time".

[0042] The CPU 21 of the server 20 determines the status of the package L based on the multiple pieces of correspondence I received, and registers the package status information J, including the determination result, in the package management data 232.

[0043] Figure 8 shows an example of the contents of the luggage management data 232. Each row of data in the cargo management data 232 corresponds to cargo status information J relating to the status of one cargo L under management at a specific date and time. Each piece of cargo status information J corresponds to one piece of corresponding information I. Each piece of cargo status information J has the data items "Date and Time", "Cargo ID", "Location", "Height", "Vehicle on which it is loaded", and "Direction of insertion hole". The "package ID" is a unique code assigned to package L. The "Date and Time" represents the date and time of the state indicated by the package status information J, and is identical to the "Date and Time" in the corresponding information I that corresponds to the package status information J. "Location" represents the location of the luggage L (the storage area 501 where it is stored, or the code of the storage area 501 while it is moving with the vehicle 10), and is the same as "Location Information Ia" of the corresponding information I that corresponds to the luggage status information J. "Height" represents the height of the luggage L and is the same as "Height Information Id" in the corresponding information I that corresponds to the luggage status information J. Therefore, "Height" is one of "H1" to "H3". "Height" represents the height of the storage area of ​​the shelf 50 when the luggage L is stored, and the height of the fork 10a when it is placed on a vehicle. "Vehicle on which the cargo L is loaded" represents the vehicle ID of vehicle 10 when the cargo L is loaded onto vehicle 10. "Vehicle on which the cargo L is loaded" is the same as the "vehicle ID" in the corresponding information I that corresponds to the cargo status information J. The "insertion hole orientation" represents the orientation of the insertion hole Pa of the pallet P at that time. In this embodiment, the cargo L is stored in the storage area 501 with the insertion hole Pa of the pallet P parallel to either the X direction or the Y direction. The "insertion hole orientation" is set to "0" if the insertion hole Pa is parallel to the X direction, and to "1" if it is parallel to the Y direction. Therefore, during the period when the cargo L is loaded onto the vehicle 10, the "insertion hole orientation" is set to "0" if the orientation information Ie in the corresponding information I for the cargo status information J is "0" or "2", and to "1" if the orientation information Ie is "1" or "3". Furthermore, when the cargo L is not loaded onto the vehicle 10 (stored in the storage area 501), the "insertion hole orientation" is determined based on the orientation information I in the corresponding information I when it was last unloaded from the vehicle 10. Note that a symbol corresponding to an intermediate orientation between the X direction and the Y direction may also be used.

[0044] The luggage status information J is generated each time the status of luggage L, identified from the corresponding information I, changes (in the example shown in Figure 8, each time at least one of the following changes: position, height, vehicle on which it is loaded, and orientation of the insertion hole), and is registered and stored in the luggage management data 232. Therefore, among the multiple luggage status information J corresponding to a certain luggage L in the luggage management data 232 at a given time, the luggage status information J with the most recent "date and time" represents the status of the luggage L at that given time. Furthermore, the multiple luggage status information J registered in the luggage management data 232 represent the time-series changes in the status of luggage L. The time-series changes in the status of luggage L include the change in the position of luggage L over time, i.e., information on the movement path of luggage L. For this reason, the luggage management data 232 corresponds to "movement path information" that represents the history of the movement path of luggage L. As shown in Figure 8, the package status information J1 to J7 in the package management data 232 represents the changes in the state of package L001 from arrival to shipment, and the package status information J8 to J11 represents a part of the changes in the state of package L004 from arrival to shipment. The package management data 232 registers and stores the package status information J for all packages under management.

[0045] Next, we will explain how the CPU 21 of the server 20 determines the status of the package L based on multiple pieces of corresponding information I. In the following, among multiple correspondence information I received at a given point in time, the one with the most recent "date and time" will be referred to as "current correspondence information I," and the correspondence information I received immediately before current correspondence information I will be referred to as "previous correspondence information I." Current correspondence information I and previous correspondence information I correspond to "at least two correspondence information pieces that were generated at different times."

[0046] The CPU 21 of server 20 determines the state of the package L based on whether or not there has been a change between the placement information Ib included in the current information I and the previous information I, respectively, when the current information I and the previous information I both contain common position information Ia and common height information Id at a given point in time. More specifically, the CPU 21 of server 20 determines that the luggage L, which was located at the position represented by the common position information Ia and the height represented by the common height information Id, was placed on vehicle 10 when the current correspondence information I and previous correspondence information I both contain common position information Ia and common height information Id at a given point in time. If the placement information Ib included in previous correspondence information I is "0" (i.e., luggage L is not placed on vehicle 10) and the placement information Ib included in current correspondence information I is "1" (i.e., luggage L is placed on vehicle 10) (in other words, when it is determined that the placement information Ib of both current correspondence information I and previous correspondence information I has changed from "0" to "1"), then it determines that luggage L, which was located at the position represented by the common position information Ia and the height represented by the common height information Id, was placed on vehicle 10. By referring to the common height information Id in this way, it is possible to determine which of the multiple storage areas 51 to 53 with different heights the luggage L was stored in before being placed on vehicle 10.

[0047] Furthermore, the CPU 21 of server 20 determines that the luggage L was unloaded from vehicle 10 at the position represented by the common position information Ia and the height represented by the common height information Id, if the placement information Ib included in the previous correspondence information I is "1" and the placement information Ib included in the current correspondence information I is "0" (in other words, if it is determined that the placement information Ib of both the current correspondence information I and the previous correspondence information I have changed from "1" to "0"). By referring to the common height information Id in this way, it is possible to determine which of the multiple storage areas 51 to 53 with different heights into which the luggage L was unloaded. At this time, by also referring to the orientation information Ie of vehicle 10 included in the current correspondence information I, it is possible to determine whether it is facing the orientation of the shelf, and thus it is possible to determine more accurately which luggage L was unloaded from the shelf.

[0048] Furthermore, the CPU 21 of the server 20 determines that the position information Ia and / or height information Id of the current correspondence information I and the previous correspondence information I are different from each other at a certain point in time, and that the placement information Ib is "1" in both cases, that the position and / or height of the luggage L has changed while it is placed on the vehicle 10, and identifies the position represented by the position information Ia of the current correspondence information I and the height represented by the height information Id as the position and height of the luggage L.

[0049] The following explanation will use specific examples. At a certain point in time (referred to as "the first point in time"), if the correspondence information I2 shown in Figure 7 has already been received, then correspondence information I2 becomes the current correspondence information I, and correspondence information I1 becomes the previous correspondence information I. In these correspondence information I1 and I2, the location information Ia is common to "inbound area", the height information Id is common to "H1", and the placement information Ib changes from "0" to "1". Also, the object information Ic of correspondence information I2 is "L001". Therefore, the CPU 21 determines that luggage L001 has been placed on the vehicle 10 in the inbound area. The first point in time corresponds to the time immediately after the situation shown in Figure 5 when luggage L001 was placed on the vehicle 10 in the inbound area 502. The luggage status information J1 of the luggage management data 232 shown in Figure 8 is generated based on the correspondence information I2.

[0050] At the second time point following the first time point, if correspondence information I4 shown in Figure 7 has already been received, then correspondence information I4 becomes the current correspondence information I, and correspondence information I3 becomes the previous correspondence information I. Figure 9 is a schematic diagram showing warehouse 500 at the second point in time. In correspondence information I3 and I4, the location information Ia is common to "A6", the height information Id is common to "H2", and the placement information Ib changes from "1" to "0". Also, the object information Ic in correspondence information I3 is "L001". Therefore, the CPU 21 determines that luggage L001 has been unloaded from the vehicle 10 into the middle storage area 52 at height H2 in the storage area 501 at location A6. The luggage status information J2 in the luggage management data 232 shown in Figure 8 is generated based on correspondence information I3, and the luggage status information J3 is generated based on correspondence information I4.

[0051] At the third time point following the second time point, if the correspondence information I6 shown in Figure 7 has been received, then correspondence information I5 becomes the previous correspondence information I, and correspondence information I6 becomes the current correspondence information I. In correspondence information I5 and I6, the location information Ia is common to "C1", the height information Id is common to "H1", and the placement information Ib changes from "0" to "1". Therefore, the CPU 21 determines that the package L, which was in the lower storage area 51 at height H1 in the storage area 501 at location C1, has been placed on the vehicle 10. At this time, the vehicle 10 cannot identify the package ID of this package L, so the object information Ic in correspondence information I6 is "N / A". However, the CPU 21 of the server 20 can determine, by referring to the package management data 232, that the package L004, whose current location is "C1" and current height is "H1" at that time, has been placed on the vehicle 10. The luggage status information J8 in the luggage management data 232 shown in Figure 8 was generated based on the correspondence information I5, and the luggage status information J9 was generated based on the correspondence information I6.

[0052] If, at the fourth time point following the third time point, correspondence information I8 shown in Figure 7 has already been received, then correspondence information I7 becomes the previous correspondence information I, and correspondence information I8 becomes the current correspondence information I. Figure 10 is a schematic diagram showing warehouse 500 at the fourth point in time. In correspondence information I7 and I8, the location information Ia is common to "D2", the height information Id is common to "H3", and the placement information Ib has changed from "1" to "0". Therefore, the CPU 21 determines that luggage L (in this case, luggage L004 identified at the third time point) has been unloaded from the vehicle 10 into the upper storage area 53 at height H3 in the storage area 501 at location D2. The luggage status information J10 in the luggage management data 232 shown in Figure 8 was generated based on correspondence information I7, and the luggage status information J11 was generated based on correspondence information I8.

[0053] At the fifth time point following the fourth time point, if the correspondence information I10 shown in Figure 7 has been received, then correspondence information I9 becomes the previous correspondence information I, and correspondence information I10 becomes the current correspondence information I. At the fifth time point, it is assumed that vehicle 10 has loaded the luggage L001, which was at position A6, and is moving towards the departure area 503. Figure 11 is a schematic diagram showing warehouse 500 at the fifth point in time. In correspondence information I9 and I10, the position information Ia is different from each other, but the placement information Ib is common to both as "1". Therefore, the CPU 21 determines that luggage L001 is moving while placed on the vehicle 10, and identifies the position of the vehicle 10 as the position of luggage L001. In luggage management data 232 shown in Figure 8, luggage status information J5 is generated based on correspondence information I9, and luggage status information J6 is generated based on correspondence information I10. In luggage status information J6, reflecting the contents of correspondence information I10, the current position of luggage L001 is "A4", the current height is "H1", and the vehicle on which it is placed is "FL03". Note that luggage status information J4 is generated based on correspondence information I (not shown) when luggage L001, which was stored at position A6, was lifted by the vehicle 10.

[0054] At the sixth time point, following the fifth time point, if the correspondence information I12 shown in Figure 7 has already been received, then correspondence information I11 becomes the previous correspondence information I, and correspondence information I12 becomes the current correspondence information I. At the sixth time point, it is assumed that the package L001, which was in transit in Figure 11, has been transported to the outbound area 503 and shipped. Figure 12 is a schematic diagram showing warehouse 500 at the sixth point in time. In correspondence information I11 and I12, location information Ia is common to "shipping area", height information Id is common to "H1", and placement information Ib has changed from "1" to "0". Therefore, the CPU 21 determines that package L001 has been unloaded in the shipping area and shipped. Package status information J7 in package management data 232 shown in Figure 8 is generated based on correspondence information I11. In Figure 12, the movement path of the package L from receiving to shipping is shown by a dashed line, derived from the package status information J1 to J7 (and the package status information J generated between package status information J1 and J2, between package status information J3 and J4, and between package status information J6 and J7, which are not shown). The CPU 21 of the server 20 may display the movement path of the package L in the warehouse 500 as shown in Figure 12 on the operation display unit 14 or the like. The display of the movement path can be displayed on any display unit.

[0055] The CPU 21 of server 20 performs the above operation each time it receives correspondence information I, thereby adding the latest status information J representing the managed package L to the package management data 232. Each package L can be managed using this package status information J. In the above example, a single vehicle 10 is carrying package L, but even if multiple vehicles 10 are carrying package L simultaneously, the same processing can be performed on the correspondence information I received from each vehicle 10. However, the combination of current correspondence information I and previous correspondence information I is selected from correspondence information I transmitted from the same vehicle 10.

[0056] <Transmission of corresponding information> Next, we will describe the corresponding information transmission process performed by the CPU 11 of the terminal device 100 of the vehicle 10. Figure 13 is a flowchart showing the control procedure for the corresponding information transmission process. The corresponding information transmission process is initiated, for example, when the operation of vehicle 10 begins.

[0057] When the corresponding information transmission process is started, the CPU 11 acquires the image captured by the camera 40 (step S101). If the entry LED marker 31 is visible in the captured image ("YES" in step S102), the CPU 11 identifies the object information Ic (cargo ID) based on the light emission pattern 60 of the entry LED marker 31 (step S103). If the entry LED marker 31 is not visible in the captured image ("NO" in step S102), or if step S103 is completed, the CPU 11 identifies the visible LED marker 30 based on the light emission pattern 60 of the ceiling LED marker 30 and derives the vehicle coordinates and position information Ia in the manner described above (step S104).

[0058] If the luggage detection unit 16 has detected luggage L ("YES" in step S105), the CPU 11 sets the placement information Ib to "1" (step S106), and if the luggage detection unit 16 has not detected luggage L ("NO" in step S105), the CPU 11 sets the placement information Ib to "0" (step S107).

[0059] The CPU 11 generates correspondence information I by associating the obtained position information Ia, placement information Ib, and object information Ic with height information Id based on the detection result of the fork height detection unit 17 and orientation information Ie based on the detection result of the orientation detection unit 18 (step S108), and transmits it to the server 20 (step S109).

[0060] If it is determined that it is time to send the next correspondence information I (YES in step S110) and that no instruction to terminate the operation of the vehicle 10 has been given (NO in step S111), the CPU 11 returns to step S101 to generate and send the next correspondence information I. If it is determined that an instruction to terminate the operation of the vehicle 10 has been given (YES in step S111), the CPU 11 terminates the correspondence information transmission process.

[0061] <Vehicle Operation Flow> Next, we will explain the flow of operations performed by the passengers in vehicle 10. Figure 14 is a flowchart showing the vehicle operation flow. The vehicle operation flow starts when the operation of vehicle 10 begins.

[0062] In the vehicle operation flow, if the passenger of vehicle 10 (the terminal device 100 operated by the passenger) is notified of an instruction to move luggage L ("YES" in step S201), and the notification is a notification of luggage L being stored ("YES" in step S202), the passenger moves vehicle 10 to the storage area 502 (step S203) and lifts luggage L with the fork 10a (step S204). On the other hand, if the notification is not a notification of luggage L being stored ("NO" in step S202), the passenger moves vehicle 10 to the designated storage area 501 (step S205) and lifts luggage L from the storage area at the designated height (step S206).

[0063] After lifting the cargo L with the forks 10a, if the designated destination for cargo L is the outbound area 503 ("YES" in step S207), the driver moves the vehicle 10 to the outbound area 503 (step S208), and unloads the cargo L from the forks 10a to perform the outbound operation (step S209). On the other hand, if the designated destination for cargo L is not the outbound area 503 ("NO" in step S207), the driver moves the vehicle 10 to the designated destination storage area 501 for cargo L (step S210), and unloads the cargo L from the forks 10a into the storage area at the designated height (step S211).

[0064] After unloading the load L from the fork 10a, if the operation of the vehicle 10 is not to be terminated ("NO" in step S212), the process returns to step S201; if the operation of the vehicle 10 is to be terminated ("YES" in step S212), the vehicle operation flow is terminated.

[0065] <Luggage Management Processing> Next, we will describe the package management process performed by the CPU 21 of server 20. Figure 15 is a flowchart showing the control procedure for the baggage management process. The baggage management process is initiated, for example, when the operation of vehicle 10 begins.

[0066] When the package management process begins, the CPU 21, upon receiving correspondence information I ("YES" in step S301), determines whether the location information Ia and height information Id of the two most recent consecutive correspondence information I (previous correspondence information I and current correspondence information I) are the same (step S302). If it is determined that the location information Ia and height information Id are the same ("YES" in step S302), the CPU 21 determines whether the placement information Ib has changed from "0" to "1" (step S303). If it is determined that the placement information Ib has changed from "0" to "1" ("YES" in step S303), and the correspondence information I includes object information Ic ("YES" in step S304), the CPU 21 identifies the package L using the object information Ic (step S305). On the other hand, if the correspondence information I does not include object information Ic ("NO" in step S304), the CPU 21 identifies luggage L in the luggage management data 232 whose position and height match those of the vehicle 10 in the correspondence information I (step S306). Once luggage L is identified in step S305 or S306, the CPU 21 determines that the identified luggage L has been placed on the vehicle 10 of the correspondence information I (step S307), and generates luggage status information J including the position, height, vehicle on which luggage L is placed, and orientation of the insertion hole, and registers it in the luggage management data 232 (step S310). Specifically, the CPU generates and registers luggage status information J in which "position" is the position information Ia of the current correspondence information I, "height" is the height information Id of the current correspondence information I, "vehicle on which luggage is placed" is the vehicle 10 of the vehicle ID of the current correspondence information I, and "orientation of the insertion hole" is the orientation corresponding to the orientation information Ie of the current correspondence information I.

[0067] If, in the two correspondence information I described above, the loading information Ib has not changed from "0" to "1" ("NO" in step S303), and the loading information Ib has changed from "1" to "0" ("YES" in step S308), the CPU 21 determines that the luggage L whose location corresponds to the vehicle 10 in correspondence information I in the luggage management data 232 has been unloaded from the vehicle 10 at the height of the height information Id in correspondence information I (step S309). The luggage L whose location corresponds to the vehicle 10 in correspondence information I can be identified by referring to the "location" of the latest luggage status information J for each luggage L. The CPU 21 then generates luggage status information J including the location, height, loading vehicle, and insertion hole orientation of the luggage L and registers it in the luggage management data 232 (step S310). Here, luggage status information J is generated and registered, where "location" is the location information Ia of the current correspondence information I, "height" is the height information Id of the current correspondence information I, "vehicle on which luggage is placed" is "-", and "direction of insertion hole" is the direction corresponding to the direction information Ie of the current correspondence information I. The luggage status information J generated in the process of S310 makes it possible to determine where on the shelf 50 luggage L is placed.

[0068] In step S302, if it is determined that at least one of the position information Ia and the height information Id is different ("NO" in step S302), and the placement information Ib in both corresponding information I is "1" ("YES" in step S311), the CPU 21 determines that the position and / or height of the luggage L has changed while it is placed on the vehicle 10, and generates luggage status information J that reflects the position information Ia and height information Id of the vehicle 10, etc., and registers it in the luggage management data 232 (step S312). Specifically, the CPU 21 first refers to the luggage management data 232 to identify the luggage L that is currently placed on the vehicle 10. This identification of luggage can be done by referring to the "vehicle on which it is placed" in the latest luggage status information J for each luggage L. The CPU 21 then generates and registers luggage status information J in which the "position" is the position information Ia of the current correspondence information I, the height information Id of the current correspondence information I, the "vehicle on which it is placed" is vehicle 10 with the vehicle ID of the current correspondence information I, and the "direction of the insertion hole" is the direction corresponding to the direction information Ie of the current correspondence information I.

[0069] If step S310 or S312 is completed, or if it is determined in step S308 that the loading information Ib has not changed from "1" to "0" ("NO" in step S308), the CPU 21 determines whether or not an instruction to terminate the operation of the server 20 has been given. If the CPU 21 determines that no instruction to terminate the operation has been given ("NO" in step S313), it returns to step S301 to generate and register the luggage status information J based on the next correspondence information I. By repeatedly executing the processes in steps S301 to S313 each time correspondence information I is received, luggage management data 232 containing information on changes in the state of luggage L (including the movement path along the time series) is generated. If it is determined that an instruction to terminate the operation of vehicle 10 has been given ("YES" in step S313), the CPU 21 terminates the luggage management process.

[0070] <Variation 1-1> In the first embodiment, an example was given in which multiple packages L can be stored at different heights in each storage area 501. However, the embodiment is not limited to this, and an example is also given in which only one package L can be stored at a certain height in each storage area 501. In this case, the height information Id is omitted in the correspondence information I of the status data 133, and the current height is omitted in the package management data 232.

[0071] <Variation 1-2> In the first embodiment, the luggage management processing (identification of the position and height of luggage L, and updating of luggage management data 232, etc.) that was performed by the CPU 21 of the server 20 may be performed by the CPU 11 of the terminal device 100 of the vehicle 10. In this case, the luggage management data 232 may be stored in the storage unit 13 of the terminal device 100. Also, if the luggage management system 1 has multiple vehicles 10, the CPU 11 of the terminal device 100 of any one of the vehicles 10 may perform the luggage management processing. In the modified example 1-2, the server 20 may be omitted. Modified example 1-2 may be combined with the modified example 1-1 described above.

[0072] <Variation 1-3> In the first embodiment, an example was given in which a vehicle 10 located in a certain storage area 501 unloads cargo L onto a shelf 50 provided within the storage area 501, or lifts cargo L from the shelf 50 and places it on a fork 10a. However, the position of the vehicle 10 and the position of the cargo L being loaded and unloaded by the fork 10a may be different.

[0073] Figure 16 shows the case where the position of the vehicle 10 and the position of the cargo L being loaded and unloaded are different. The warehouse 500, partially shown in Figure 16, is configured such that the width of each storage area 501 is smaller than the total length of the vehicle 10, and when the main body of the vehicle 10 is located in a certain storage area 501, the forks 10a of the vehicle 10 extend into the adjacent storage area 501. In the example shown in Figure 16, the main body of the vehicle 10 is in storage area 501 at position A2, and the forks 10a extend into the adjacent storage area 501 at position B2. The position of the main body of the vehicle 10 is assumed to be the position information Ia of the vehicle 10.

[0074] Even in such cases, the state of the cargo L can be appropriately managed by using the orientation information Ie of the vehicle 10. For example, in the state shown in Figure 16, the position information Ia of the vehicle 10 is "A2" and the orientation information Ie of the vehicle 10 is "1" (+Y direction). By referring to the orientation information Ie in addition to the position information Ia, it can be determined that the fork 10a is at position B2. Therefore, if the loading information Ib changes from "1" to "0" or from "0" to "1" in this state, it can be determined that loading or unloading of cargo L has occurred at position B2. Furthermore, as shown in Figure 16, when vehicle 10, which is at position A2, loads or unloads cargo L at this position, the position of the fork 10a can be any of positions A1, B2, or A3, depending on the orientation of vehicle 10. For this reason, if only the position information Ia of vehicle 10 is used, it is not possible to accurately determine the location where cargo L was loaded or unloaded. However, by referring to the orientation information Ie in addition to the position information Ia, it is possible to accurately determine the location where cargo L was loaded or unloaded. For example, in the state shown in Figure 16, if the orientation information Ie is "0" (vehicle 10 is facing the -X direction), it can be determined that cargo L was loaded and unloaded at storage area 501 at position A1; if the orientation information Ie is "1" (vehicle 10 is facing the +Y direction), it can be determined that cargo L was loaded and unloaded at storage area 501 at position B2; and if the orientation information Ie is "3" (vehicle 10 is facing the +X direction), it can be determined that cargo L was loaded and unloaded at storage area 501 at position A3. Note that Figure 16 is just one example of a situation in which using orientation information Ie is effective, and of course, orientation information Ie can be used in other situations as well. For example, if there are multiple shelves 50 within a single storage area 501, and the orientation of the vehicle 10 for loading and unloading goods differs for each shelf 50, then the orientation information Ie, in addition to the position information Ia, can be used to determine which of the multiple shelves 50 the loading and unloading of goods took place on.

[0075] Furthermore, by adding a data item for "direction of insertion hole" to the cargo status information J of the cargo management data 232, it is possible to more accurately identify the cargo to be loaded or unloaded. For example, as shown in Figure 16, if the pallet P of cargo L stored at position C2 is oriented so that the insertion hole Pa is parallel to the X direction, it can be determined that only the vehicle 10 at position C1 or position C3 can load or unload the cargo, and the vehicle 10 at position B2 cannot load or unload it. This is because even if the fork 10a is inserted from position B2 in the +Y direction, it cannot be inserted into the insertion hole Pa because its orientation does not match. Therefore, for example, the "direction of insertion hole" information in the cargo status information J can be used to guide the vehicle 10. To give one example, if it is desired to place cargo L on the vehicle 10 in the state shown in Figure 16, the vehicle 10 should be guided to lift cargo L from the position C1 side or position C3 side, taking into account the orientation of the insertion hole Pa. Furthermore, the "direction of the insertion hole" information can also be used to assist in determining the status of the cargo. For example, if there are vehicles 10 at positions C1 and B2, and cargo L is loaded or unloaded at position C2, it can be determined that the vehicle 10 at position C1 was the one that performed the loading or unloading. In this way, by combining the "direction of the insertion hole" information of the cargo status information J with the position information Ia and orientation information Ie of the corresponding information I, the status of the cargo can be determined more accurately.

[0076] (Second embodiment) Next, a second embodiment will be described. In the following, components common to the first embodiment will be denoted by the same reference numerals and their descriptions will be omitted, while differences from the first embodiment will be described.

[0077] <Configuration of the luggage management system> Figure 17 shows the configuration of the luggage management system 1 according to the second embodiment. In the second embodiment, the vehicle 10 has an LED marker 30, and multiple (two in Figure 17) cameras 40 are installed on the ceiling of the warehouse 500. Based on the images of the vehicle 10 taken by the two cameras 40, the three-dimensional position of the LED marker 30 (i.e., the three-dimensional position of the vehicle 10) within the warehouse 500 can be determined by triangulation. In the second embodiment, the cameras 40 correspond to "reference objects".

[0078] Figure 18 is a block diagram showing the functional configuration of the vehicle 10 in the second embodiment. The differences between the vehicle 10 of the second embodiment and the vehicle 10 of the first embodiment (Figure 2) are that the captured image data 132 and status data 133 are not stored in the storage unit 13 of the terminal device 100, and that the camera 40 is replaced with an LED marker 30. The LED marker 30 lights up based on a control signal transmitted from the CPU 11 of the terminal device 100.

[0079] Figure 19 is a block diagram showing the functional configuration of the server 20 in the second embodiment. The differences between the server 20 of the second embodiment and the server 20 of the first embodiment (Figure 3) are that the storage unit 23 stores captured image data 233 and status data 234, and that it communicates with the camera 40 and receives captured image data 233 from the camera 40. In the second embodiment, the server 20 corresponds to an "information processing device," and the CPU 21 of the server 20 corresponds to "one or more processing units."

[0080] <How the baggage management system works> Next, the operation of the luggage management system 1 in the second embodiment will be described. In the second embodiment, the LED marker 30 provided on the vehicle 10 emits light in a light emission pattern 60 that includes various information related to the state of the vehicle 10.

[0081] Figure 20 illustrates the light emission pattern 60 of the LED marker 30 in the second embodiment. As shown in Figure 20, the LED marker 30 emits light in a light emission pattern 60 that includes vehicle ID, mounting information Ib, height information Id, and orientation information Ie. In the example shown in Figure 20, the vehicle ID is represented by the combination of light emission colors in unit periods S1 and S2, the orientation information Ie is represented by the combination of light emission colors in unit periods S3 and S4, the mounting information Ib is represented by the light emission color in unit period S5, and the height information Id is represented by the combination of light emission colors in unit periods S6 and S7. However, this is an example and can be changed as appropriate. For example, a table data may be prepared in advance that associates the codes identified from the light emission pattern 60 with various states of the vehicle 10, and the state of the vehicle 10 may be identified by referring to the codes identified from the light emission pattern 60 in the table data. The light emission pattern 60 of the LED marker 30 is determined by the CPU 11 of the terminal device 100 according to the state of the vehicle 10.

[0082] The LED marker 30, which emits light in this light emission pattern 60, is captured by the camera 40, and the captured image data 233 is transmitted to the server 20. The CPU 21 of the server 20 analyzes the light emission pattern 60 of the LED marker 30 visible in the captured image to identify the vehicle 10 (vehicle ID), and derives the coordinates and position information Ia of the vehicle 10 based on the identification result and the position of the LED marker 30 in the captured image. In addition to the vehicle ID, the CPU 21 also identifies mounting information Ib, height information Id, and orientation information Ie from the analyzed light emission pattern 60. Then, it generates correspondence information I that associates the derived or identified vehicle ID, coordinates, position information Ia, mounting information Ib, height information Id, and orientation information Ie. The CPU 21 of the server 20 repeatedly performs the acquisition of position information Ia, mounting information Ib, height information Id, and orientation information Ie, and the generation of correspondence information I, at a predetermined frequency, and registers each generated correspondence information I in the status data 234. As a result, status data 234 with the same content as the status data 133 of the first embodiment is generated. Furthermore, the CPU 21 of the server 20 can identify the luggage ID of luggage L when it is placed on the vehicle 10 by comparing the vehicle 10's position information Ia, height information Id, and orientation information Ie with the latest position and height of each luggage L in the luggage management data 232. Therefore, in the status data 234, if the placement information Ib is "1", object information Ic representing the luggage ID of the placed luggage L is entered.

[0083] Furthermore, each time the server 20 generates correspondence information I, the CPU 21 generates luggage status information J based on the generated correspondence information I, similar to the first embodiment, and registers it in the luggage management data 232.

[0084] <Transmission of corresponding information> Next, in the second embodiment, the LED marker light emission process performed by the CPU 11 of the terminal device 100 of the vehicle 10 will be described. Figure 21 is a flowchart showing the control procedure for the LED marker light emission process in the second embodiment. The LED marker illumination process is initiated, for example, when the vehicle 10 starts operating.

[0085] When the LED marker illumination process is started, the CPU 11 identifies the orientation of the vehicle 10 based on the detection result by the orientation detection unit 18 and generates orientation information Ie (step S401), identifies whether or not luggage L is placed based on the detection result by the luggage detection unit 16 and generates placement information Ib (step S402), and identifies the height of the fork 10a based on the detection result by the fork height detection unit 17 and generates height information Id (step S403).

[0086] The CPU 11 generates a light emission pattern 60 that includes the vehicle ID obtained in advance and the mounting information Ib, height information Id, and orientation information Ie identified (generated) in steps S401 to S403 (step S404), and sends a control signal to the LED marker 30 to illuminate the LED marker 30 with the generated light emission pattern 60 (step S405).

[0087] If the CPU 11 determines that no instruction has been given to terminate the operation of the vehicle 10 ("NO" in step S406), it returns to step S401. If the CPU 11 determines that an instruction has been given to terminate the operation of the vehicle 10 ("YES" in step S406), the CPU 11 terminates the LED marker illumination process.

[0088] <Correspondence information generation process> Next, the correspondence information generation process performed by the CPU 21 of the server 20 in the second embodiment will be described. Figure 22 is a flowchart showing the control procedure for the correspondence information generation process in the second embodiment. The corresponding information generation process is initiated, for example, when the operation of vehicle 10 begins.

[0089] When the corresponding information generation process is started, the CPU 21 acquires the image captured by the camera 40 (step S501). Based on the light emission pattern 60 of the LED marker 30 shown in the captured image, the CPU 21 acquires the vehicle ID, mounting information Ib, height information Id, and orientation information Ie of the vehicle 10 (step S502). The CPU 21 also derives the vehicle coordinates and position information Ia in the manner described above based on the light emission pattern 60 of the LED marker 30 (step S503).

[0090] The CPU 21 generates correspondence information I by associating the obtained position information Ia, placement information Ib, height information Id, and orientation information Ie, and registers it in the status data 234 (step S504).

[0091] If the CPU 21 determines that it is time to generate the next correspondence information I (YES in step S505) and that no instruction to terminate the operation of the vehicle 10 has been given (NO in step S506), it returns to step S501 to generate the next correspondence information I. If it determines that an instruction to terminate the operation of the vehicle 10 has been given (YES in step S506), the CPU 21 terminates the correspondence information generation process.

[0092] Note that the vehicle operation flow and cargo management process in the second embodiment are the same as those in the first embodiment, so their explanation will be omitted.

[0093] <Variation 2-1> Some of the information included in the light emission pattern 60 of the LED marker 30 may be transmitted to the server 20 via data communication (for example, wireless communication via Wi-Fi) between the terminal device 100 of the vehicle 10 and the server 20.

[0094] Furthermore, variations 1-1 and 1-3 of the first embodiment can also be applied to the second embodiment.

[0095] (effect) As described above, the CPU 11 of the terminal device 100 according to the first embodiment and the CPU 21 of the server 20 according to the second embodiment acquire location information Ia of the vehicle 10, orientation information Ie of the vehicle 10, and placement information Ib indicating whether or not luggage L is placed on the vehicle 10, and generate correspondence information I which associates the location information Ia, orientation information Ie, and placement information Ib with each other in chronological order. With such correspondence information I, the relationship between the vehicle 10 and luggage L at the location represented by the acquired location information Ia of the vehicle 10 can be more appropriately understood. That is, if the placement information Ib of correspondence information I is "1", it can be determined that luggage L moves with the vehicle 10 and that location information Ia represents the positions of the vehicle 10 and luggage L. Also, if the placement information Ib of correspondence information I is "0", it can be determined that luggage L is not placed on the vehicle 10 and that the position of luggage L does not change even if the vehicle 10 moves. Furthermore, by referring to the orientation information Ie, the vehicle 10 and luggage L can be associated more appropriately.

[0096] Furthermore, the CPU 11 according to Modification 1-2 of the first embodiment and the CPU 21 according to the second embodiment repeatedly perform the acquisition of position information Ia, orientation information Ie, and placement information I, and the generation of corresponding information I, and determine the state of the package L based on the multiple corresponding information I generated. According to this, by determining the state of the package L based on each of the multiple corresponding information I in order according to the time series of the corresponding information I, it is possible to identify changes in the state of the package L and the latest state.

[0097] Furthermore, the CPU 11 according to Modification 1-2 of the first embodiment and the CPU 21 according to the second embodiment determine the state of the package L based on at least two correspondence pieces I from among the multiple correspondence pieces I that were generated at different times. This makes it possible to determine the state of the package L with a simple process of comparing the correspondence pieces I.

[0098] Furthermore, the CPU 11 according to Modification 1-2 of the first embodiment and the CPU 21 according to the second embodiment determine the state of the luggage L based on the position information Ia, placement information Ib, and orientation information Ie included in the corresponding information I. This makes it possible to appropriately determine the state of the luggage L based on the position and orientation of the vehicle 10 on which the luggage L is placed.

[0099] Furthermore, the CPU 11 according to Modification 1-2 of the first embodiment and the CPU 21 according to the second embodiment generate luggage management data 232 as luggage L movement path information based on a plurality of corresponding pieces of information I. This makes it possible to retrospectively identify the luggage L movement path in chronological order.

[0100] In the first embodiment, the position information Ia is generated based on the positional relationship between the LED marker 30 (reference object) whose position is fixed and the vehicle 10. In the second embodiment, the position information Ia is generated based on the positional relationship between the fixed-position camera 40 (reference object) and the vehicle 10. In this way, by using a method to determine the relative position of the vehicle 10 from a reference object, the position of the vehicle 10 can be determined more easily and with greater accuracy compared to a method that directly determines the absolute position of the vehicle 10.

[0101] In the second embodiment, the vehicle 10 has an LED marker 30 that emits light in a light emission pattern 60 that can identify the vehicle 10, the reference object is a camera 40 that images the LED marker 30, and the position information Ia is generated based on the identification result of the vehicle 10 based on the light emission pattern 60 of the LED marker 30 in the image captured by the camera 40 and the position of the LED marker 30 in the captured image. This makes it possible to identify the position of the vehicle 10 without transmitting or receiving radio waves. Furthermore, the position of the vehicle 10 can be identified with high accuracy at the pixel-level resolution of the captured image. In addition, if multiple vehicles 10 (LED markers 30) are captured in the image by the camera 40, the positions of these multiple vehicles 10 can be identified based on the captured image.

[0102] Furthermore, the information processing methods according to the first and second embodiments are information processing methods executed by the CPU 11 or CPU 21 of the information processing device as a computer, which acquire location information Ia of the vehicle 10, orientation information Ie of the vehicle 10, and placement information Ib indicating whether or not luggage L is placed on the vehicle 10, and generate correspondence information I which associates the location information Ia, orientation information Ie, and placement information Ib with each other in chronological order. With such correspondence information I, the relationship between the vehicle 10 and luggage L at the location represented by the acquired location information Ia of the vehicle 10 can be grasped more appropriately. In addition, by referring to the orientation information Ie, the vehicle 10 and luggage L can be associated more appropriately.

[0103] Furthermore, the program 131 according to the first embodiment causes the CPU 11 of the terminal device 100 to execute the following processes: acquiring location information Ia of the vehicle 10, acquiring orientation information Ie of the vehicle 10, acquiring placement information Ib indicating whether or not luggage L is placed on the vehicle 10, and generating correspondence information I which associates the location information Ia, orientation information Ie, and placement information Ib with each other in chronological order. Furthermore, the program 231 according to the second embodiment causes the CPU 21 of the server 20 to execute each of the above processes. With such correspondence information I, the relationship between the vehicle 10 and luggage L at the location represented by the acquired location information Ia of the vehicle 10 can be understood more appropriately. In addition, by referring to the orientation information Ie, the vehicle 10 and luggage L can be associated more appropriately.

[0104] (others) The above-described embodiments are merely examples of the information processing apparatus, information processing method, and program according to the present invention, and are not limited thereto. For example, in the above embodiment, a forklift was used as an example of a mobile body, but it is not limited to this. The mobile body can be anything that can move with an object placed on it, and may be a vehicle other than a forklift (for example, a trolley), or it may be a robot or a person.

[0105] Furthermore, the object is not limited to the goods L stored in warehouse 500, but may be any object whose location is controlled.

[0106] Furthermore, while the location information Ia is exemplified as the area to which the coordinates of the vehicle 10 belong (storage area 501, etc.), it is not limited to this. Location information Ia can be any information relating to the position of the moving object, and may be, for example, coordinates that directly represent the position of the moving object.

[0107] Furthermore, while an example using a combination of red, green, and blue light was given as an example of optical communication using the LED marker 30, the method is not limited to this. For example, you may change the colors used, or you may use two or fewer colors, or four or more colors. Alternatively, Li-Fi (Light Fidelity), which transmits information using rapidly modulated light, may be used. Furthermore, the light used is not limited to visible light; infrared or ultraviolet light may also be used.

[0108] Furthermore, the method for acquiring the position information Ia of a moving object is not limited to optical communication using LED markers 30, etc. For example, various known methods can be used, such as a method of calculating the current position by receiving and decoding radio waves transmitted from a Global Navigation Satellite System (GNSS) positioning satellite, or a method of determining the positional relationship with a beacon based on signals from a beacon installed at a predetermined location.

[0109] Furthermore, the method for identifying package L in the receiving area 502 is not limited to the light emission pattern 60 of the receiving LED marker 31, but various known methods can be used. In addition, the CPU 21 of the server 20 may acquire information (package ID) that identifies the incoming package L and reflect it in the package management data 232.

[0110] Furthermore, while the above description discloses examples in which the HDD and SSD of the storage units 13 and 23 are used as computer-readable media for the program according to the present invention, the invention is not limited to these examples. Other computer-readable media that can be used include information recording media such as flash memory and CD-ROM. In addition, a carrier wave can also be used as a medium for providing the data of the program according to the present invention via a communication line.

[0111] Furthermore, it goes without saying that the detailed configuration and operation of each component, such as the luggage management system 1, vehicle 10, server 20, and terminal device 100 in the above embodiment, can be appropriately modified without departing from the spirit of the present invention.

[0112] Although embodiments of the present invention have been described, the scope of the present invention is not limited to the embodiments described above, but includes the scope of the invention as described in the claims and its equivalents. The invention described in the claims initially attached to the application for this patent is listed below. The claim numbers listed below are the same as those in the claims initially attached to the application for this patent. [Note] <Claim 1> Obtain the location information of the moving object, The orientation information of the moving body is acquired, Acquire placement information indicating whether or not an object different from the aforementioned moving body is placed on the moving body. Correspondence information is generated by chronologically associating the aforementioned position information, the orientation information of the moving body, and the previously described position information with each other. An information processing device comprising one or more processing units. <Claim 2> The first or more processing units described above are: The acquisition of the aforementioned position information, the orientation information of the moving body, the previously described position information, and the generation of the corresponding information are repeatedly performed. The state of the object is determined based on the multiple corresponding pieces of information that have been generated. The information processing apparatus according to claim 1. <Claim 3> The first or more processing units described above are: Based on at least two of the generated correspondence pieces, each generated at a different time, the state of the object is determined. The information processing apparatus according to claim 2. <Claim 4> The first or more processing units described above are: Based on the position information, the previously described placement information, and the orientation information of the moving body included in the correspondence information, the state of the object is determined. The information processing apparatus according to claim 1. <Claim 5> The first or more processing units described above are: Based on the multiple pieces of correspondence information, the movement path information of the object is generated. The information processing apparatus according to claim 2. <Claim 6> The position information is generated based on the positional relationship between a fixed reference object and the moving object. The information processing apparatus according to any one of claims 1 to 5. <Claim 7> The moving body has a light-emitting device that emits light in a light emission pattern that can identify the moving body, The aforementioned reference object is an imaging device that images the light-emitting device, The position information is generated based on the identification result of the moving object based on the light emission pattern of the light emission device in the image captured by the imaging device, and the position of the light emission device in the captured image. The information processing apparatus according to claim 6. <Claim 8> An information processing method performed by a computer in an information processing device, Obtain the location information of the moving object, The orientation information of the moving body is acquired, Acquire placement information indicating whether or not an object different from the aforementioned moving body is placed on the moving body. Correspondence information is generated by chronologically associating the aforementioned position information, the orientation information of the moving body, and the previously described position information with each other. Information processing methods. <Claim 9> In the computer of the information processing device, Process to acquire the location information of a moving object. A process for obtaining information on the orientation of the moving object. A process for acquiring placement information indicating whether or not an object different from the aforementioned moving body is placed on the moving body. A process for generating correspondence information by chronologically associating the aforementioned position information, the orientation information of the moving body, and the previously described position information with each other. A program that executes the command. [Explanation of Symbols]

[0113] 1. Luggage Management System (Information Processing System) 10. Vehicles (mobile devices) 10a Fork (mounting section) 10b Mast 11 CPU (1 or more processing units (first embodiment)) 12 RAM 13 Storage section 131 Programs 132 Image data 133 Status Data 14 Operation display section 15 Communications Department 16. Luggage detection unit 17 Fork height detection unit 18 Direction detection unit 19 Communication Path 20. Server (Information Processing Device (Second Embodiment)) 21 CPU (1 or more processing units (second embodiment)) 22 RAM 23 Memory section 231 Programs 232. Package Management Data 233 Image data 234 Status Data 24 Communications Department 25 buses 30 LED Markers (Light-emitting device, reference object (first embodiment)) 31 LED marker for vehicle entry 40. Camera (imaging device, reference object (second embodiment)) 50 shelves 51 Lower storage area 52 Middle storage area 53 Upper storage area 60 Lighting Patterns 100 Terminal device (information processing device (first embodiment)) 500 warehouse 501 Storage Area (Area) 502 Entry Area (Region) 503 Departure Area (Region) I, I1~I12 Correspondence Information J, J1~J11 Baggage Status Information Ia Location information Ib. Posting Information Ic object information ID Height information Ie orientation information L, L001, L004 Luggage (object) P Palette Pa insertion hole S1-S24 Unit Period

Claims

1. Obtain the location information of the moving object, The orientation information of the moving body is acquired, Acquire placement information indicating whether or not an object different from the aforementioned moving body is placed on the moving body. Correspondence information is generated by chronologically associating the aforementioned position information, the orientation information of the moving body, and the previously described position information with each other. Equipped with one or more processing units, The position information is generated based on the positional relationship between a fixed reference object and the moving object. Information processing device.

2. The one or more processing units described above are: The acquisition of the aforementioned position information, the orientation information of the moving body, the previously described position information, and the generation of the corresponding information are repeatedly performed. The state of the object is determined based on the multiple corresponding pieces of information that have been generated. The information processing apparatus according to claim 1.

3. The one or more processing units described above are: Based on at least two of the generated correspondence pieces, each generated at a different time, the state of the object is determined. The information processing apparatus according to claim 2.

4. The one or more processing units described above are: Based on the position information, the previously described placement information, and the orientation information of the moving body included in the correspondence information, the state of the object is determined. The information processing apparatus according to claim 1.

5. The one or more processing units described above are: Based on the multiple pieces of correspondence information, the movement path information of the object is generated. The information processing apparatus according to claim 2.

6. The moving body has a light-emitting device that emits light in a light-emitting pattern that can identify the moving body, The aforementioned reference object is an imaging device that images the light-emitting device, The position information is generated based on the identification result of the moving object based on the light emission pattern of the light emission device in the image captured by the imaging device, and the position of the light emission device in the captured image. The information processing apparatus according to claim 1.

7. An information processing method performed by a computer of an information processing device, Obtain the location information of the moving object, The orientation information of the moving body is acquired, Acquire placement information indicating whether or not an object different from the aforementioned moving body is placed on the moving body. Correspondence information is generated by chronologically associating the aforementioned position information, the orientation information of the moving object, and the previously described position information with each other. The position information is generated based on the positional relationship between a fixed reference object and the moving object. Information processing methods.

8. The computer of the information processing device, Process to acquire the location information of a moving object. A process for obtaining information on the orientation of the moving object. A process for acquiring placement information indicating whether or not an object different from the aforementioned moving body is placed on the moving body. A process for generating correspondence information by chronologically associating the aforementioned position information, the orientation information of the moving body, and the previously described position information with each other. Make it run, The position information is generated based on the positional relationship between a fixed reference object and the moving object. program.

Images