Work inference device and work inference system

Abstract

A work inference device (2) acquires position information indicating the position of a movable body (1), acquires conveyance information indicating the conveyance state related to whether the movable body (1) is conveying or not conveying a cargo (3), infers, on the basis of the temporal change in the position and the conveyance state, work executed by the movable body (1) using a prescribed work determination rule, and outputs a work report including information indicating the inferred work.

Description

Work estimation device and work estimation system

[0001] The present disclosure relates to an activity estimation device and an activity estimation system.

[0002] In logistics and manufacturing sites, there is a general demand to reduce the man-hours of workers as much as possible. The key to improving productivity is how to implement measures to eliminate waste on-site. For this reason, it is important to visualize what workers are doing (i.e., the work they are doing), and for example, technology has been developed for analyzing movement lines that measure the distance people travel.

[0003] For example, Patent Literature 1 discloses an information processing device that processes position information of an object moved by a moving body. This device outputs position information of the object obtained based on position information of the moving body acquired based on an image captured by an imaging unit and holding information indicating whether the object is being held or not held by the moving body.

[0004] Japanese Patent Application Laid-Open No. 2022-130032

[0005] When trying to improve the work performed by a mobile object, sufficient information about the work cannot be obtained by analyzing the movement of the mobile object based on its location information alone. Furthermore, it is undesirable to significantly increase the cost and effort required to obtain information about the work. Therefore, it is necessary to estimate the work performed by a mobile object without significantly increasing the cost and effort.

[0006] An object of the present disclosure is to provide an activity estimation device and an activity estimation system that can estimate an activity performed by a moving object without significantly increasing costs and effort.

[0007] A work estimation device according to one aspect of the present disclosure acquires location information indicating the location of a mobile object, acquires transport information indicating a transport status regarding whether the mobile object is transporting cargo or not, estimates work performed by the mobile object using a predetermined work determination rule based on changes over time in the location and the transport status, and outputs a work report including information indicating the estimated work.

[0008] According to one aspect of the present disclosure, it is possible to estimate the work performed by a mobile object without significantly increasing cost and effort.

[0009] 1 is a block diagram showing the configuration of a work estimation system according to an embodiment. FIG. 2 is a diagram schematically showing a warehouse 30 where work is performed by the mobile object 1 of FIG. 1. FIG. 3 is a sequence diagram showing the operation of the work estimation system of FIG. 1. FIG. 4 is a sequence diagram showing a modified example of the operation of the work estimation system of FIG. 1. FIG. 5 is a flowchart showing the work estimation process executed by the processing device 21 of FIG. 1. FIG. 6 is a flowchart showing a subroutine of the entry and exit detection process (step S12) of FIG. 5. FIG. 7 is a flowchart showing a subroutine of the temporary placement detection process (step S13) of FIG. 5. FIG. 8 is a table showing an example of location information acquired by the processing device 21 of FIG. 1. FIG. 9 is a table showing an example of transport information acquired by the processing device 21 of FIG. 1. FIG. 10 is a table showing a first example of combined information generated in step S11 of FIG. 5. FIG. 11 is a diagram for explaining extraction of stay period information in step S21 of FIG. 6. FIG. 12 is a table showing an example of stay period information extracted in step S21 of FIG. 6. FIG. 13 is a table showing a second example of combined information generated in step S11 of FIG. 5. FIG. 14 is a diagram for explaining detection of completion of entry in steps S24 and S25 of FIG. 7 is a table showing an example of the end time of each task determined by executing the warehousing and retrieval detection process (step S12) of FIG. 5. FIG. 8 is a table showing a third example of combined information generated in step S11 of FIG. 5. FIG. 9 is a diagram for explaining the detection of unloading and loading in steps S31 and S32 of FIG. 7. FIG. 10 is a table showing an example of the unloading and loading times determined by executing the temporary placement detection process (step S13) of FIG. 5. FIG. 11 is a table showing a first example of a work report generated in step S14 of FIG. 6. FIG. 12 is a diagram showing a second example of a work report generated in step S14 of FIG.

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, more detailed descriptions than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art.

[0011] The inventor(s) provide the accompanying drawings and the following description to enable those skilled in the art to fully understand the present disclosure, and do not intend for them to limit the subject matter described in the claims.

[0012] 1 is a block diagram showing the configuration of an activity estimation system according to an embodiment. The activity estimation system includes one or more moving bodies 1 and an activity estimation device 2.

[0013] The mobile object 1 transports a load 3. The mobile object 1 includes a platform 1a that holds the load 3. The mobile object 1 may be a forklift, in which case the platform 1a may be a fork. The mobile object 1 may also be an unmanned device such as an automated guided vehicle (AGV), or may be a hand truck.

[0014] The work estimation device 2 estimates the work performed by the mobile object 1. The work performed by the mobile object 1 includes, for example, the storing and retrieving of cargo 3 from a warehouse, as will be described later.

[0015] The mobile object 1 is provided with a terminal device 10 and cameras 13 and 14 .

[0016] The camera 13 captures images or moving images of the surroundings of the mobile object 1. The camera 14 captures images or moving images of the loading platform 1a. The terminal device 10 includes a processing device 11 and a communication device 12.

[0017] The processing device 11 generates position information indicating the position of the mobile object 1 based on an image of the surroundings of the mobile object 1. The processing device 11 also generates transportation information indicating a transportation status, as to whether the mobile object 1 is transporting or not transporting luggage 3, based on an image of the loading platform 1a. The communication device 12 is connected to the work estimation device 2 via a wireless communication line such as a mobile phone network or a wireless LAN, and transmits the position information and transportation information to the work estimation device 2. Additionally or alternatively, the communication device 12 may transmit images or videos captured by the camera 13 and / or camera 14 directly to the work estimation device 2.

[0018] The terminal device 10 may be, for example, a mobile phone. One or both of the cameras 13 and 14 may be built into the terminal device 10 or may be connected to the outside of the terminal device 10.

[0019] The task estimation device 2 includes a processing device 21 , a communication device 22 , storage devices 23 and 24 , an input device 25 , and an output device 26 .

[0020] The processing device 21 estimates the work performed by the moving object 1 by executing a work estimation process, which will be described later with reference to FIG.

[0021] The communication device 22 is connected to the terminal device 10 via a wireless communication line such as a mobile phone network or a wireless LAN, and receives location information and transportation information from the terminal device 10. Additionally or alternatively, the communication device 22 may receive images or videos captured by the camera 13 and / or the camera 14 from the terminal device 10.

[0022] The storage device 23 stores in advance area information including the geographical locations of passages, facilities, etc. in the area in which the mobile object 1 moves.

[0023] The storage device 24 stores the location information, transportation information, and / or images or video images received from the terminal device 10 .

[0024] The input device 25 includes a keyboard, a pointing device, and the like for obtaining user input.

[0025] The output device 26 outputs a work report including information indicating the estimated work. The output device 26 may include a display device, or may include a communication device connected to a remote computer or the like via a communication line.

[0026] The task estimation device 2 may be a general-purpose computer or a dedicated device.

[0027] [Operation of the Embodiment] FIG. 2 is a diagram schematically illustrating a warehouse 30 in which work is performed by the mobile object 1 of FIG. 1. The warehouse 30 represents an area within which the mobile object 1 can move, such as a room, floor, or building. The warehouse 30 includes multiple shelves 31, loading / unloading areas 32-1 and 32-2, and temporary storage areas 33-1 and 33-2. Transport devices 34-1 and 34-2, such as trucks, elevators, and Omnilifters (registered trademark), are located outside the warehouse 30. The mobile object 1-1 moves a package 3-1 from the transport device 34-1 to the shelf 31 (storage), and the mobile object 1-2 moves a package 3-2 from the shelf 31 to the transport device 34-2 (shipment). The loading / unloading areas 32-1 and 32-2 are spatially connected to the outside of the warehouse 30. When loading or unloading, the package 3 always passes through the loading / unloading areas 32-1 and 32-2. Furthermore, when the moving bodies 1-1 and 1-2 enter and leave the warehouse, they may temporarily unload the cargo 3-1 and 1-3 in the temporary storage areas 33-1 and 33-2.

[0028] The temporary storage areas 33-1 and 33-2 may be provided inside or outside the loading / unloading areas 32-1 and 32-2, respectively.

[0029] The incoming / outgoing areas 32-1 and 32-2 are also collectively referred to as "incoming / outgoing areas 32." The temporary storage areas 33-1 and 33-2 are also collectively referred to as "temporary storage areas 33." The transport devices 34-1 and 34-2 are also collectively referred to as "transport devices 34."

[0030] Separate storage areas and delivery areas may be provided for storage and delivery instead of a common storage / delivery area. Also, separate temporary storage areas that can be used for storage and delivery may be provided instead of a common temporary storage area that can be used for both storage and delivery.

[0031] FIG. 3 is a sequence diagram showing the operation of the task estimation system of FIG.

[0032] In step S1, the processing device 11 of the terminal device 10 uses the camera 13 to capture an image of the surroundings of the moving object 1. In step S2, the processing device 11 identifies the position of the moving object 1 based on the image of the surroundings of the moving object 1 and generates position information. To identify the position of the moving object 1, the processing device 11 may, for example, calculate the relative position of the moving object 1 with respect to a reference position using Visual-SLAM. Additionally or alternatively, the processing device 11 may identify the position of the moving object 1 by referring to a marker, such as an AR marker, having a known position. By combining Visual-SLAM and the marker, it is possible to reduce the cumulative error caused by Visual-SLAM. The terminal device 10 transmits the position information and the image of the surroundings of the moving object 1 to the task estimation device 2.

[0033] In step S3 , the processing device 21 of the task estimation device 2 stores the position information and an image of the surroundings of the moving object 1 in the storage device 24 .

[0034] In step S4, the processing device 11 of the terminal device 10 takes an image of the loading platform 1a. In step S5, the processing device 11 determines whether or not there is luggage 3 based on the image of the loading platform 1a and generates transport information. The processing device 11 may determine whether or not there is luggage 3 using a machine learning model trained to detect luggage 3 on the loading platform 1a. The terminal device 10 transmits the transport information and the image of the loading platform 1a to the work estimation device 2.

[0035] In step S6, the processing device 21 of the task estimation device 2 stores the transport information and the image of the loading platform 1a in the storage device 24.

[0036] The processing device 11 of the terminal device 10 and the processing device 21 of the activity estimation device 2 repeat steps S1 to S6 for several hours to several days.

[0037] In step S7, the processing device 21 executes a task estimation process to estimate the task performed by the moving object 1, and generates and outputs a task report including information indicating the estimated task. The task estimation process will be described later with reference to FIG. 5 .

[0038] Steps S4 and S5 may be performed in parallel or simultaneously with steps S1 and S2.

[0039] FIG. 4 is a sequence diagram showing a modified example of the operation of the work estimation system of FIG. 1 . The position information and transport information may be generated by the processing device 21 of the work estimation device 2 instead of the processing device 11 of the terminal device 10. The processing device 11 of the terminal device 10 captures an image of the surroundings of the mobile object 1 and transmits it to the work estimation device 2. Then, in step S2A, the processing device 21 of the work estimation device 2 identifies the position of the mobile object 1 based on the image of the surroundings of the mobile object 1 and generates position information. The processing device 11 of the terminal device 10 also captures an image of the loading platform 1a and transmits it to the work estimation device 2. Then, in step S5A, the processing device 21 of the work estimation device 2 determines the presence or absence of luggage 3 based on the image of the loading platform 1a and generates transport information. Generating the position information and transport information by the processing device 21 of the work estimation device 2 can simplify the processing of the terminal device 10.

[0040] Step S4 may be performed in parallel or simultaneously with step S1.

[0041] 3 and 4 may be combined. For example, the processing device 11 of the terminal device 10 may generate the position information, and the processing device 21 of the work estimation device 2 may generate the transportation information. Alternatively, the processing device 11 of the terminal device 10 may generate the transportation information, and the processing device 21 of the work estimation device 2 may generate the position information.

[0042] As described above, the camera 13 may capture a moving image of the surroundings of the moving object 1, and the camera 14 may capture a moving image of the loading platform 1a. In this case, the moving image of the surroundings of the moving object 1 may be transmitted to the work estimation device 2 and stored in the storage device 24. Similarly, the moving image of the loading platform 1a may be transmitted to the work estimation device 2 and stored in the storage device 24.

[0043] FIG. 5 is a flowchart showing the task estimation process executed by the processing device 21 of FIG.

[0044] In step S11, the processing device 21 combines the position information and the transportation information to generate combined information.

[0045] FIG. 8 is a table showing an example of position information acquired by the processing device 21 of FIG. 1. FIG. 9 is a table showing an example of transport information acquired by the processing device 21 of FIG. 1. FIG. 10 is a table showing a first example of combined information generated in step S11 of FIG. 5. The position information indicates changes over time in the position of the mobile object 1. The transport information indicates changes over time in the transport status, regarding whether the mobile object 1 is transporting the luggage 3 or not. The combined information is generated by associating the position information and transport information at the same time with each other. The combined information indicates changes over time in the position and transport status that are associated with each other.

[0046] Referring again to FIG. 5, in step S12, the processing device 21 executes the entry and exit extraction process to determine the end times of various tasks performed by the mobile object 1.

[0047] FIG. 6 is a flowchart showing a subroutine of the entry and exit detection process (step S12) of FIG.

[0048] In step S21, the processing device 21 extracts, from the combined information, stay period information of the mobile object 1 in the loading / unloading area 32. The stay period information includes the stay period from the time of entry of the mobile object 1 into the loading / unloading area 32 to the time of exit. The stay period information further includes the transport state at the time of entry and the transport state at the time of exit.

[0049] FIG. 11 is a diagram for explaining the extraction of stay period information in step S21 of FIG. 6. Points P(t1) to P(t4) indicate the positions of the mobile object 1 at times t1 to t4 in FIG. 10. When the mobile object 1 moves from point P(t1) to point P(t2), that is, at time t2, the mobile object 1 enters the loading / unloading area 32. When the mobile object 1 moves from point P(t3) to point P(t4), that is, at time t4, the mobile object 1 exits the loading / unloading area 32. Furthermore, a black circle indicates that the mobile object 1 is transporting luggage 3, and a white circle indicates that the mobile object 1 is not transporting luggage 3. In the example of FIG. 11, the mobile object 1 transports luggage 3 when entering the loading / unloading area 32 and does not transport luggage 3 when exiting the loading / unloading area 32.

[0050] Similar to the case where there is luggage at the time of entry and no luggage at the time of exit as described with reference to Figure 11, information on the length of stay of the mobile object 1 in the loading / unloading area 32 is also extracted for the cases where there is no luggage at the time of entry and luggage at the time of exit, where there is no luggage at the time of entry and no luggage at the time of exit, and where there is luggage at the time of entry and luggage at the time of exit.

[0051] Fig. 12 is a table showing an example of stay period information extracted in step S21 of Fig. 6. As shown in Fig. 12, the stay period information includes an entry time, a transport state at the time of entry, an exit time, and a transport state at the time of exit.

[0052] If the length of time from the time the mobile object 1 enters the loading / unloading area 32 to the time it leaves is less than a predetermined threshold, the processing device 21 may discard it without extracting it as stay period information.

[0053] Referring again to FIG. 6, in step S22, the processing device 21 selects one of the extracted stay periods.

[0054] In step S23, the processing device 21 determines whether or not luggage 3 was being transported when entering and exiting the loading / unloading area 32, based on the stay period information. If there was no luggage at the time of entry and luggage at the time of exit, the process proceeds to step S24. If there was luggage at the time of entry and luggage at the time of exit, the process proceeds to step S26. If there was no luggage at the time of entry and luggage at the time of exit, the process proceeds to step S27. If there was luggage at the time of entry and luggage at the time of exit, the process proceeds to step S28. The processing device 21 estimates the work performed by the mobile object 1 using a predetermined work determination rule based on a combination of the transport state at the time of entry and the transport state at the time of exit, in other words, the change over time in the position and transport state.

[0055] If there is no luggage when entering and there is luggage when leaving, the mobile unit 1 is considered to be in the process of storing luggage 3 in the warehouse 30. In this case, storing is completed when luggage 3 is removed from the loading platform 1a and stored on the destination shelf 31. If there is luggage when entering and there is no luggage when leaving, the mobile unit 1 has removed luggage 3 from the warehouse 30. If there is no luggage when entering and there is no luggage when leaving, the mobile unit 1 was in an idle state, having not performed either storing or retrieving. If there is luggage when entering and there is luggage when leaving, it is unclear whether the mobile unit 1 entered and left the storage / retrieval area 32 while still transporting the same luggage 3, or whether it removed one luggage 3 and stored another luggage 3; in other words, it is unclear whether it performed storing or retrieving.

[0056] The processing device 21 estimates which of these four states (or tasks) the mobile body 1 is in. The processing device 21 identifies a plurality of time-contiguous work sections, each associated with one of these four states (or tasks). To identify each work section, the processing device 21 identifies, for a first and second task that are time-contiguous, the time section from the end time of the first task to the end time of the second task as the work section associated with the second task.

[0057] In step S24, the processing device 21 detects, based on the combined information, the time when the transport state changes from a state in which the luggage 3 is being transported to a state in which the luggage 3 is not being transported after the mobile object 1 exits from inside the first area 32 to the outside, i.e., the time when the luggage 3 is unloaded from the loading platform 1a. In step S25, the processing device 21 detects the time when the luggage 3 is unloaded from the loading platform 1a as the time when storage is completed.

[0058] Fig. 13 is a table showing a second example of the combined information generated in step S11 of Fig. 5. Detection of the warehousing completion time will be described with reference to the combined information of Fig. 13.

[0059] FIG. 14 is a diagram for explaining the detection of the end of warehousing in steps S24 and S25 in FIG. 6. Points P(t11) to P(t15) indicate the positions of the moving object 1 at times t11 to t15 in FIG. 13. When the moving object 1 moves from point P(t11) to point P(t12), that is, at time t12, the moving object 1 enters the loading / unloading area 32. When the moving object 1 moves from point P(t13) to point P(t14), that is, at time t14, the moving object 1 exits the loading / unloading area 32. When the moving object 1 moves from point P(t14) to point P(t15), that is, at time t15, the moving object 1 reaches an area near the shelf 31. The moving object 1 does not carry any luggage 3 when entering the loading / unloading area 32, but carries any luggage 3 when exiting the loading / unloading area 32. Furthermore, at point P (t15), the luggage 3 is removed from the loading platform 1a.

[0060] The storage ends when the luggage 3 is removed from the loading platform 1a, that is, at time t15.

[0061] 6 again, in step S26, the processing device 21 detects the time when the mobile object 1 exits from inside the loading / unloading area 32 to the outside as the time of exit completion. In step S27, the processing device 21 detects the time when the mobile object 1 exits from inside the loading / unloading area 32 to the outside as the time of idle state completion. In step S28, the processing device 21 detects the time when the mobile object 1 exits from inside the loading / unloading area 32 to the outside as the time of unknown work completion. The exit, idle state, and unknown work end when the mobile object 1 exits from inside the loading / unloading area 32 to the outside, as in the case described with reference to FIG. 11 .

[0062] In step S29, the processor 21 determines whether all the stay periods have been processed, and if YES, the process proceeds to step S13 in FIG. 5, and if NO, the process proceeds to step S30 in FIG.

[0063] In step S30, the processing device 21 selects another one of the extracted stay periods, and repeats steps S23 to S29.

[0064] Fig. 15 is a table showing an example of the end time of each task determined by executing the entry and exit detection process (step S12) of Fig. 5. As shown in Fig. 15, the end time of each task section associated with entry, exit, idle state, and unknown task can be known.

[0065] Referring back to FIG. 5, in step S13, the processing device 21 executes a temporary placement extraction process to detect the unloading and loading of luggage 3 in the temporary placement area 33.

[0066] FIG. 7 is a flowchart showing a subroutine of the temporary placement detection process (step S13) of FIG.

[0067] In step S31, the processing device 21 detects, based on the combined information, the time when the mobile object 1 is in the temporary storage area 33 and the transport status changes from "baggage present" to "baggage absent." In this case, it is estimated that the mobile object 1 has unloaded the baggage 3. In step S32, the processing device 21 detects, based on the combined information, the time when the mobile object 1 is in the temporary storage area 33 and the transport status changes from "baggage absent" to "baggage present." In this case, it is estimated that the mobile object 1 has loaded the baggage 3. The processing device 21 estimates the work performed by the mobile object 1 using a predetermined work determination rule, based on the change over time in the transport status when the mobile object 1 is in the temporary storage area 33.

[0068] Fig. 16 is a table showing a third example of the combined information generated in step S11 of Fig. 5. Detection of unloading and loading will be described with reference to the combined information of Fig. 16 .

[0069] FIG. 17 is a diagram for explaining the detection of unloading and loading in steps S31 and S32 in FIG. 7. Points P(t21) to P(t26) indicate the positions of the moving object 1 at times t21 to t26 in FIG. 16. When the moving object 1 moves from point P(t21) to point P(t22), that is, at time t22, the moving object 1 enters the temporary storage area 33. At time t23, the moving object 1 remains in the same position as point P(t22) (i.e., point P(t23)), and the transport status changes from "baggage present" to "baggage absent." The moving object 1 moves from point P(t23) to point P(t24). At time t25, the moving object 1 remains in the same position as point P(t24) (i.e., point P(t25)), and the transport status changes from "baggage absent" to "baggage present." When the moving object 1 moves from point P (t25) to point P (t26), that is, at time t26, the moving object 1 exits the temporary storage area 33.

[0070] Fig. 18 is a table showing an example of the unloading and loading times determined by executing the temporary placement detection process (step S13) of Fig. 5. As shown in Fig. 18, the unloading and loading times in the temporary placement area 33 can be known.

[0071] As described above, if separate temporary storage areas for receiving and for retrieving are provided, the unloading and loading of luggage 3 may be detected in each of these temporary storage areas.

[0072] Referring back to FIG. 5, in step S14, the processing device 21 generates a work report and outputs the work report via the output device 26.

[0073] FIG. 19 is a table showing a first example of the work report generated in step S14 of FIG. 6. Each row of the table in FIG. 19 indicates a work section associated with one of the following: receiving, retrieving, idle, and unknown work. The work sections are consecutive in time. The end time of each work section is obtained by executing step S12 of FIG. 5 (see FIG. 15). The start time of each work section is the end time of the immediately preceding work section. The start time of the first work section is set, for example, to the start time of work at the warehouse 30.

[0074] Fig. 20 is a diagram showing a second example of the work report generated in step S14 of Fig. 6. The content of Fig. 19 may be displayed as a timeline graph. The work report may further include the times of unloading and loading in the temporary storage area 33.

[0075] The processing device 21 may generate information indicating the movement line of the mobile object 1 based on the temporal change in the position information. In this case, the processing device 21 may generate the work report so as to further include information indicating the movement line of the mobile object 1 when the mobile object 1 performs multiple tasks. This makes it possible to refer to the temporal change in the position of the mobile object 1 from when the mobile object 1 performs a certain task to when the mobile object 1 performs the next task, for example, and to estimate the tasks performed by the mobile object 1 with higher accuracy.

[0076] The processing device 21 of the work estimation device 2 may execute the work estimation process each time it receives location information and transportation information from the terminal device 10, or may accumulate the location information and transportation information acquired over a predetermined period of time in the storage device 24 and execute the work estimation process in batch processing.

[0077] According to the work estimation system of the embodiment, it is possible to estimate the work performed by the mobile body 1, such as warehousing, retrieval, idle state, or unknown work, simply by acquiring the position information and transportation information of the mobile body 1. In particular, when a common loading / unloading area is used for loading and unloading, it is possible to estimate whether the mobile body 1 was staying in the loading / unloading area to perform loading or unloading.

[0078] According to the work estimation system of the embodiment, a mobile object with a special mechanism is not required, and it is only necessary to provide a terminal device 10 such as a mobile phone and cameras 13 and 14 on the mobile object 1, thereby avoiding a significant increase in costs.

[0079] According to the task estimation system according to the embodiment, it is not necessary to refer to the identifiers of the luggage 3, and it is not necessary to distinguish between multiple luggage 3.

[0080] In this way, the task estimation system according to the embodiment can estimate the task performed by the moving object 1 without significantly increasing costs and effort.

[0081] By estimating the work performed by the mobile object 1 and improving the work, the burden on the worker can be reduced and well-being can be achieved at the workplace.

[0082] [Example] A case where the work estimation system according to the embodiment is applied to an actual warehouse will be described below. The warehouse according to the example had one receiving / retrieving area, one temporary storage area for receiving, and one temporary storage area for retrieving. One day's worth of work performed by three mobile objects in this warehouse was analyzed. The work was either receiving, retrieving, or unknown work, and did not include idle states. By analyzing this warehouse using the work estimation system according to the embodiment, the following results were obtained.

[0083] ・Result 1: Number of temporary storages For one mobile object, the total number of unloading and loading operations in the temporary storage areas for storage and retrieval was approximately 200. Assuming that each unloading and loading operation takes 15 seconds, the time lost due to unloading and loading in the temporary storage area is approximately 50 minutes per day. ・Result 2: Proportion of storage and retrieval Retrieval accounted for more than half of the total working time per day, with storage accounting for approximately 20-30% of retrieval. ・Result 3: Time periods of storage and retrieval For all three mobile objects, storage and retrieval were not performed alternately; only one of storage and retrieval was performed continuously in each time period. Furthermore, for all three mobile objects, storage was performed around 11:00 and 14:00.

[0084] From the analysis results of warehousing, shipping, and temporary storage, areas that need improvement, such as long work times or frequent temporary storage, can be extracted, and the causes of the issues can be further analyzed by referring to video images, thereby deriving improvement proposals for the work.

[0085] Improvement plan 1: Adding queues and scheduling in storage areas Sometimes, there is no free space in the loading / unloading area because the previous unloading has not yet been completed, and cargo must be unloaded into a temporary storage area. This may be caused by conflicting tasks due to the mixing of mobile units that start work early and mobile units that start work later. In response to this, it is expected that the lack of free space can be resolved by, for example, increasing the number of queues in the loading / unloading area's storage areas and / or changing the queue arrangement.

[0086] Improvement plan 2: Increasing the temporary storage area for retrieval When retrieving goods, unnecessary movement can occur when goods are unloaded in a location that is not a temporary storage area. For example, it takes 10 seconds to move, 10 seconds to load, and another 10 seconds to move, resulting in 20 seconds of unnecessary movement. In response to this, it is expected that unnecessary movement can be reduced by reviewing the storage capacity of the shelves and converting part of the shelves into a temporary storage area. In this case, the mobile unit will, in principle, transport goods to be retrieved to the loading / unloading area. If there is no available storage space in the loading / unloading area, the mobile unit will unload the goods in the temporary storage area.

[0087] Improvement plan 3: Increasing the temporary storage area for warehousing When warehousing, unloading cargo in a location that is not a temporary storage area can result in unnecessary movement. For example, it can take 5 seconds to move, 10 seconds to load, and another 5 seconds to move, resulting in 10 seconds of unnecessary movement. This can be caused, for example, by the need for each mobile unit to evacuate its assigned cargo from the primary temporary storage area. In response to this, streamlining the temporary storage area and assigning a separate area to each mobile unit is expected to reduce unnecessary movement.

[0088] For example, if there are five instances of cargo being unloaded at locations other than temporary storage areas when leaving and entering the warehouse, implementing improvement plan 2 and improvement plan 3 can reduce unnecessary movement at the time of leaving and entering the warehouse by 100 seconds and 50 seconds, respectively.

[0089] [Effects of the embodiment, etc.] The work estimation device 2 according to the embodiment acquires location information indicating the location of the mobile body 1, acquires transport information indicating the transport status as to whether the mobile body 1 is transporting luggage 3 or not, estimates the work performed by the mobile body 1 using a predetermined work determination rule based on changes in the location and transport status over time, and outputs a work report including information indicating the estimated work.

[0090] This configuration makes it possible to estimate the work performed by the mobile object 1 without significantly increasing costs and effort.

[0091] The work estimation device 2 according to the embodiment may estimate the work performed by the mobile body 1 based on a first transport state when the mobile body 1 enters a predetermined first area 32 from the outside to the inside, and a second transport state when the mobile body 1 exits the first area 32 from the inside to the outside.

[0092] This configuration makes it possible to know the change in the position and transport state of the moving body 1 over time.

[0093] According to the work estimation device 2 according to the embodiment, the first area 32 may be part of the warehouse 30. In this case, if the first transport state indicates that the luggage 3 is not being transported, the second transport state indicates that the luggage 3 is being transported, and the transport state changes from the state of transporting the luggage 3 to the state of not transporting the luggage 3 after the mobile object 1 exits from inside to outside the first area 32, the work estimation device 2 estimates that the mobile object 1 has brought the luggage 3 into the warehouse 30. Furthermore, if the first transport state indicates that the luggage 3 is being transported, and the second transport state indicates that the luggage 3 is not being transported, the work estimation device 2 estimates that the mobile object 1 has taken the luggage 3 out of the warehouse 30.

[0094] This configuration makes it possible to estimate that the moving body 1 has stored and removed the cargo 3.

[0095] When the task estimation device 2 according to the embodiment cannot estimate the task performed by the moving body 1 using a predetermined task determination rule, the task estimation device 2 may estimate that the task performed by the moving body 1 is unknown.

[0096] This configuration makes it possible to estimate that the mobile object 1 has performed work other than storing and retrieving the cargo 3.

[0097] According to the work estimation device 2 according to the embodiment, the work estimation device 2 may identify a plurality of work sections that are consecutive in time and each of which is associated with one of a plurality of works performed by the mobile object 1. In this case, the work report includes a plurality of work sections.

[0098] This configuration makes it possible to identify multiple work sections that are consecutive in time.

[0099] According to the work estimation device 2 of the embodiment, identifying a plurality of work sections may include, for first and second works that are temporally consecutive, identifying a time section from the end time of a first work to the end time of a second work as a work section associated with the second work.

[0100] This configuration makes it possible to identify multiple work sections that are consecutive in time.

[0101] The task estimation device 2 according to the embodiment may generate information indicating the movement line of the moving object 1 based on a temporal change in the location information. In this case, the task report further includes information indicating the movement line of the moving object 1 when the moving object 1 performs a plurality of tasks.

[0102] This configuration allows the work performed by the moving body 1 to be estimated with higher accuracy.

[0103] According to the work estimation device 2 according to the embodiment, the moving body 1 may be a forklift.

[0104] This configuration allows the work performed by the forklift to be estimated.

[0105] According to the task estimation device 2 according to the embodiment, the position information may be acquired based on an image of the surroundings of the moving object 1 captured by the first camera 13 provided on the moving object 1 .

[0106] This configuration does not require a moving body with a special mechanism, and a significant increase in cost can be avoided.

[0107] According to the work estimation device 2 according to the embodiment, the mobile object 1 may include a loading platform 1 a. The transportation information may be acquired using machine learning based on an image of the loading platform 1 a captured by a second camera 14 provided on the mobile object 1.

[0108] This configuration does not require a moving body with a special mechanism, and a significant increase in cost can be avoided.

[0109] The work estimation system according to the embodiment includes a mobile object 1 equipped with a loading platform 1a, a camera 14 mounted on the mobile object 1 and configured to capture images of the loading platform 1a, a processing device, and an output device. The processing device acquires position information indicating the position of the mobile object 1, acquires transport information indicating whether the mobile object 1 is transporting a load 3 based on the image of the loading platform 1a, and estimates the work performed by the mobile object 1 using predetermined work determination rules based on changes in the position and transport status over time. The output device outputs a work report including information indicating the estimated work.

[0110] This configuration makes it possible to estimate the work performed by the mobile object 1 without significantly increasing costs and effort.

[0111] [Other Embodiments] As described above, the first to third embodiments have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made as appropriate.

[0112] Furthermore, the accompanying drawings and detailed description are provided to explain the embodiments. Therefore, the components described in the accompanying drawings and detailed description may include not only components essential for solving the problem, but also components that are not essential for solving the problem in order to illustrate the above technology. Therefore, the fact that these non-essential components are described in the accompanying drawings or detailed description should not be interpreted as immediately identifying these non-essential components as essential.

[0113] Furthermore, since the above-described embodiments are intended to illustrate the technology of the present disclosure, various modifications, substitutions, additions, omissions, etc. may be made within the scope of the claims or their equivalents.

[0114] In the above-described embodiment, the processing device 21 executes both the entry and exit detection process (step S12) and the temporary placement detection process (step S13). However, the processing device 21 may execute only one of the entry and exit detection process and the temporary placement detection process. Furthermore, instead of estimating whether the mobile object 1 has entered the warehouse, left the warehouse, been idle, or performed an unknown task, the processing device 21 may only estimate whether the mobile object 1 has entered the warehouse, or may only estimate whether the mobile object 1 has left the warehouse. Furthermore, instead of detecting both the unloading and loading of the luggage 3 in the temporary placement area 33, the processing device 21 may only detect the unloading of the luggage 3 in the temporary placement area 33, or may only detect the loading of the luggage 3 in the temporary placement area 33.

[0115] Instead of outputting the work report shown in FIG. 19 or FIG. 20, the processor 21 may output only the work improvement plan derived from the analysis results of the receiving, shipping, and temporary storage.

[0116] The location information is not limited to being obtained based on images captured by the camera 13, but may also be obtained by other methods, such as receiving a beacon signal transmitted from a base station installed in the warehouse 30 or receiving a GPS signal.

[0117] The transportation information is not limited to being obtained based on images taken by the camera 14, but may also be obtained by other methods, such as a switch, weight sensor, pressure sensor, or distance sensor, light sensor, passage sensor, color sensor, acceleration sensor, gyro sensor, vibration sensor, barcode reader, or QR code (registered trademark) reader provided on the loading platform 1a.

[0118] It is not limited to a single work estimation device 2 communicating with a terminal device 10 and performing the work estimation process, but a first computing device may communicate with one or more terminal devices 10 to obtain location information and transportation information (and / or images), and a second computing device may perform the work estimation process.

[0119] The mobile body 1 is not limited to directly unloading and loading the luggage 3 onto the conveying device 34, but may unload and load the luggage 3 in a temporary storage area close to the conveying device 34. In this case, a mobile body dedicated to moving the luggage 3 between the temporary storage area and the conveying device 34 is used.

[0120] The work estimation system according to the embodiment may be applied to a case where a mobile object is capable of moving both inside and outside a warehouse (or other facility). In this case, if a mobile object enters a warehouse carrying an item and exits the warehouse without carrying the item, the mobile object is determined to have stored the item. Also, if a mobile object enters a warehouse without carrying an item and exits the warehouse carrying the item, the mobile object is determined to have removed the item.

[0121] The work estimation system according to the embodiment is not limited to work in the warehouse 30, and may be applied to work in a factory, for example. In this case, if a mobile object carries a package and enters a manufacturing facility and then exits the manufacturing facility without carrying the package, the mobile object is determined to have performed a parts supply work. If this condition is not met, for example, if the mobile object enters the manufacturing facility without carrying the package, the mobile object is determined to have performed another work.

[0122] By comparing the planned work and the actual results, if the work is delayed, the work may be automatically optimized. For example, if a forklift is transporting a load, the work estimation device 2 may instruct the forklift driver to optimize the work. Also, if an automated guided vehicle (AGV) is transporting a load, the work estimation device 2 may instruct the automated guided vehicle to follow a revised optimal route.

[0123] [Summary of the embodiment] The work estimation device according to the first aspect acquires location information indicating the location of a mobile body, acquires transport information indicating the transport status regarding whether the mobile body is transporting cargo or not, estimates the work performed by the mobile body using a predetermined work determination rule based on the location and changes in the transport status over time, and outputs a work report including information indicating the estimated work.

[0124] According to the work estimation device of the second aspect, in the work estimation device of the first aspect, the work performed by the mobile object is estimated based on a first transport state when the mobile object enters a predetermined first area from the outside into the inside, and a second transport state when the mobile object exits from the inside of the first area to the outside.

[0125] According to a third aspect of the work estimation device, in the work estimation device according to the second aspect, the first area is part of a warehouse, and the work estimation device estimates that the mobile object has entered the warehouse if the first transport state indicates that the baggage is not being transported and the second transport state indicates that the baggage is being transported, and the transport state changes from a state in which the baggage is being transported to a state in which the baggage is not being transported after the mobile object exits from inside to outside the first area, and estimates that the mobile object has removed the baggage from the warehouse if the first transport state indicates that the baggage is being transported and the second transport state indicates that the baggage is not being transported.

[0126] The work estimation device according to a fourth aspect is the work estimation device according to one of the first to third aspects, and when the work performed by the moving object cannot be estimated using the predetermined work determination rule, it estimates that the work performed by the moving object is unknown.

[0127] According to the work estimation device of the fifth aspect, in the work estimation device of one of the first to fourth aspects, the work estimation device identifies a plurality of work sections that are consecutive in time, each of which is associated with one of a plurality of works performed by the mobile object, and the work report includes the plurality of work sections.

[0128] According to the work estimation device of the sixth aspect, in the work estimation device of the fifth aspect, identifying the plurality of work sections includes, for first and second works that are temporally consecutive, identifying a time section from the end time of the first work to the end time of the second work as a work section associated with the second work.

[0129] According to a seventh aspect of the work estimation device, in the work estimation device according to the fifth or sixth aspect, the work estimation device generates information indicating a line of movement of the moving object based on a change over time in the position information, and the work report further includes information indicating a line of movement of the moving object when the moving object performs a plurality of tasks.

[0130] According to an eighth aspect of the work estimation device, in the work estimation device according to one of the first to seventh aspects, the moving body is a forklift.

[0131] According to the ninth aspect of the task estimation device, in the task estimation device according to one of the first to eighth aspects, the position information is acquired based on an image of the surroundings of the moving body captured by a first camera provided on the moving body.

[0132] According to a tenth aspect of the work estimation device, in the work estimation device according to one of the first to ninth aspects, the mobile body is equipped with a loading platform, and the transportation information is acquired using machine learning based on an image of the loading platform captured by a second camera provided on the mobile body.

[0133] The work estimation system according to an eleventh aspect comprises a mobile body equipped with a loading platform; a camera mounted on the mobile body and capturing images of the loading platform; a processing device; and an output device, wherein the processing device acquires location information indicating the location of the mobile body; acquires transport information indicating the transport status as to whether the mobile body is transporting cargo or not based on the image of the loading platform; estimates the work performed by the mobile body using a predetermined work determination rule based on the location and changes in the transport status over time; and the output device outputs a work report including information indicating the estimated work.

[0134] An operation estimation device and an operation estimation system according to an aspect of the present disclosure are applicable to logistics sites, manufacturing sites, and the like.

[0135] 1, 1-1, 1-2 Mobile object 2 Work estimation device 3, 3-1, 3-2 Baggage 10 Terminal device 11 Processing device 12 Communication device 13, 14 Camera 21 Processing device 22 Communication device 23, 24 Storage device 25 Input device 26 Output device 30 Warehouse 31 Shelf 32-1, 32-2 Intake / out area 33-1, 33-2 Temporary storage area 34-1, 34-2 Transport device

Claims

1. A work estimation device that acquires location information indicating the location of a mobile object, acquires transport information indicating a transport status regarding whether the mobile object is transporting a load or not, estimates work performed by the mobile object using a predetermined work determination rule based on changes over time in the location and the transport status, and outputs a work report including information indicating the estimated work.

2. The work estimation device according to claim 1, which estimates the work performed by the mobile object based on a first transport state when the mobile object enters a predetermined first area from the outside into the inside, and a second transport state when the mobile object exits the first area from the inside to the outside.

3. The work estimation device according to claim 2, wherein the first area is part of a warehouse, and when the first transport state indicates that the luggage is not being transported and the second transport state indicates that the luggage is being transported, and when the transport state changes from a state in which the luggage is being transported to a state in which the luggage is not being transported after the mobile object has exited from inside to outside the first area, it is estimated that the mobile object has entered the warehouse, and when the first transport state indicates that the luggage is being transported and the second transport state indicates that the luggage is not being transported, it is estimated that the mobile object has removed the luggage from the warehouse.

4. The task estimation device according to claim 1, wherein, if the task performed by the mobile object cannot be estimated using the predetermined task determination rule, the task performed by the mobile object is estimated to be unknown.

5. The work estimation device according to claim 1, wherein the work estimation device identifies a plurality of work sections that are consecutive in time, each of which is associated with one of a plurality of works performed by the mobile object, and the work report includes the plurality of work sections.

6. The work estimation device according to claim 5, wherein identifying the plurality of work intervals includes identifying, for first and second works that are consecutive in time, a time interval from an end time of the first work to an end time of the second work as a work interval associated with the second work.

7. The task estimation device according to claim 5, wherein the task estimation device generates information indicating the movement line of the mobile object based on temporal changes in the location information, and the task report further includes information indicating the movement line of the mobile object when the mobile object performs multiple tasks.

8. The task estimation device according to claim 1, wherein the moving object is a forklift.

9. The task estimation device according to claim 1, wherein the position information is acquired based on an image of the surroundings of the moving object captured by a first camera provided on the moving object.

10. The work estimation device according to claim 1, wherein the mobile body is equipped with a loading platform, and the transportation information is acquired using machine learning based on an image of the loading platform captured by a second camera provided on the mobile body.

11. A work estimation system comprising: a mobile body with a loading platform; a camera mounted on the mobile body and capturing images of the loading platform; a processing device; and an output device, wherein the processing device acquires position information indicating the position of the mobile body; acquires transport information indicating the transport status as to whether the mobile body is transporting a load or not based on the image of the loading platform; estimates work performed by the mobile body using a predetermined work determination rule based on changes over time in the position and the transport status; and the output device outputs a work report including information indicating the estimated work.

Images