Work classification device, work classification method, and program

Abstract

To provide a work classification device capable of making effective use of the location information of a moving body by classifying work performed by the moving body into units of work while using the location information of the moving body.SOLUTION: The work classification device includes a processor and a communications unit. The communications unit receives a piece of location information and a piece of information about the holding state of the moving body from the moving body. The processor is configured to acquire the movement of the moving body between areas based on the location information, and classify the work of the moving body into units of work based on the holding state and the movement of the moving body between areas.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to a task classification device, a task classification method, and a program. [Background technology]

[0002] There are mobile objects that transport luggage indoors, such as in a logistics warehouse, while measuring their positions. Patent Document 1 discloses a luggage location management device that measures the position of a vehicle using positioning technologies such as a global positioning system (GPS), a wireless local area network (LAN) positioning, and infrared positioning.

[0003] Non-Patent Document 1 discloses a positioning method called Visual-SLAM (Visual Simultaneous Localization and Mapping). In Visual-SLAM, a mobile object equipped with a camera moves while capturing images of its surroundings, and the amount of movement of the mobile object is calculated based on the amount of movement of feature points in the multiple captured images. This makes it possible for Visual-SLAM to estimate the current position of the mobile object and generate a map based on the trajectory of the mobile object. Note that calculating the position, direction, etc. of a mobile object based on multiple captured images is called visual odometry. As a function of this visual odometry, the amount of movement of a mobile object may be calculated based on the amount of movement of feature points in the multiple captured images.

[0004] One known positioning method is one that uses markers. For example, a positioning device stores image data of markers attached (sticked) to objects such as pillars or walls of a building, and map information including the installation positions of the markers. The marker information and the installation positions are associated with each other.

[0005] The positioning device calculates the relative position of the moving object with respect to the marker based on the marker image captured by the camera. The positioning device refers to map information to acquire the installation position of the marker captured by the camera, and calculates the position of the moving object on the map of the map information based on the acquired installation position and the calculated relative position of the moving object with respect to the marker.

[0006] Patent Document 2 discloses a classification system that classifies the operating status of a conveying device using a first signal transmitted from a first detector that detects the movement of the conveying device and a second signal transmitted from a second detector that detects the presence or absence of an article on the conveying device. Patent Document 2 does not take into consideration the area through which the conveying device moves when classifying the operating status of the conveying device. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-219229 [Patent Document 2] Japanese Patent Application Publication No. 2019-191709 [Patent Document 3] International Publication No. 2020 / 137315 [Non-patent literature]

[0008] [Non-Patent Document 1] R. Mur-Artal, et al., "ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras", IEEE Transactions on Robotics, Volume: 33, Issue: 5, Oct. 2017 Summary of the Invention [Problem to be solved by the invention]

[0009] As described above, several positioning technologies for determining the position of a mobile object have been proposed, but it is desirable to effectively utilize the position information of the mobile object obtained by the positioning technologies.

[0010] Non-limiting examples of the present disclosure contribute to providing an information processing device, a task classification method, and a program that classify tasks performed by a mobile object into task units using location information of the mobile object. [Means for solving the problem]

[0011] A task classification device according to one embodiment of the present disclosure is a task classification device including a processor and a communication unit, wherein the communication unit receives location information from a mobile object and information regarding the object holding state of the mobile object, and the processor obtains the movement of the mobile object between areas based on the location information, and classifies the tasks of the mobile object by task unit based on the holding state and the movement of the mobile object between areas.

[0012] A task classification method according to one embodiment of the present disclosure is a task classification method for a task classification device, which receives location information from a mobile object and information regarding the object holding state of the mobile object, obtains the movement of the mobile object between areas based on the location information, and obtains task units of the mobile object based on the holding state and the movement of the mobile object between areas.

[0013] A program according to one embodiment of the present disclosure causes a task classification device to perform processing to receive location information from a mobile object and information regarding the object holding state of the mobile object, obtain movement of the mobile object between areas based on the location information, and obtain task units of the mobile object based on the holding state and movement of the mobile object between areas.

[0014] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0015] According to an embodiment of the present disclosure, the tasks performed by the mobile body can be classified by task unit using the location information of the mobile body.

[0016] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features. [Brief explanation of the drawings]

[0017] [Figure 1] A diagram showing a moving object equipped with a positioning device [Figure 2] A top view of the area in which the moving object moves [Figure 3] Diagram explaining marker positioning [Figure 4] Diagram explaining feature point positioning [Figure 5] FIG. 1 is a diagram illustrating an example of the configuration of a task classification system according to an embodiment of the present disclosure. [Figure 6] FIG. 10 is a diagram showing an example of task unit pattern information that defines task units for a mobile object. [Figure 7] A diagram explaining the classification of work units in a mobile device [Figure 8A] A diagram explaining how information on the state of holding an object is acquired on a moving object. [Figure 8B] A diagram explaining how information on the state of holding an object is acquired on a moving object. [Figure 9] Diagram explaining the area [Figure 10] A diagram showing an example of area information [Figure 11] A diagram explaining the timing of generating movement trajectory data. [Figure 12] A diagram showing an example of the data structure of movement trajectory data [Figure 13] A diagram showing an example of the data structure of movement trajectory data [Figure 14]A Diagram explaining division of movement trajectory data [Figure 15] An example of the data structure of a work unit list [Figure 16] An example of a screen showing the work results for one day [Figure 17A] A diagram showing an example of a screen during the inventory issue cycle [Figure 17B] A diagram showing an example of a screen during the issue cycle [Figure 18] A diagram showing an example of a partial analysis screen [Figure 19] A diagram showing an example of a partial analysis screen [Figure 20] Flowchart showing an example of the operation of the positioning device [Figure 21] A flowchart showing an example of the operation of generating a task unit list of a task classification device. [Figure 22] A flowchart showing an example of the operation of the task classification device in task analysis. [Figure 23] FIG. 1 is a diagram illustrating an example of functional blocks of a positioning device. [Figure 24] FIG. 1 is a diagram showing an example of functional blocks of a video storage device. [Figure 25] FIG. 1 is a diagram showing an example of a functional block of a task classification device. [Figure 26] FIG. 1 is a diagram illustrating an example of hardware of an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or redundant explanation of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following explanation and to facilitate understanding by those skilled in the art.

[0019] The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0020] <Method for measuring the position of moving objects> FIG. 1 is a diagram showing a mobile object 2 equipped with a positioning device 1. The mobile object 2 is, for example, a vehicle such as a forklift or a picking cart. The mobile object 2, for example, stores and retrieves items from a logistics warehouse. The mobile object 2 is equipped with the positioning device 1 and a camera 3.

[0021] The positioning device 1 receives image data of an image captured by the camera 3. Based on the image data received from the camera 3, the positioning device 1 measures the position of the moving object 2 by positioning based on image recognition of markers (images of the markers) attached to the object, and by positioning based on the amount of movement of feature points in the image, such as visual odometry. That is, the positioning device 1 measures the position of the moving object 2 using two positioning methods (see, for example, Patent Document 3).

[0022] The position of the positioning device 1 may be regarded as the position of the moving object 2. That is, the positioning device 1 may measure its own position and use it as the position of the moving object 2.

[0023] Hereinafter, positioning based on image recognition of markers may be referred to as marker positioning, and positioning based on the amount of movement of feature points within an image may be referred to as feature point positioning. Positioning based on both marker positioning and feature point positioning may be referred to as a hybrid positioning method.

[0024] The camera 3 is, for example, a stereo camera. The camera 3 may be a depth camera or a compound eye camera, as long as it outputs image data that allows the positioning device 1 to calculate the distance to the target object.

[0025] FIG. 2 is a diagram showing an area A2a in which the moving object 2 moves, viewed from above. The square and circle shapes shown in the area A2a represent objects such as luggage stored in a building such as a logistics warehouse, pillars of the building, and walls of the building. Markers A2b to A2e are attached to the objects as shown in FIG. 2. Note that, for ease of understanding, the markers A2b to A2e are drawn on the top surface of the object in FIG. 2, but the markers A2b to A2e may also be attached to the side of the object so that the camera 3 can easily capture the object.

[0026] Map information (map data) of the area A2a is stored in the positioning device 1. In the map information, for example, two-dimensional coordinates are set.

[0027] The map information includes marker information such as image data of the markers A2b to A2e and position information indicating the attachment positions of the markers A2b to A2e. The marker information and the attachment positions are associated with each other.

[0028] The area in which the moving object 2 moves may be indoors or outdoors. The term "attaching" may be rephrased as "installing."

[0029] Marker positioning Fig. 3 is a diagram illustrating marker positioning. Fig. 3 shows an image A3a captured by a camera 3. Image A3a includes a marker A3b.

[0030] The positioning device 1 receives an image A3a captured by the camera 3. The positioning device 1 calculates the relative position of the moving object 2 with respect to the marker A3b based on the appearance of the marker A3b, such as the angle (tilt), shape, and size of the image of the marker A3b included in the received image A3a. For example, the positioning device 1 calculates the relative position, such as the distance and direction of the moving object 2 with respect to the marker A3b. The positioning device 1 calculates the position (absolute position) of the moving object 2 on the map of the map information based on the calculated relative position and the position information of the marker A3b included in the map information.

[0031] Feature point positioning and hybrid positioning There may be cases where the marker A3b is not present in the location within the area A2a where the moving object 2 moves or in the traveling direction (the shooting direction of the camera 3) of the moving object 2. In this case, the positioning device 1 measures the position of the moving object 2 by feature point positioning.

[0032] Fig. 4 is a diagram illustrating feature point positioning. Fig. 4 shows an image A4a captured by the camera 3. The black squares shown in the image A4a indicate feature points calculated by the positioning device 1. Note that the image A4a does not include markers.

[0033] The positioning device 1 receives an image A4a captured by the camera 3. If the received image A4a does not include a marker image, the positioning device 1 calculates feature points for each frame (or n frames, where n is a positive integer), and calculates the relative movement amount, such as the movement distance and movement direction (rotation angle) of the moving object 2, from the difference (deviation) in the positions of the feature points for each frame.

[0034] For example, the positioning device 1 calculates edges, such as corners of an object, where the shape changes and where the color changes, as feature points from the image A4a. The positioning device 1 calculates the feature points for each frame. The positioning device 1 calculates the relative movement amount of the moving object 2 from the difference in the positions of the feature points for each frame.

[0035] The calculation of the feature points is not limited to the above example. The feature points may be calculated using existing techniques. The feature points may also be referred to as feature amounts.

[0036] After calculating the amount of relative movement, the positioning device 1 calculates the position of the moving object 2 within the area A2a based on the calculated amount of relative movement and the positioning result of the marker positioning. For example, the positioning device 1 adds the amount of relative movement to the positioning result of the marker positioning to calculate the position (absolute position) of the moving object 2 on the map of the map information.

[0037] The positioning device 1 performs marker positioning when the marker is again included in the image received from the camera 3. When the marker is no longer included in the image received from the camera 3, the positioning device 1 performs feature point positioning based on the position of the moving object 2 measured by marker positioning.

[0038] In this way, the positioning device 1 performs marker positioning when the marker is photographed by the camera 3, and performs feature point positioning when the marker is not photographed by the camera 3. As a result, in the hybrid positioning method, even if the positioning result of the feature point positioning contains an error, the error is corrected by the positioning result of the marker positioning.

[0039] Furthermore, compared to the Visual-SLAM positioning method, the hybrid positioning method has the advantage that it does not require data such as point cloud data of the location where positioning is performed (for example, area A2a shown in FIG. 2) and that processing is lighter.

[0040] <Consideration> Although the hybrid positioning allows highly accurate positioning results to be easily obtained for the moving object 2, it is desirable to make effective use of the obtained positioning results.

[0041] For example, if the positioning results obtained by hybrid positioning can be used to classify the work performed by the mobile body 2 into work content (work units) such as "storing" and "unstoring" of cargo, it will be possible to understand the work performance of the mobile body 2 and improve the work efficiency of the mobile body 2.

[0042] Therefore, in this disclosure, the positioning results of the moving object 2 are used to classify the tasks performed by the moving object 2 into task units.

[0043] <System configuration> Figure 5 is a diagram showing an example configuration of a task classification system according to an embodiment of the present disclosure. As shown in Figure 5, the task classification system includes the mobile object 2 shown in Figure 1, a task classification device 11, a video storage device 12, and a terminal 13. In Figure 5, the same components as in Figure 1 are assigned the same reference numerals.

[0044] The task classification device 11 and the video storage device 12 may be configured by an information processing device such as a server or a personal computer, and the positioning device 1 and the terminal 13 may be configured by an information processing device such as a tablet or a personal computer.

[0045] The positioning device 1 wirelessly communicates with the video storage device 12 via a wireless network such as Wi-Fi (registered trademark) or local 5G. The positioning device 1 transmits the positioning result of the moving object 2 measured by hybrid positioning and image data of the camera 3 to the video storage device 12.

[0046] The task classifier 11 communicates with a video storage device 12 via a network, such as the Internet.

[0047] The task classification device 11 receives the positioning results of the mobile object 2 measured by the positioning device 1 via the video storage device 12. The task classification device 11 uses the positioning results of the mobile object 2 obtained via the video storage device 12 to classify (acquire) tasks performed by the mobile object 2 in task units. As will be described later, the task classification device 11 also uses information other than the positioning results of the mobile object 2 to classify the tasks performed by the mobile object 2 in task units.

[0048] The task classification device 11 analyzes the tasks of the mobile object 2 in response to a request from the terminal 13. The task classification device 11 analyzes the tasks of the mobile object 2 using the classified task units and transmits the task analysis results to the terminal 13.

[0049] The video storage device 12 receives the positioning results of the moving object 2 transmitted from the positioning device 1 and the image data of the camera 3. The video storage device 12 transmits the received positioning results of the moving object 2 to the task classification device 11. The video storage device 12 stores the received image data of the camera 3.

[0050] The terminal 13 communicates with the task classification device 11 and the video storage device 12 via a network such as the Internet.

[0051] In response to a user operation, the terminal 13 requests the task analysis results of the moving object 2 from the task classification device 11. The terminal 13 receives the task analysis results requested from the task classification device 11 from the task classification device 11 and displays them on the display.

[0052] In response to a user operation, the terminal 13 requests image data of the camera 3 from the task classification device 11. When the task classification device 11 receives a request for image data of the camera 3 from the terminal 13, it redirects the image data request to the video storage device 12. The video storage device 12 transmits the image data in response to the redirection to the terminal 13. The terminal 13 displays an image based on the image data transmitted from the video storage device 12 on a display.

[0053] Although FIG. 5 shows one mobile object 2, the task classification system may have two or more mobile objects.

[0054] <Work unit pattern information> In a logistics warehouse, a mobile object 2 such as a forklift or a picking cart performs tasks such as "storing" and "retrieving" items. For example, when the mobile object 2 moves from a berth area to a warehouse area of ​​the logistics warehouse while holding (or grasping) a package, the mobile object 2 can be considered to have performed a "storing" task. For example, when the mobile object 2 moves from a warehouse area to a berth area of ​​the logistics warehouse while holding a package, the mobile object 2 can be considered to have performed a "retrieving" task.

[0055] Therefore, a task unit for the mobile object 2 is defined based on the state of holding an object in the mobile object 2 and the movement (transition) of the mobile object 2 between areas.

[0056] Fig. 6 is a diagram showing an example of task unit pattern information that defines task units of the moving object 2. The task unit pattern information shown in Fig. 6 is stored in task classification device 11. The task unit pattern information has area transition information indicated by arrow A6a and information on the holding state of an object on the moving object 2 indicated by arrow A6b.

[0057] As indicated by arrow A6a, the area transition information is made up of four pieces of information (patterns). The area transition patterns of the mobile object 2 include a pattern in which the mobile object 2 moves from one berth area to another (moving within a berth area), a pattern in which the mobile object 2 moves from a berth area to a warehouse area, a pattern in which the mobile object 2 moves from a warehouse area to a berth area, and a pattern in which the mobile object 2 moves from one warehouse area to another (moving within a warehouse area).

[0058] As indicated by arrow A6b, the information on the holding state is composed of two pieces of information (patterns). The patterns of the holding state of an object in the moving object 2 include a non-holding pattern in which the moving object 2 does not hold an object, and a holding pattern in which the moving object 2 holds an object.

[0059] Four pieces of area transition information are associated with two pieces of holding state information, and each association defines a task unit of the moving object 2.

[0060] For example, when the mobile object 2 moves from berth area to berth area (moves within the berth area) while holding an object, the mobile object 2 can be considered to be performing the task of adjusting the placement of the object within the berth area. Therefore, a task unit "adjust placement" is defined at the position where "holding" and "berth area to berth area" in the task unit pattern information correspond.

[0061] For example, when the mobile object 2 moves from a berth area to a warehouse area while holding an item, the mobile object 2 can be considered to be performing the work of putting the item into storage. Therefore, the task unit "storage" is defined at the position where "holding" in the task unit pattern information corresponds to "from berth area to warehouse area" in the area transition.

[0062] For example, when the mobile object 2 moves from a warehouse area to a berth area while holding an item, the mobile object 2 can be considered to be performing the work of retrieving the item. Therefore, the task unit "retrieval" is defined at the position where "retention" in the task unit pattern information corresponds to "from warehouse area to berth area" in the area transition.

[0063] For example, when the mobile object 2 moves from warehouse area to warehouse area (moving within the warehouse area) while holding an item, the mobile object 2 can be considered to be performing the task of digging up the item (inventory transfer task). Therefore, the task unit "dig up" is defined at the position where "holding" in the task unit pattern information corresponds to "warehouse area to warehouse area" in the area transition.

[0064] For example, when the moving object 2 moves from one berth area to another (moves within a berth area) without holding an object, the moving object 2 can be considered to be performing a non-holding movement task. Therefore, a task unit "non-holding movement" is defined at a position where "non-holding" in the task unit pattern information corresponds to "from berth area to berth area."

[0065] For example, when the mobile object 2 moves from a berth area to a warehouse area without holding an object, the mobile object 2 can be considered to be performing a movement task for retrieval. Therefore, a task unit "movement for retrieval" is defined at the position where "not holding" in the task unit pattern information corresponds to "from berth area to warehouse area" in the area transition.

[0066] For example, when the mobile object 2 moves from a warehouse area to a berth area without holding an object, the mobile object 2 can be considered to be performing a movement task for warehousing. Therefore, a task unit "movement for warehousing" is defined at the position where "not holding" in the task unit pattern information corresponds to "from warehouse area to berth area" in the area transition.

[0067] For example, if the mobile object 2 moves from warehouse area to warehouse area (moves within the warehouse area) without carrying any goods, the mobile object 2 can be considered to be performing a wasteful task. Therefore, a task unit "wasteful movement" is defined at the position where "non-holding" in the task unit pattern information corresponds to "warehouse area to warehouse area" in the area transition.

[0068] Although four examples of area transition patterns have been given above, the number of area transition patterns is not limited to these. For example, the number of area transition patterns may be increased according to the number of areas through which the moving object 2 moves.

[0069] Although two examples of holding state patterns have been given above, this is not limiting. For example, the holding state patterns may be increased by adding the type, size, or weight of the object held by the mobile object 2 to the holding state of the object held by the mobile object 2. For example, if the type of object is added to the holding state patterns, task units according to the type of object, such as the storage and delivery of object A and the storage and delivery of object B, may be added as task units.

[0070] <Classification of work units> As shown in Figure 6, when task unit pattern information is defined, the tasks performed by the mobile body 2 are classified (discriminated) in the task units of the task unit pattern information based on the state of the mobile body 2 holding items and the movement of the mobile body 2 between areas.

[0071] FIG. 7 is a diagram for explaining the classification of task units in the mobile object 2. FIG. 7 shows a task image of the mobile object 2, the object holding state of the mobile object 2, area transitions of the mobile object 2, and task units of the mobile object 2. Holding states 0 and 1 shown in FIG. 7 correspond to holding states 0 and 1 shown in FIG. 6. Area transitions A and B shown in FIG. 7 correspond to area transitions A and B shown in FIG. 6.

[0072] A task unit of the moving object 2 can be divided, for example, at the boundary of a change in the holding state of the moving object 2. In other words, when the moving object 2 holds or puts down an object, it can be considered that some task in the moving object 2 has changed, and a task unit can be divided. For example, a task unit of the moving object 2 can be divided at the boundary of sections A7a, A7b, A7c, and A7d shown in FIG. 7.

[0073] As shown in section A7a in Fig. 7, the moving object 2 moves within the berth area without holding an object. In this case, the task unit of the moving object 2 in section A7a is classified as "non-holding movement" based on the non-holding (0) of the task unit pattern information shown in Fig. 6 and the area transition from berth area (A) to berth area (A).

[0074] As shown in section A7b of Fig. 7, the mobile object 2 carries an object and moves from the berth area to the warehouse area. In this case, the task unit of the mobile object 2 in section A7b is classified as "warehousing" based on the retention (1) of task unit pattern information shown in Fig. 6 and the area transition from the berth area (A) to the warehouse area (B).

[0075] As shown in section A7c in Fig. 7, the mobile object 2 unloads the goods and moves from the warehouse area to the berth area. In this case, the task unit of the mobile object 2 in section A7c is classified as "movement for warehousing" based on the non-holding (0) of the task unit pattern information shown in Fig. 6 and the area transition from the warehouse area (B) to the berth area (A).

[0076] As shown in section A7d of Fig. 7, the mobile object 2 carries an object and moves from the berth area to the warehouse area. In this case, the task unit of the mobile object 2 in section A7d is classified as "warehousing_2" (second warehousing task) based on the task unit pattern information retention (1) shown in Fig. 6 and the area transition from the berth area (A) to the warehouse area (B).

[0077] In this way, the work performed by the mobile object 2 is classified into task units based on the state of holding items on the mobile object 2 and the movement between areas of the mobile object 2. Therefore, if the state of holding items on the mobile object 2 and the movement between areas of the mobile object 2 can be obtained, the work of the mobile object 2 can be classified into task units by referring to the task unit pattern information shown in Fig. 6.

[0078] The following describes how to obtain information on the holding state and information on the area where the mobile object 2 is located (mobile object area information). Note that the mobile object area information is obtained in order to obtain information on the movement of the mobile object 2 between areas (area transition).

[0079] -Getting information on the holding status 8A and 8B are diagrams for explaining the acquisition of information on the state of holding an object in the moving object 2. For the sake of simplicity, the moving object 2 will be described as a forklift.

[0080] For hybrid positioning, the forklift is equipped with a camera 3 that captures an image in front of the forklift. Because the camera 3 captures an image in front of the forklift, the angle of view of the camera 3 includes the fork part of the forklift. The positioning device 1 takes advantage of the fact that the angle of view of the camera 3 includes the fork part of the forklift, i.e., the part where an object is held (carried), and uses image data from the camera 3 to acquire information about the state of the forklift holding an object.

[0081] Fig. 8A shows an image in which a load A8a is held on the forks of a forklift, and Fig. 8B shows an image of image data in which no load is held on the fork of the forklift.

[0082] The positioning device 1 uses existing image analysis technology to determine whether or not an object is loaded on the fork of the forklift.

[0083] For example, as shown in FIG. 8A, when the forks of the forklift are not included (not shown) in the image data of the camera 3, the positioning device 1 determines that the forklift is holding an object.

[0084] For example, as shown in FIG. 8B, when a fork A8b of the forklift is included (shown) in the image data of the camera 3, the positioning device 1 determines that the forklift is not holding an object.

[0085] In this way, the positioning device 1 determines whether the forklift is holding an object or not based on the image data of the camera 3. That is, the positioning device 1 acquires the holding state of the moving object 2 based on the image data of the camera 3.

[0086] The positioning device 1 transmits the positioning result (movement trajectory data) of the moving object 2, including the holding state, to the video storage device 12. The movement trajectory data transmitted to the video storage device 12 is transmitted to the task classification device 11. The positioning result (movement trajectory data) of the moving object 2, including the holding state, will be described below in <Movement trajectory data>.

[0087] In the above description, the positioning device 1 determines whether the forklift is holding an object based on whether an image of the fork is included in the image of the camera 3, but this is not limiting. For example, the positioning device 1 may obtain information on the holding state of an object based on whether an image of an empty pallet is included in the image of the camera 3, or whether an image of the object is included in the image of the camera 3.

[0088] Furthermore, the positioning device 1 acquires information on the holding state of an object using image data from the camera 3, but this is not limiting. For example, the forklift may be equipped with a weight sensor that measures the weight of an object held by the forks. The positioning device 1 may acquire information on the holding state of an object based on a value from the weight sensor transmitted from the forklift.

[0089] · Acquisition of mobile area information Fig. 9 is a diagram illustrating areas. Fig. 9 shows area A2a in which the mobile object 2 shown in Fig. 2 moves. Area A2a in Fig. 9 is hatched to indicate a berth area A9a and a warehouse area A9b. Note that A and B shown in parentheses in Fig. 9 correspond to A and B shown in Fig. 6 and Fig. 7.

[0090] A plurality of areas may be set for the area A2a in which the mobile object 2 moves, based on task unit pattern information. In other words, a plurality of areas may be set for the area A2a in which the mobile object 2 moves, depending on how the task units are defined. For example, if the task units of the mobile object 2 are defined using eight patterns as shown in FIG. 6, the area in which the mobile object 2 moves may be classified into two areas, a berth area A9a and a warehouse area A9b, as shown in FIG. 9.

[0091] The task classification device 11 stores information (area information) relating to the area in which the mobile object 2 moves. For example, the task classification device 11 stores area information indicating which part of the area A2a is the berth area A9a and which part of the area A2a is the warehouse area A9b.

[0092] 10 is a diagram showing an example of area information. The area information is stored in the task classification device 11 as described above.

[0093] The area information is indicated, for example, by two-dimensional coordinates in map information of region A2a. For example, when an area is defined as a quadrilateral shape as shown in Fig. 9, the area information may be defined by the vertices of the quadrilateral shape as shown in Fig. 10. In this case, the area in which the moving object 2 is located is determined depending on whether the positioning result of the moving object 2 falls within any of the quadrilateral shapes indicated in the area information.

[0094] The task classification device 11 receives the positioning results of the moving object 2 via the video storage device 12. Based on the received positioning results of the moving object 2, the task classification device 11 refers to the area information shown in Fig. 10 and acquires moving object area information (information on the area where the moving object 2 is located).

[0095] <Trajectory data> The positioning device 1 performs hybrid positioning and generates movement trajectory data (positioning results) of the moving object 2.

[0096] 11 is a diagram illustrating the timing of generating movement trajectory data. Fig. 11 shows a flow line A11a of a moving object 2. The moving object 2 moves in the direction indicated by an arrow A11b in Fig. 11.

[0097] The positioning device 1 generates movement trajectory data in time series at points A11c, A11d, A11e, A11f, and A11g shown in Fig. 11, for example. Each piece of movement trajectory data for points A11c, A11d, A11e, A11f, and A11g includes information on the holding state of the moving object 2 at the positions of points A11c, A11d, A11e, A11f, and A11g. The generated movement trajectory data is transmitted to the video storage device 12 and then to the task classification device 11.

[0098] Fig. 12 is a diagram showing an example of the data configuration of movement trajectory data. As shown in Fig. 12, the movement trajectory data includes a cart ID, a cart name, an operator ID, an operator name, a frame number, position coordinates X and Y, a holding state, and a time.

[0099] The vehicle ID is an identifier given to the moving object 2. The vehicle name is a name given to the moving object 2.

[0100] The worker ID is an identifier assigned to the worker who operates the moving object 2. The worker name is the name of the worker who operates the moving object 2.

[0101] The frame number is the frame number of the image captured by the camera 3. The frame number is the frame number of the image frame when the position coordinates, which will be explained next, are measured.

[0102] The position coordinates X and Y are coordinates within the area A2a in which the moving object 2 moves. The position coordinates X and Y are obtained by the positioning device 1 performing hybrid positioning.

[0103] The holding state indicates the state of holding an object by the moving body 2. For example, the holding state "0" indicates that the moving body 2 is not holding an object. The holding state "1" indicates that the moving body 2 is holding an object. The holding state is generated based on the image data of the camera 3 as described in the above "acquisition of holding state information".

[0104] The movement trajectory data shown in FIG. 12 is generated by the positioning device 1 mounted on the moving body 2 and transmitted to the video storage device 12. Then, the movement trajectory data is transmitted to the work classification device 11.

[0105] When the work classification device 11 receives the movement trajectory data, it refers to the area information as described in the above "acquisition of moving body area information" and acquires the moving body area information (information on the area where the moving body 2 is located) of the moving body 2.

[0106] For example, the work classification device 11 refers to the area information described in FIG. 10 based on the position coordinates X, Y of the received movement trajectory data of the moving body 2 and acquires the moving body area information of the moving body 2. More specifically, when the position coordinates X, Y of the movement trajectory data are inside the quadrilateral with (xa1, ya1), (xa2, ya2), (xa3, ya3), (xa4, ya4) shown in FIG. 10 as vertices, the work classification device 11 acquires "birth area (A)" as the moving body area information.

[0107] When the work classification device 11 acquires the moving body area information of the moving body 2, it generates area movement trajectory data (A movement trajectory data) by including it in the movement trajectory data received from the video storage device 12.

[0108] FIG. 13 is a diagram showing an example of the data configuration of A movement trajectory data. The A movement trajectory data shown in FIG. 13 is different from the movement trajectory data shown in FIG. 12 in that it has moving body area information.

[0109] The mobile object area information indicates the area in which the mobile object 2 is located. For example, the mobile object area information "A" shown in Fig. 13 indicates that the mobile object 2 is located in the bath area.

[0110] <Working Unit List> The task classification device 11 generates a task unit list using the task unit pattern information described in FIG. 6 and the A movement trajectory data described in FIG.

[0111] As explained above in <Classification of Task Units>, task units of the moving object 2 can be divided, for example, at the boundary of a change in the holding state of the moving object 2. Therefore, the task classification device 11 divides the A movement trajectory data based on the holding state.

[0112] Fig. 14 is a diagram illustrating divisions of movement trajectory data A. Fig. 14 shows the berth area A9a and warehouse area A9b described in Fig. 9. Fig. 14 shows a movement trajectory A14a of the moving object 2 indicated by a solid line and a movement trajectory A14b of the moving object 2 indicated by a dotted line.

[0113] In FIG. 14, the moving object 2 puts down the luggage it is holding at point L and moves to point M. The moving object 2 holds the luggage at point M and moves to point N. The numbers in parentheses L, M, and N in FIG. 14 indicate the state of the items held by the moving object 2. Note that point L in FIG. 14 corresponds, for example, to the point indicated by arrow A7e in the work image shown in FIG. 7. Point M in FIG. 14 corresponds, for example, to the point indicated by arrow A7f in the work image shown in FIG. 7. Point N in FIG. 14 corresponds, for example, to the point indicated by arrow A7g in the work image shown in FIG. 7.

[0114] As described above, the moving object 2 puts down the luggage it was carrying at point L and moves to point M. The moving object 2 carries the luggage at point M and moves to point N. Therefore, the A movement trajectory data for the movement trajectory A14a shown by the solid line includes a holding state of "0". The A movement trajectory data for the movement trajectory A14b shown by the dotted line includes a holding state of "1".

[0115] The task classification device 11 separates the chronologically arranged A movement trajectory data based on the retention state. In the example of Fig. 14, the task classification device 11 separates the chronologically arranged A movement trajectory data from point L to point N in Fig. 14 into A movement trajectory data from point L to point M and A movement trajectory data from point M to point N.

[0116] When the task classification device 11 divides the A movement trajectory data in a held state, it refers to the moving object area information in the divided A movement trajectory data, and acquires (extracts) the movement of the moving object 2 between areas (area transition).

[0117] In the example of FIG. 14, when moving from point L to point M, the mobile object 2 moves from the warehouse area (B) to the berth area (A). Therefore, the task classification device 11 extracts an area transition from B to A. Also, when moving from point M to point N, the mobile object 2 moves from the berth area (A) to the warehouse area (B). Therefore, the task classification device 11 extracts an area transition from B to A.

[0118] When the task classification device 11 extracts an area transition, it refers to the task unit pattern information shown in Figure 6 based on the extracted area transition and the retention state of the A movement trajectory data separated by the retention state, and obtains a task unit.

[0119] In the example of FIG. 14, the task classification device 11 acquires "movement for warehousing" as a task unit from point L to point M (see also "movement for warehousing" shown as a task unit in FIG. 7). The task classification device 11 acquires "warehousing" as a task unit from point M to point N (see also "warehousing_2" shown as a task unit in FIG. 7).

[0120] When the task classification device 11 acquires the task units of the moving object 2, it generates a task unit list of the moving object 2.

[0121] Fig. 15 is a diagram showing an example of the data configuration of a task unit list. Elements in Fig. 15 that are different from those in Fig. 12 will be described.

[0122] The work unit ID is an identifier for the work unit.

[0123] The start area name indicates the area where the moving object 2 started working on the task unit, and the end area name indicates the area where the moving object 2 finished working on the task unit.

[0124] The task unit name indicates the task name of the task unit executed by the moving body 2.

[0125] The task start date and time indicates the start date and time of the task unit. The task end date and time indicates the end date and time of the task unit.

[0126] The task time indicates the time required for the task unit executed by the moving object 2.

[0127] One task unit list is generated for each segment of movement trajectory data A. For example, in the example of Fig. 14, one task unit list is generated for movement trajectory data A from point L to point M. One task unit list is generated for movement trajectory data A from point M to point N.

[0128] In addition, the area name at the start of the task unit list from point L to point M in Figure 14 will be "B". The area name at the end will be "A". The task unit name will be "movement for inventory". The area name at the start of the task unit list from point M to point N in Figure 14 will be "A". The area name at the end will be "B". The task unit name will be "entering".

[0129] <Work analysis> The task classification device 11 visualizes the tasks of the mobile object 2 in response to a request from the terminal 13. The task classification device 11 visualizes the tasks of the mobile object 2 based on the generated task unit list and displays it on the display of the terminal 13. Once the task unit list is obtained, various task analyses such as those described below become possible.

[0130] Work experience 16 is a diagram showing an example of a screen for work performance. Screen A16a displays the work performance for one day for truck ID "1234" (worker ID "4321"). Works A, B, ..., H shown at the top of screen A16a indicate work units that can be performed by the moving object 2.

[0131] The horizontal axis displayed on the screen A16a indicates task units. The task units on the horizontal axis are arranged in chronological order. For example, in the example of FIG. 16, task A on the left side indicates that task B, task A, task C, and so on were performed in that order.

[0132] The vertical axis displayed on screen A16a indicates the time (seconds) required for each task. For example, it can be seen that "Task A" on the far left took approximately 20 seconds. It can also be seen that "Task B," which was performed after "Task A" on the far left, took approximately 35 seconds.

[0133] When the cursor is placed over the bar graph of a task unit displayed on screen A16a, a pop-up showing the details of the task unit is displayed. For example, when cursor A16c is placed over the bar graph of "Task A" indicated by arrow A16b in Fig. 16, a pop-up A16d showing the start and end times of "Task A" is displayed. The task classification device 11 can obtain the start and end times of "Task A" from the task start date and time and task end date and time included in the task unit list.

[0134] Clicking on the bar graph of a task unit displayed on screen A16a displays a video of the task unit. For example, if cursor A16c is placed over the bar graph of "Task A" indicated by arrow A16b in FIG. 16 and clicked, a video of "Task A" is displayed. Note that the A movement trajectory data used to create the task unit list includes a frame number (see FIG. 13). Task classification device 11 can display a video of the task of the task unit on the display of terminal 13 by requesting the video storage device 12 for the video corresponding to the frame number.

[0135] Overall analysis (KPI: Key Performance Indicator) 17A is a diagram showing an example of a screen in the warehousing task cycle. A screen A17a shown in FIG. 17A displays the warehousing task cycle for the cart ID "1234" (worker ID "4321").

[0136] The horizontal axis displayed on screen A17a indicates the date. The vertical axis indicates the total time (seconds) for inventory work on each date. In the example of screen A17a, it can be seen that the time required for inventory work per day becomes shorter as the days pass. The date of the inventory task cycle displayed on screen A17a can be specified from terminal 13. The task classification device 11 can obtain information on the inventory task cycle from the task unit name "inventory" included in the task unit list and the task time.

[0137] 17B is a diagram showing an example of a screen in the task cycle of delivery. A screen A17b shown in FIG. 17B displays the task cycle of delivery for the cart ID "1234" (worker ID "4321").

[0138] The horizontal axis displayed on screen A17b indicates the date. The vertical axis indicates the total time (seconds) for shipping work on each date. In the example of screen A17b, it can be seen that the time required for shipping work per day becomes shorter as the days pass. The date of the shipping task cycle displayed on screen A17b can be specified from terminal 13. The task classification device 11 can obtain information on the shipping task cycle from the task unit name "shipping" included in the task unit list and the work time.

[0139] Partial analysis (time period analysis) Fig. 18 is a diagram showing an example of a screen in partial analysis. A screen A18a shown in Fig. 18 displays the work ratio by time of day for all carts (all workers).

[0140] The horizontal axis displayed on screen A18a indicates time period. The vertical axis indicates time (seconds). In the example of screen A18a, the proportions of entering, leaving, entering and leaving, other, and breaks for each time period on February 6th are shown. The date for the hourly work proportions displayed on screen A18a can be specified from terminal 13. The work classification device 11 can obtain information on hourly work proportions from the work unit list for all carts for the date specified by terminal 13.

[0141] ·Partial analysis (daily analysis) Fig. 19 is a diagram showing an example of a screen in partial analysis. A screen A19a shown in Fig. 19 displays the work ratio by time period for each day for all carts (all workers).

[0142] The horizontal axis displayed on screen A19a indicates the date. The vertical axis indicates the time (seconds). In the example of screen A19a, the proportions of incoming, outgoing, incoming and outgoing, other, and breaks are shown for each day from February 6th to February 10th. The date for the daily work proportions displayed on screen A19a can be specified from terminal 13. The work classification device 11 can obtain information on the daily work proportions from the list of work units for all carts for the date specified by terminal 13.

[0143] <Operation flow> Positioning device operation flow Fig. 20 is a flowchart showing an example of the operation of the positioning device 1. The positioning device 1 executes the process of the flowchart shown in Fig. 20 at regular intervals, for example.

[0144] The positioning device 1 determines the position of the moving object 2 by hybrid positioning (S1).

[0145] The positioning device 1 acquires the state in which the moving object 2 is being held based on the image data of the camera 3 (S2).

[0146] The positioning device 1 generates movement trajectory data including the positioning result of S1 and the holding state acquired in S2 (S3). For example, the positioning device 1 generates the movement trajectory data shown in FIG.

[0147] The positioning device 1 transmits the movement trajectory data generated in S3 to the video storage device 12 (S4).

[0148] The video storage device 12 transmits the movement trajectory data transmitted from the positioning device 1 to the task classification device 11.

[0149] Operation flow of the work classification device Fig. 21 is a flowchart showing an example of the operation of generating a task unit list by the task classification device 11. The task classification device 11 executes the processing of the flowchart shown in Fig. 21, for example, at a specific time of day or in response to a request from the terminal 13. It is assumed that the task classification device 11 receives movement trajectory data from the video storage device 12 and stores it in a storage unit.

[0150] The task classification device 11 refers to the area information based on the position coordinates included in the movement trajectory data, and acquires the moving object area information of the moving object 2 (S11). For example, the task classification device 11 refers to the area information shown in Fig. 10 based on the position coordinates X, Y included in the movement trajectory data shown in Fig. 12, and acquires the moving object area information of the moving object 2.

[0151] The task classification device 11 generates movement trajectory data A by including the moving object area information of the moving object 2 acquired in S11 in the movement trajectory data (S12).

[0152] The task classification device 11 divides the A movement trajectory data arranged in time series, generated in S12, based on the holding state included in the A movement trajectory data (S13).

[0153] The task classification device 11 refers to the moving object area information in the series of A movement trajectory data divided based on the holding state, and acquires the movement (area transition) between areas of the moving object 2 (S14).

[0154] The task classification device 11 references the task unit pattern information shown in FIG. 6 based on the area transition acquired in S14 and the retention state of the A movement trajectory data divided by the retention state, and acquires task units (S15).

[0155] The task classification device 11 generates a task unit list based on the task units acquired in S15 and the A movement trajectory data separated in S13 (S16). For example, the task classification device 11 generates the task unit list shown in FIG.

[0156] 22 is a flowchart showing an example of the operation of task analysis by the task classification device 11. The task classification device 11 receives a task analysis request from the terminal 13 (S21).

[0157] In response to the task analysis request received in S21, the task classification device 11 processes the information in the task unit list and generates task analysis image data (S22). For example, the task classification device 11 generates task analysis image data for displaying the screens shown in Figures 16 to 19.

[0158] The task classification device 11 transmits the task analysis image data generated in S22 to the terminal 13 (S23).

[0159] <Function block> Positioning device functional block Fig. 23 is a diagram showing an example of functional blocks of the positioning device 1. As shown in Fig. 23, the positioning device 1 has a control unit 21, an IR (Infrared Rays) camera data receiving unit 22, a WEB camera video receiving unit 23, a positioning unit 24, a storage and analysis unit 25, a video compression unit 26, and an upload unit 27.

[0160] The control unit 21 controls each unit of the positioning device 1. The control unit 21 controls the hybrid positioning of the positioning device 1 and the acquisition of information on the holding state of an object in the mobile body 2. For example, the control unit 21 controls the hybrid positioning by referring to a setting file that has information such as map information of the area A2a in which the mobile body 2 moves. For example, the control unit 21 controls the acquisition of information on the holding state of an object by referring to a model file used for image analysis of the holding state of an object.

[0161] The IR camera data receiving unit 22 receives data acquired from the IR camera 3a of the camera 3. The WEB camera video receiving unit 23 receives video data output from the WEB camera 3b of the camera 3.

[0162] The positioning unit 24 performs hybrid positioning based on the image data of the IR camera 3a received by the IR camera data receiving unit 22, and measures the position of the positioning device 1 (moving object 2).

[0163] The holding analysis unit 25 acquires information on the state of holding an object by the moving object 2 based on the video data of the web camera 3b received by the web camera video receiving unit 23.

[0164] The video compression unit 26 compresses the video data of the web camera 3b received by the web camera video receiving unit 23.

[0165] Control unit 21 generates movement trajectory data by adding information on the state of holding objects by moving object 2 to the positioning result of positioning device 1 measured by positioning unit 24. Upload unit 27 uploads the movement trajectory data and compressed video data (compressed images) to video storage device 12 via a wireless network. Note that the frame numbers of the video data are linked to the frame numbers included in the movement trajectory data.

[0166] Video storage function block Fig. 24 is a diagram showing an example of functional blocks of the video storage device 12. As shown in Fig. 24, the video storage device 12 has a control unit 31, a receiving unit 32, an API (Application Programming Interface) 33, a movement trajectory data storage unit 34, a storage 35, a screen unit 36, and a cart master storage unit 37.

[0167] The control unit 31 controls each unit of the video storage device 12. For example, the control unit 31 transmits the movement trajectory data stored in the movement trajectory data storage unit 34 to the task classification device 11.

[0168] The receiving unit 32 receives the movement trajectory data and the compressed image uploaded from the positioning device 1. The API 33 stores the movement trajectory data received by the receiving unit 32 in the movement trajectory data storage unit 34. The API 33 stores the compressed image received by the receiving unit 32 in the storage 35.

[0169] In response to the redirect from the task classification device 11 , the screen unit 36 ​​accesses the movement trajectory data storage unit 34 and the storage 35 , acquires a predetermined compressed image, and transmits it to the terminal 13 .

[0170] The vehicle master storage unit 37 stores information such as base information of a base where the mobile object 2 works, such as a logistics warehouse, the vehicle ID, the vehicle name, and the camera ID of the camera 3 mounted on the mobile object 2.

[0171] ·Work classification device functional block Fig. 25 is a diagram showing an example of functional blocks of the task classification device 11. As shown in Fig. 25, the task classification device 11 has a visualization master storage unit 41, a control unit 42, a movement trajectory data storage unit 43, an A movement trajectory data storage unit 44, a task unit list storage unit 45, a base information master storage unit 46, and a screen unit 47.

[0172] The visualization master storage unit 41 stores task unit pattern information (see FIG. 6) and area information (see FIG. 10).

[0173] The control unit 42 controls each unit of the task classification device 11. The control unit 42 generates A movement trajectory data by referring to the visualization master storage unit 41 and the movement trajectory data storage unit 43. The control unit 42 stores the generated A movement trajectory data in the A movement trajectory data storage unit 44.

[0174] The control unit 42 generates a task unit list by referring to the visualization master storage unit 41 and the A movement trajectory data storage unit 44. The control unit 42 stores the generated task unit list in the task unit list storage unit 45.

[0175] The control unit 42 performs a task analysis by referring to the task unit list storage unit 45 in response to a request from the terminal 13. The control unit 42 transmits image data of the task analysis results to the terminal 13 via the screen unit 47.

[0176] The base information master storage unit 46 stores base information of bases where the mobile objects 2 work, such as logistics warehouses, and user information of users who use the terminals 13 to perform work analysis.

[0177] <Hardware> 26 is a diagram showing an example of hardware of the information processing device 50. The positioning device 1, the task classification device 11, and the video storage device 12 may have the hardware configuration shown in FIG. As shown in FIG. 26, the information processing device 50 includes a processor 51, a storage unit 52, an input unit 53, an output unit 54, and a communication unit 55.

[0178] The processor 51 controls the entire information processing device 50. The processor 51 may be configured by, for example, a CPU (Central Processing Unit).

[0179] The storage unit 52 stores a program for operating the processor 51. The storage unit 52 also stores data for the processor 51 to perform calculation processing, data for the processor 51 to control each unit, etc. The storage unit 52 may be configured by a storage device such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, or an HDD (Hard Disk Drive).

[0180] The input unit 53 receives signals from an input device such as a keyboard or a touch panel, and outputs the signals to the processor 51 .

[0181] The output unit 54 outputs a signal from the processor 51 to an output device such as a display.

[0182] The communication unit 55 communicates with other devices via short-range wireless communication such as Wi-Fi or Bluetooth (registered trademark), or via a wireless communication network of 3GPP (registered trademark). The communication unit 55 communicates with other devices via a network cable such as an Ethernet cable.

[0183] 23 may be realized by a processor 51. The functions of the IR camera data receiving unit 22, the WEB camera video receiving unit 23, and the uploading unit 27 may be realized by a communication unit 55.

[0184] 24 may be realized by a processor 51. The functions of the movement trajectory data storage unit 34, the storage 35, and the cart master storage unit 37 may be realized by a storage unit 52. The functions of the receiving unit 32 and the screen unit 36 ​​may be realized by a communication unit 55.

[0185] 25 may be realized by a processor 51. The functions of the visualization master storage unit 41, the movement trajectory data storage unit 43, the A movement trajectory data storage unit 44, and the task unit list storage unit 45 may be realized by a storage unit 52. The function of the screen unit 47 may be realized by a communication unit 55.

[0186] <Summary of the embodiment> As described above, the task classification device 11 receives the positioning results (position information) from the mobile object 2 and information related to the object holding state of the mobile object 2, and acquires the movement of the mobile object 2 between areas based on the positioning results. The task classification device 11 acquires task units of the mobile object 2 based on the received holding state and the acquired movement of the mobile object 2 between areas. This allows the task classification device 11 to classify tasks performed by the mobile object 2 by task unit, using the positioning results of the mobile object 2. In other words, the positioning results of the mobile object 2 are effectively used by the task classification device 11.

[0187] <Variation 1> Although the positioning device 1 measures the position of the moving object 2 by hybrid positioning, the present invention is not limited to this. The positioning device 1 may measure the position of the moving object 2 by marker positioning, feature point positioning, or other positioning methods.

[0188] <Variation 2> Although the positioning device 1 transmits the positioning results of the moving object 2 and the image from the camera 3 to the image storage device 12, this is not limited to this. The positioning device 1 may also transmit the positioning results of the moving object 2 and the image from the camera 3 to the task classification device 11. In this case, the task classification system omits the image storage device 12.

[0189] <Variation 3> Although the positioning device 1 acquires the object holding state of the moving object 2 based on the image data of the camera 3, this is not limited to this. The task classification device 11 or the video storage device 12 may acquire the object holding state of the moving object 2 based on the image data of the camera 3.

[0190] <Variation 4> Although the task classification device 11 displays the task analysis results of the mobile object 2 on the display of the terminal 13, this is not limitative. The task classification device 11 may also display the task analysis results of the mobile object 2 on a display provided in the task classification device 11.

[0191] <Variation 5> The work analysis device 11 may display the movement trajectory of the moving object 2 on the display of the terminal 13 based on the position coordinates included in the movement trajectory data or the A movement trajectory data. The work analysis device 11 may evaluate whether the moving object 2 is moving along the shortest route based on the movement trajectory of the moving object 2. The work analysis device 11 may evaluate safety based on the movement trajectory of the moving object 2 to determine whether the moving object 2 is moving along a safe route.

[0192] <Variation 6> The operation system may be applied to applications other than logistics warehouses. For example, the operation system may be applied to a system in a factory that delivers parts from a material storage area to each process on a production line.

[0193] Although the embodiments have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims. It is understood that such modifications or alterations also fall within the technical scope of the present disclosure. Furthermore, the components in the embodiments may be combined in any manner without departing from the spirit of the present disclosure.

[0194] In the above-described embodiments, the notation "... part" used for each component may be replaced with other notations such as "... circuitry," "... assembly," "... device," "... unit," or "... module."

[0195] The present disclosure can be realized by software, hardware, or software linked to hardware. Each functional block used in the description of the above embodiments may be partially or entirely realized as an LSI, which is an integrated circuit, and each process described in the above embodiments may be partially or entirely controlled by a single LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of a single chip that includes some or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may be called an IC, system LSI, super LSI, or ultra LSI.

[0196] The integrated circuit method is not limited to LSI, but may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI, may be used. The present disclosure may be realized as digital processing or analog processing.

[0197] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that such technology may be used to integrate functional blocks. The application of biotechnology, etc. is also a possibility. [Industrial Applicability]

[0198] The present disclosure provides: [Explanation of symbols]

[0199] 1. Positioning device 2. Mobile 3 Camera 11 Work classification device 12 Video storage device 13 Terminals

Claims

1. A task classification device comprising a processor and a communication unit, The communication unit receiving location information from a mobile object and information regarding a holding state of an object by the mobile object; The processor: acquiring movement of the mobile object between areas based on the location information; classifying the work of the mobile object into work units based on the holding state and the movement of the mobile object between areas; Work classification device.

2. the processor refers to area information that defines an area range based on the location information to obtain the area in which the mobile object is located, and obtains movement of the mobile object between areas; The task classification device according to claim 1 .

3. The processor refers to a task unit of the mobile body defined based on the holding state of the object in the mobile body and the movement of the mobile body between areas, and acquires a task unit corresponding to the received holding state and the acquired movement of the mobile body between areas. The task classification device according to claim 1 .

4. The processor generates a work unit list including the work units and work times of the work units. The task classification device according to claim 1 .

5. The processor analyzes the work of the mobile object based on the work unit list and outputs the analysis result to a display of a terminal. The task classification device according to claim 4 .

6. The processor generates graph image data of the task time for each task unit of the mobile object based on the task unit list, and displays the graph image data on a display of a terminal. The task classification device according to claim 4 .

7. When a unit of work is specified on the display of the terminal, the processor displays the start time and end time of the specified unit of work on the display of the terminal. The task classification device according to claim 6 .

8. When a task unit is specified on the display of the terminal, the processor displays an image related to the specified task unit and taken by a camera mounted on the moving body on the display of the terminal. The task classification device according to claim 6 .

9. The holding state indicates whether the object is being held or not held by the moving body. The task classification device according to claim 1 .

10. A task classification method for a task classification device, comprising: receiving location information from a mobile object and information regarding a holding state of an object by the mobile object; acquiring movement of the mobile object between areas based on the location information; acquiring a task unit of the mobile object based on the holding state and movement of the mobile object between areas; Work classification method.

11. To the work classification device, receiving location information from a mobile object and information regarding a holding state of an object by the mobile object; acquiring movement of the mobile object between areas based on the location information; acquiring a task unit of the mobile object based on the holding state and movement of the mobile object between areas; A program that executes a process.

Images