Information output system, information output method, and information output program

Abstract

To provide an information output system, an information output method and an information output program, capable of performing suitable presentation information.SOLUTION: An information output system 1 according to a present embodiment comprises: image acquiring means 10 that acquires an image of an operation space 100 including an AGV50, a plurality of components 60 and an operator 70; determining means 20 that determines the component 60 to be loaded on the AGV50 based on moving-body information, etc. showing the AGV50 in the acquired image; and image projecting means 30 that projects a projection image including a reference projecting image, etc. showing the component 60 determined to be loaded on the AGV50. The determining means 20 generates a layout to change a position of the component 60 from an analysis of a moving line 80 for the operator 70 to move in the operation space 100, and the image projecting means 30 projects a projection image showing a position of the component 60 changed in the layout.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information output system, an information output method, and an information output program.

Background Art

[0002] Patent Document 1 discloses a technique for outputting information related to work, including how to work and the order of work, to a worker by performing irradiation display in the work environment.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The technique of Patent Document 1 is assumed for work on a desktop. Therefore, it is difficult to output information between works in a more complex work space where a plurality of dynamic objects may exist by irradiation display. Even if the work in a complex work space is extracted by machine learning or the like, the working ability of the worker varies depending on the worker, so appropriate information between works is not always output. There is a desire for an information output system that can provide appropriate information presentation between works in a complex work space.

[0005] The present disclosure has been made to solve such problems, and provides an information output system, an information output method, and an information output program capable of performing appropriate information presentation.

Means for Solving the Problems

[0006] The information output system according to this embodiment includes: an image acquisition means for acquiring an image of a workspace including a mobile body, a plurality of parts, and at least one worker; a determination means for determining which parts to be mounted on the mobile body based on at least one of first mobile body information showing the mobile body in the acquired image, second mobile body information showing the time the mobile body appears in the workspace, and third mobile body information showing the position of the mobile body until it appears in the workspace; and an image projection means for projecting a projection image including at least one of a first reference projection image showing the parts determined to be mounted on the mobile body, a second reference projection image showing which of the mobile bodies appearing in the workspace the parts will be mounted on, and a third reference projection image showing the storage position of the mobile body. The determination means generates a layout that changes the position of the parts in the workspace from an analysis of the worker's movement path in the workspace, and the image projection means projects the projection image showing the changed position of the parts in the layout.

[0007] In the above-described information output system, the determination means determines the mounting order of the plurality of components, and the image projection means projects the projection image showing the mounting order of the plurality of components.

[0008] In the above information output system, the image acquisition means acquires multiple movement paths of multiple different workers as they move through the workspace when the component is mounted on the moving body, and the determination means generates a layout in which the component is arranged in such a way that the differences in the movement paths of the multiple different workers are short.

[0009] In the above-described information output system, the determination means generates the layout with the position of the third part changed if the difference in the movement paths of the multiple different workers from the second part to the third part is greater than the difference in the movement paths of the multiple different workers from the first part to the second part.

[0010] The information output method according to this embodiment acquires an image of a workspace including a mobile body, a plurality of parts, and at least one worker; determines which parts to be mounted on the mobile body based on at least one of the following: first mobile body information showing the mobile body in the acquired image, second mobile body information showing the time the mobile body appears in the workspace, and third mobile body information showing the position of the mobile body until it appears in the workspace; projects a projection image including at least one of the following: a first reference projection image showing the parts determined to be mounted on the mobile body, a second reference projection image showing which of the mobile bodies appearing in the workspace the parts will be mounted on, and a third reference projection image showing the storage position of the mobile body; generates a layout that changes the position of the parts in the workspace from an analysis of the worker's movement path in the workspace; and projects the projection image showing the changed position of the parts in the layout.

[0011] The information output program according to this embodiment causes an image acquisition means to acquire an image of a workspace including a mobile body, a plurality of parts, and at least one worker, determines which parts to be mounted on the mobile body based on at least one of the following: first mobile body information showing the mobile body in the acquired image, second mobile body information showing the time when the mobile body appears in the workspace, and third mobile body information showing the position of the mobile body until it appears in the workspace, projects a projection image onto the image projection means that includes at least one of the following: a first reference projection image showing the parts determined to be mounted on the mobile body, a second reference projection image showing which of the mobile bodies appearing in the workspace the parts will be mounted on, and a third reference projection image showing the storage position of the mobile body, generates a layout that changes the position of the parts in the workspace from an analysis of the worker's movement path in the workspace, and projects the projection image showing the changed position of the parts in the layout onto the image projection means. [Effects of the Invention]

[0012] According to this embodiment, it is possible to provide an information output system, an information output method, and an information output program that can present appropriate information. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic diagram illustrating an information output system according to Embodiment 1. [Figure 2] This figure illustrates the information stored by the decision-making means in the information output system according to Embodiment 1. [Figure 3] This is a block diagram illustrating the decision-making means in the information output system according to Embodiment 1. [Figure 4] This figure illustrates an image acquired by the recognition unit in the information output system according to Embodiment 1. [Figure 5] This figure illustrates an image determined by the recognition determination unit in the information output system according to Embodiment 1. [Figure 6] This figure illustrates a recognized image generated by the recognized image generation unit in the information output system according to Embodiment 1. [Figure 7] This figure illustrates a projected image generated by the projected image generation unit in the information output system according to Embodiment 1. [Figure 8] This figure illustrates a projected image projected by the image projection means in the information output system according to Embodiment 1. [Figure 9] This is a flowchart illustrating an example of the information output method according to Embodiment 1. [Figure 10] This flowchart illustrates a method for determining which components to be mounted on an AGV in the information output method according to Embodiment 1. [Figure 11] This is a schematic diagram illustrating an information output system according to Embodiment 2. [Figure 12] This is a flowchart illustrating an example of the information output method according to Embodiment 2. [Figure 13] This flowchart illustrates an example of how to store information in the information output method according to Embodiment 2. [Figure 14] This is a diagram illustrating the information stored in the storage unit in the information output system 2 according to Embodiment 2. [Figure 15] This is a flowchart diagram illustrating the method for generating a layout in the information output method according to Embodiment 2. [Figure 16] This is a diagram illustrating the layout generated by the recognition determination unit in the information processing method according to Embodiment 2. [Figure 17] This is a diagram illustrating the projection image generated by the projection image generation unit in the information output method according to Embodiment 2. [Figure 18] This is a diagram illustrating the projection image projected by the image projection means in the information output method according to Embodiment 2.

Mode for Carrying Out the Invention

[0014] Hereinafter, the present disclosure will be described through embodiments, but the present disclosure is not limited to the following embodiments. Also, not all of the configurations described in the embodiments are essential as means for solving the problems. For clarity of explanation, the following description and drawings have been appropriately omitted and simplified. In each drawing, the same reference numerals are assigned to the same elements, and duplicate explanations are omitted as necessary.

[0015] (Embodiment 1) The information output system according to Embodiment 1 will be described. In this embodiment, the information output system may be replaced with an information output device, or the information output device may be replaced with an information output system. Also, the information output system of this embodiment may include an information output device.

[0016] <Configuration of the Information Output System> Figure 1 is a schematic diagram illustrating an information output system according to Embodiment 1. As shown in Figure 1, the information output system 1 is located in a predetermined workspace 100. The workspace 100 includes a mobile body, a plurality of parts 60, and at least one worker 70. The mobile body includes, for example, an autonomous guided vehicle (AGV50). In the following description, the mobile body will be described as AGV50. In this case, the workspace 100 includes AGV50, a plurality of parts 60, and at least one worker 70. Note that the mobile body is not limited to a wheeled transport vehicle that moves autonomously, such as AGV50, but may also include mobile bodies that can move using methods other than wheeled drive, such as hovercraft or linear motor systems. Furthermore, the mobile body may include not only cases where the vehicle itself (the computing device inside the vehicle) performs autonomous movement control, but also cases where an external server transmits instructions on how to move autonomously to the vehicle by sending control signals, etc., or where the vehicle's movement control is performed by remote operation by a person. The workspace 100 is, for example, a factory where a production line for assembling parts 60 is installed. However, the workspace 100 is not limited to a factory where a parts assembly line is installed; it may also be a parts distribution center, parts warehouse, etc., as long as it includes an AGV 50, multiple parts 60, and at least one worker 70.

[0017] Here, for the sake of explaining the information output system 1, we introduce the XYZ Cartesian coordinate axis. The floor surface of the workspace 100 is defined as the XY plane, and the direction perpendicular to the floor surface is defined as the Z axis.

[0018] The information output system 1 comprises a position acquisition means, a determination means 20, and an image projection means 30. The position acquisition means includes, for example, an image acquisition means 10. In the following description, the position acquisition means will be described as the image acquisition means 10. In this case, the information output system 1 comprises the image acquisition means 10, the determination means 20, and the image projection means 30. Note that the acquisition of position information by the position acquisition means is not limited to the acquisition of position information by the image acquisition means 10, but may also be performed by acquiring position information from sensors mounted on the AGV 50, or by acquiring position information from other environmental sensors, etc. The image acquisition means 10, the determination means 20, and the image projection means 30 of the information output system 1 are interconnected by communication lines such as wired and wireless lines. The communication lines may be partially wired and partially wireless. The information output system 1 may integrate at least one of the image acquisition means 10, the determination means 20, and the image projection means 30, or each may be a separate unit.

[0019] The image acquisition means 10 acquires images of the workspace 100, which includes the AGV 50, multiple parts 60, and at least one worker 70. The images are, for example, videos. However, the images may also be still images. The image acquisition means 10 acquires images of the workspace 100 in real time. The image acquisition means 10 is, for example, a video camera. A PoE (Power over Ethernet) type camera is preferred for the image acquisition means 10. However, the image acquisition means 10 is not limited to a video camera; it may also be a still camera, radar, infrared sensor, etc., as long as it can acquire images of the workspace 100.

[0020] The image acquisition means 10 is, for example, positioned on the ceiling of the workspace 100 and fixed facing downwards. This eliminates blind spots and allows for the acquisition of images of the parts 60. The image acquisition means 10 is not limited to the ceiling of the workspace 100; it may be positioned on the sides of the workspace 100 or elsewhere, or even outside the workspace 100, as long as it can acquire images of the workspace 100.

[0021] The determination means 20 determines the components 60 to be mounted on the AGV 50 based on the AGV 50 in the acquired image. The determination means 20 is, for example, an information processing device such as a PC (Personal Computer). A PC with a GPU (Graphics Processing Unit) is preferred for the determination means 20. However, the determination means 20 is not limited to an information processing device such as a PC, but may also be a portable device such as a smartphone or tablet, as long as it can determine the components 60 to be mounted on the AGV 50 based on the AGV 50 in the acquired image.

[0022] The determination means 20 may pre-associate and store the AGV 50 and the components 60 mounted on the AGV 50. Figure 2 is a diagram illustrating the information stored by the determination means 20 in the information output system 1 according to Embodiment 1.

[0023] As shown in Figure 2, the determination means 20 stores the association between AGVs 51 to AGV 53 and multiple parts 61 to 64. Specifically, the determination means 20 stores the association between AGV 51 and parts 61, 62, and 63. The determination means 20 also stores that the mounting order of the multiple parts 60 to be mounted on AGV 51 is part 61, part 62, and part 63.

[0024] If the determination means 20 determines that the image acquired by the image acquisition means 10 includes the AGV 51, it determines that the components 60 to be mounted on the AGV 51 are components 61, 62, and 63. The determination means 20 may also determine that the mounting order of the multiple components 60 to be mounted on the AGV 51 is component 61, component 62, and component 63.

[0025] The image projection means 30 projects a projection image showing the component 60 that has been determined to be mounted on the AGV 50. For example, the image projection means 30 projects a projection image indicating that the component 60 has been determined to be mounted on the AGV 50 will be mounted. The image projection means 30 is, for example, a projector. However, the image projection means 30 is not limited to a projector; it may also be an information processing device or portable device such as a PC, smartphone, or tablet, as long as it has the function of projecting a projection image indicating that the component 60 has been determined to be mounted on the AGV 50 will be mounted. Furthermore, the image projection means 30 may include an output unit that outputs signals for lighting control, such as where and how to illuminate a lighting device. In this case, the lighting device may be an external product such as an IoT (Internet of Things) connected to the internet, such as an IoT lamp. The output unit may be connected to such a lighting device to project the projection image onto the lighting device. Therefore, the image projection means 30 includes an output unit, which outputs a signal for lighting control to an external product, a lighting device, causing the lighting device to project an image showing the component 60 that has been determined to be mounted on the AGV 50.

[0026] The image projection means 30 is, for example, positioned on the ceiling of the workspace 100 and fixed facing downwards. This eliminates blind spots and allows images to be projected onto the parts 60. The image projection means 30 is not limited to the ceiling of the workspace 100; it may be positioned on the sides of the workspace 100 or elsewhere, or even outside the workspace 100, as long as it can project an image onto the parts 60. The image projection means 30 may be positioned in the same location as the image acquisition means 10. It is preferable that the image projection means 30 projects the image with light of the 10,000 lumen class.

[0027] The image projection means 30 may project a projection image indicating the mounting order of the multiple components 60. Specifically, the image projection means 30 may project a projection image indicating the mounting order of the multiple components 60. For example, the image projection means 30 may project projection images of the numbers "1" on component 61, "2" on component 62, and "3" on component 63 to indicate that the mounting order is component 61, component 62, and component 63.

[0028] In this way, the information output system 1 recognizes the AGV 50, multiple parts 60, and worker 70 in a workspace 100 such as a production line, and gives instructions for loading the parts 60 by projecting projected images such as projection mapping. The worker 70 moves around the workspace 100 and loads the parts 60 onto the AGV 50. The path that the worker 70 takes to move around the workspace 100 when loading the parts 60 onto the AGV 50 is called the movement path 80.

[0029] Figure 3 is a block diagram illustrating the determination means 20 in the information output system 1 according to Embodiment 1. As shown in Figure 3, the determination means 20 includes a recognition unit 21, a recognition determination unit 22, a recognition image generation unit 23, a projected image generation unit 24, and an integration unit 25. The determination means 20 may further include a storage unit 26 and a display unit 27. The recognition unit 21, recognition determination unit 22, recognition image generation unit 23, projected image generation unit 24, integration unit 25, storage unit 26, and display unit 27 each have the functions of a recognition means, a recognition determination means, a recognition image generation means, a display generation means, an integration means, a storage means, and a display means, respectively.

[0030] The recognition unit 21 acquires an image from the image acquisition means 10. Figure 4 is a diagram illustrating an image acquired by the recognition unit 21 in the information output system 1 according to Embodiment 1. As shown in Figure 4, the recognition unit 21 recognizes necessary objects and people within the range visible in the acquired image. Specifically, the recognition unit 21 recognizes the AGV 50, multiple parts 60, and worker 70, etc., that are visible in the acquired image. The recognition unit 21 outputs images of the recognized AGV 50, multiple parts 60, and worker 70 to the recognition judgment unit 22.

[0031] The recognition and determination unit 22 determines what objects and people the recognition unit 21 has recognized in the image input from the recognition unit 21. Specifically, the recognition and determination unit 22 determines whether the image input from the recognition unit 21 contains information about the AGV 50, multiple parts 60, and worker 70. For example, the recognition and determination unit 22 determines whether the image contains information about the AGV 50, multiple parts 60, and worker 70 based on the position, shape, color, and movement of the objects in the image. The recognition and determination unit 22 may also determine whether the image contains information about the AGV 50, multiple parts 60, and worker 70 based on tags such as QR codes (registered trademarks) attached to the objects in the image.

[0032] Figure 5 is an example of an image judged by the recognition judgment unit 22 in the information output system 1 according to Embodiment 1. As shown in Figure 5, if the recognition judgment unit 22 determines that the image input from the recognition unit 21 contains information about an AGV 50, multiple parts 60, and a worker 70, it determines whether the AGV 50, multiple parts 60, and worker 70 match the AGV 50, multiple parts 60, and worker 70 stored in the storage unit 26. The recognition judgment unit 22 may, for example, determine whether the AGV 50, multiple parts 60, and worker 70 stored in the storage unit 26 match based on the position, shape, color, movement, and tags of the objects shown in the image.

[0033] The recognition determination unit 22 determines, based on the matching AGV 50, whether the component 60 to be mounted on the AGV 50 is present in the image if the AGV 50 in the input image matches the AGV 50 stored in the storage unit 26. If the recognition determination unit 22 determines that the component 60 to be mounted on the AGV 50 is present in the image input from the recognition unit 21, it outputs information about the component 60 to the recognition image generation unit 23 and the projection image generation unit 24.

[0034] The recognition image generation unit 23 generates a recognition image in which labels are assigned to the AGV 50, multiple parts 60, and worker 70 output from the recognition judgment unit 22. Figure 6 is a diagram illustrating a recognition image generated by the recognition image generation unit 23 in the information output system 1 according to Embodiment 1. As shown in Figure 6, the recognition image generation unit 23 assigns the label "51" to the AGV 50 in the image recognized by the recognition unit 21, meaning AGV 51. The recognition image generation unit 23 assigns the labels "61", "62", "63", and "64" to the multiple parts 60 in the image recognized by the recognition unit 21, meaning parts 61, 62, 63, and 64. The recognition image generation unit 23 assigns the label "71" to the worker 70 in the image recognized by the recognition unit 21, meaning worker 71. In this way, the recognition image generation unit 23 generates a recognition image in which labels are assigned to the AGV 50, multiple parts 60, and worker 70. The recognition image generation unit 23 may display the generated recognition image on the display unit 27. The recognition image generation unit 23 outputs the generated recognition image to the integration unit 25.

[0035] The projection image generation unit 24 generates a projection image to be projected onto the component 60 that the recognition determination unit 22 has determined to be mounted on the AGV 50. Figure 7 is an example of a projection image generated by the projection image generation unit 24 in the information output system 1 according to Embodiment 1. As shown in Figure 7, the projection image generation unit 24 generates a projection image to be projected onto the component 60 that it has determined to be mounted on the AGV 50, for example, a circular spot with the mounting order "1", "2", "3", etc. In Figure 7, the projection image is displayed superimposed on the recognition image in an integrated format. The projection image generation unit 24 outputs the generated projection image to the integration unit 25.

[0036] As shown in Figure 7, the integration unit 25 integrates the projection image generated by the projection image generation unit 24 with the recognition image generated by the recognition image generation unit 23. This allows the integration unit 25 to associate the projection image with the position of the recognized component 60 in the recognition image. Furthermore, the information output system 1 can display the projection image projected onto the component 60 mounted on the AGV 50 in the display unit 27. Additionally, the projection image generation unit 24 or the integration unit 25 outputs the projection image to the image projection means 30. Figure 8 is an example of a projection image projected by the image projection means 30 in the information output system 1 according to Embodiment 1. As shown in Figure 8, the image projection means 30 projects the projection image onto multiple component 60 mounted on the AGV 50.

[0037] Next, the information output method of this embodiment will be described. Figure 9 is a flowchart illustrating the information output method according to Embodiment 1. As shown in step S10 of Figure 9, an image is acquired. Specifically, for example, the image acquisition means 10 acquires an image of the workspace 100 including the AGV 50, a plurality of parts 60, and at least one worker 70. The determination means 20 causes the recognition unit 21 to acquire the image of the workspace 100 from the image acquisition means 10.

[0038] Next, as shown in step S20, the acquired image is judged. Specifically, the judgment means 20 causes the recognition judgment unit 22 to judge the components 60 to be mounted on the AGV 50 based on the AGV 50 in the image. For example, the judgment means 20 stores in the storage unit 26 in advance the association between the AGV 51 and the components 61 to 63 to be mounted on the AGV 51. Then, if the recognition judgment unit 22 determines that the acquired image contains the AGV 51, it determines that the components 60 to be mounted on the AGV 51 are components 61, 62, and 63.

[0039] The memory unit 26 may also store that the mounting order of the multiple components 60 mounted on the AGV 51 is component 61, component 62, and component 63. In that case, the recognition determination unit 22 may determine that the mounting order of the multiple components 60 mounted on the AGV 51 is component 61, component 62, and component 63.

[0040] Next, as shown in step S30, projection images indicating that a component 60 is to be mounted on the AGV 50 are projected onto the image projection means 30. Specifically, projection images are generated by the projection image generation unit 24. Then, for example, projection images generated by the projection image generation unit 24 are projected onto the image projection means 30 for components 61, 62, and 63 that are to be mounted on the AGV 51. For multiple components 60, projection images indicating the mounting order may be projected onto the image projection means 30.

[0041] Figure 10 is a flowchart illustrating a method for determining which parts 60 to be mounted on the AGV 50 in the information output method according to Embodiment 1. As shown in step S21 of Figure 10, the acquired image is recognized. Specifically, the recognition unit 21 recognizes the AGV 50, multiple parts 60, and worker 70, etc., that appear in the image acquired by the image acquisition means 10. If "NO" is not found in step S21, the process ends. If "YES" is found in step S21, the process proceeds to step S22.

[0042] Next, as shown in step S22, the recognized image is determined. Specifically, the recognition determination unit 22 determines whether the image input from the recognition unit 21 contains information about the AGV 50, multiple parts 60, and worker 70. If the recognition determination unit 22 determines that the image input from the recognition unit 21 contains information about the AGV 50, multiple parts 60, and worker 70, it determines whether the AGV 50, multiple parts 60, and worker 70 match the AGV 50, multiple parts 60, and worker 70 stored in the storage unit 26. If the AGV 50 in the input image matches the AGV 50 stored in the storage unit 26, the recognition determination unit 22 determines, based on the matched AGV 50, whether the parts 60 to be mounted on the AGV 50 are present in the image. If the recognition determination unit 22 determines "YES" that the image input from the recognition unit 21 contains the component 60 to be mounted on the AGV 50, it outputs information about the component 60 to the recognition image generation unit 23 and the projection image generation unit 24. Then, the process proceeds to steps S23 and S24.

[0043] On the other hand, if the image input from the recognition unit 21 does not contain information about the AGV 50, multiple parts 60, and worker 70 ("NO"), if the image input from the recognition unit 21 contains information about the AGV 50, multiple parts 60, and worker 70 but does not match the AGV 50, multiple parts 60, and worker 70 stored in the storage unit 26 ("NO"), or if the image input from the recognition unit 21 does not contain any parts 60 to be mounted on the AGV 50 ("NO"), the process is terminated.

[0044] Next, as shown in step S23, a recognition image is generated. Specifically, the recognition image generation unit 23 generates a recognition image in which labels are attached to the AGV 50, multiple parts 60, and worker 70 output from the recognition judgment unit 22.

[0045] Furthermore, as shown in step S24, a projection image is generated. Specifically, the projection image generation unit 24 generates a projection image to be projected onto the position of the component 60 that the recognition determination unit 22 has determined to be mounted on the AGV 50.

[0046] Next, as shown in step S25, the recognized image and the projected image are integrated. This links the projected image to the positions of the recognized parts 60, etc., in the recognized image. The integration unit 25 or the projected image generation unit 24 then outputs the projected image, which corresponds to the positions of the recognized parts 60, etc., in the recognized image, to the image projection means 30. In this way, the image projection means 30 can output information.

[0047] Next, the effects of this embodiment will be described. The information output system 1 of this embodiment acquires images of the workspace 100, including the AGV 50, multiple parts 60, and the worker 70, in real time using the image acquisition means 10. Therefore, even if the parts 60 to be mounted are changed or the layout of the parts 60 is changed, the position of the parts 60 to be mounted can be grasped in real time. As a result, a projection image can be appropriately projected onto the parts 60 to be mounted on the AGV 50. Therefore, the worker 70 can mount the appropriate parts 60 on the AGV 50. The worker 70 does not need to look at leaflets or other documents with instructions, which reduces work errors and shortens the training period for the work.

[0048] The information output system 1 of this embodiment can be installed in a single installation work, which involves installing an image acquisition means 10 such as a camera and an image projection means 30 such as a projector. This allows for changes in the mounted components 60 and changes in the layout of the components 60 without requiring new installation work. In contrast, the method of providing lamps on the component rack on which the components 60 are mounted to indicate the mounting order of the components 60 requires new installation work, such as installing lamps and wiring, when the mounted components 60 or the layout of the components 60 is changed. Thus, the information output system 1 of this embodiment can improve productivity by eliminating the need for new installation work.

[0049] In processes that include cell production, where fluctuations in the number of products assembled, changes in the number of workers, changes in product types, and the occurrence of defects can occur, the cycle time (process work time to ensure even timing of production processes) becomes longer than on a normal line. As a result, situations arise where it becomes difficult to know which task should be done next. In addition, depending on the operating status of the equipment (including AGVs, etc.) and the status of the cooperating workers, downtime may occur.

[0050] In this embodiment, the information output system 1 can display appropriate procedures for each worker 70 using an image projection means 30 such as a projector, based on the recognition results of an image acquisition means 10 such as a camera that provides an overview of the entire situation in the workspace 100 of such a production process, and a judgment means 20. Therefore, additional work such as pressing special buttons for the workers 70 can be reduced, and productivity can be improved.

[0051] (Embodiment 2) Next, the information output system according to Embodiment 2 will be described. Figure 11 is a schematic diagram illustrating the information output system according to Embodiment 2. As shown in Figure 11, the information output system 2 of this embodiment generates the layout of the parts 60 from the analysis of the worker's movement paths 81 and 82. The information output system 2 then outputs the information of the generated layout.

[0052] Figure 12 is a flowchart illustrating an information output method according to Embodiment 2. As shown in step S40 of Figure 12, in the information output system 2 of this embodiment, the image acquisition means 10 acquires an image of the worker's movement path 80 when the part 60 is loaded onto the AGV 50. The movement path 80 is the trajectory of the worker 70 moving within the workspace 100. The image acquisition means 10 may acquire images of multiple movement paths 80. The image acquisition means 10 may acquire an image of the movement path 81 of a specific worker 71 moving through the workspace 100. In addition, the image acquisition means 10 may acquire images of multiple movement paths 80 of multiple different workers 70 moving through the workspace 100 when the part 60 is loaded onto the AGV 50. As mentioned above, the location information means is not limited to the image acquisition means 10, but may also be obtained by acquiring location information from sensor information of a terminal carried by the worker 70, or by acquiring location information from another environmental sensor, etc. In that case, the movement of the worker 70 may be acquired not only by the image acquisition means 10, but also by sensors on a terminal carried by the worker 70, or by other environmental sensors. Furthermore, even if the information acquired in this way is not directly analyzed in the image, it may be linked to the acquired image to determine where in the image the information should be output.

[0053] Next, as shown in step S50 of Figure 12, the decision means 20 generates a layout in which the multiple parts 60 are arranged in the workspace 100. To generate the layout, the decision means 20 stores the movement paths 80 of the multiple workers 70. Then, the decision means 20 generates the layout of the multiple parts 60 from the analysis of the stored movement paths 80.

[0054] Figure 13 is a flowchart illustrating an example of an information storage method in the information output method according to Embodiment 2. As shown in step S51 of Figure 13, the recognition unit 21 recognizes the AGV 50, multiple parts 60, worker 70, and movement path 80, etc., that appear in the image acquired by the image acquisition means 10. If "NO" is true in step S51 and the acquired image does not recognize the AGV 50, multiple parts 60, worker 70, and movement path 80, etc., the process ends. If "YES" is true in step S51 and the acquired image does recognize the AGV 50, multiple parts 60, worker 70, and movement path 80, etc., the process proceeds to step S52.

[0055] Next, as shown in step S52, the recognition determination unit 22 determines whether the image input from the recognition unit 21 contains information about the AGV 50, multiple parts 60, worker 70, and movement path 80. If the recognition determination unit 22 determines that the image input from the recognition unit 21 contains information about the AGV 50, multiple parts 60, worker 70, and movement path 80, it determines whether the AGV 50 and worker 70 match the AGV 50 and worker 70 stored in the storage unit 26.

[0056] If the recognition and determination unit 22 determines that the AGV 50 and worker 70 in the input image match the AGV 50 and worker 70 stored in the storage unit 26, it stores the movement path 80 in association with the AGV 50 and worker 70 in the storage unit 26, as shown in step S53. On the other hand, if the recognition and determination unit 22 determines that the AGV 50 and worker 70 in the input image do not match the AGV 50 and worker 70 stored in the storage unit 26, it adds new AGV 50 and worker 70 and stores the movement path 80 in association with the added AGV 50 and worker 70.

[0057] Figure 14 is a diagram illustrating the information stored by the storage unit 26 in the information output system 2 according to Embodiment 2. As shown in Figure 14, the storage unit 26 may store the AGV 50 and the movement paths 80 of the worker 70 relative to the AGV 50. For example, the storage unit 26 may store the AGVs 51 to 53 and the movement paths 81 to 83 of the workers 71 to 73 relative to the AGVs 51 to 53.

[0058] Figure 15 is a flowchart illustrating a method for generating a layout in the information output method according to Embodiment 2. As shown in step S55 of Figure 15, the recognition and judgment unit 22 analyzes the movement paths 81 to 83 of multiple different workers 71 to 73 relative to AGV 51 to AGV 53. Specifically, for example, the recognition and judgment unit 22 analyzes the lengths of the movement paths 81 to 83 of workers 71 to 73 relative to AGV 51, the vectors of each position included in the movement paths 81 to 83, the movement speed of workers 71 to 73, etc. Then, the recognition and judgment unit 22 generates a layout of the workspace 100 from the analysis of these movement paths 80.

[0059] For example, the recognition and judgment unit 22 compares the movement path 81 of a skilled worker 71 with the movement path 82 of a novice worker 72. The recognition and judgment unit 22 may display the difference between the movement path 81 of the skilled worker 71 and the movement path 82 of the novice worker 72 on the display unit 27. The recognition and judgment unit 22 may also find differences between the movement paths 81 and 82 of a skilled worker 71 and a novice worker 72 from part 62 to part 63. Specifically, the recognition and judgment unit 22 may find that the difference between the movement paths 81 and 82 of multiple different workers 71 and 72 from part 62 to part 63 is greater than the difference between the movement paths 81 and 82 of multiple different workers 71 and 72 from part 61 to part 62. In that case, the recognition and judgment unit 22 may generate a layout with the position of part 63 changed.

[0060] Figure 16 is an example of a layout generated by the recognition and determination unit 22 in the information processing method according to Embodiment 2. As shown in Figure 16, the recognition and determination unit 22 may generate a layout in which the position of part 63 is brought closer to part 62 so that the movement path 80 from part 62 to part 63 is shortened, in order to reduce the difference between the movement paths 81 and 82 from part 62 to part 63. In this way, the recognition and determination unit 22 generates a layout in which part 63 is positioned to shorten the portion where the difference between the movement paths 81 and 82 of multiple different workers 71 and 72 is large. In this manner, as shown in step S56 of Figure 15, the recognition and determination unit 22 generates a layout that changes the position of part 60 in the workspace 100 from the analysis of the movement path 80.

[0061] The recognition and determination unit 22 outputs the generated layout to the projection image generation unit 24. The projection image generation unit 24 generates a projection image to be projected onto the workspace 100. Figure 17 is an example of a projection image generated by the projection image generation unit 24 in the information output method according to Embodiment 2. As shown in Figure 17, the projection image generation unit 24 generates a projection image that indicates the position of the part 60 for the position of the changed part 60 in the generated layout. For example, in the generated layout, the position of the changed part 63 is a position aligned with part 62 on the -Y axis side. Therefore, the projection image generation unit 24 generates a projection image of "63" to indicate that the position of the changed part 63 is the position of part 63, which is aligned with part 62 on the -Y axis side. The projection image generation unit 24 outputs the projection image indicating the position of part 60 to the image projection means 30.

[0062] Next, as shown in step S60 of Figure 12, the image projection means 30 projects a projection image indicating the position of the modified part 60 in the generated layout. Specifically, the image projection means 30 projects a projection image indicating that it is the position of part 60 at the position of the modified part 60 in the generated layout. Figure 18 is a diagram illustrating a projection image projected by the image projection means in the information output method according to Embodiment 2. As shown in Figure 18, the image projection means 30 projects a projection image of "63" indicating that it is the position of part 63 at the position of the modified part 63 in the generated layout. Note that the position on which the image projection means 30 projects the projection image is not limited to the position of part 60 itself. For example, if projection is not possible for reasons such as the image becoming unclear when projected, or if projection is possible but the floor in the foreground can be projected more cleanly without objects or unevenness, the image may not be projected directly onto the position of the modified part 60 itself. In such cases, the image projection means 30 may project a projection image indicating the position of the modified part 60 from nearby with an arrow, or it may project text related to the position of the modified part 60. Thus, the image projection means 30 may output a projected image corresponding to the position of the modified part 60.

[0063] Next, the effects of this embodiment will be described. The information output system 2 of this embodiment generates a layout of parts 60 etc. in the workspace 100 from the analysis of the worker's movement path 80. For example, the determination means 20 generates a layout in which the parts 60 are arranged in such a way that the differences in the movement paths 80 of multiple different workers 70 are large. This reduces the differences in work between skilled workers 71 and novice workers 72, enabling uniform work. In addition, workers 70 can be instructed on the assembly order of the parts 60 using the shortest distance, thereby shortening work time and the work proficiency period. Other configurations and effects are included in the description of Embodiment 1.

[0064] It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention. For example, the image projection means 30 is not limited to projecting a projection image (first reference projection image) showing a component 60 that has been determined to be mounted on the AGV 50. The image projection means 30 may project a projection image (second reference projection image) that includes information to help understand which AGV 50 will be mounted on among the AGV 50s that appear in the workspace 100 over time, targeting a single component 60. Furthermore, the AGV 50 may be equipped with an installation structure that allows the component 60 to be placed in multiple different locations, such as shelves. In this case, the image projection means 30 may project a projection image (third reference projection image) that includes information to help understand which location within the installation structure the component will be placed in, targeting a single component 60.

[0065] The determination means 20 is not limited to determining the parts 60 to be mounted on the AGV 50 based on the AGV 50 (first mobile information) in the acquired image. In addition to determining the parts 60 based on the AGV 50 shown in the image acquired by the image acquisition means 10, the determination means 20 may also determine the parts 60 even when the AGV 50 is not shown in the image, that is, when the AGV 50 is not present in the work space 100. For example, information other than images may include information on the AGV 50 that can be determined based on the AGV 50's operation schedule, which AGV 50 will appear in the work space 100 at what time, and information on the AGV 50 that can be determined by communication from a distant AGV 50, which indicates the position and status of the AGV 50. Therefore, the determination means 20 may determine the parts 60 to be mounted on the AGV 50 based on information indicating the time when the AGV 50 will appear in the work space 100 (second mobile information) and information indicating the position of the AGV 50 until it appears in the work space 100 (third mobile information). As a result, the information output system may, based on the information of the determined part 60, project at an appropriate timing, position, and content according to, for example, the position and state of the AGV 50 that has entered the image of the workspace 100.

[0066] Furthermore, the image acquisition means 10 may, but is not limited to, acquire the movement path 80 of the worker 70 as they move through the workspace 100 when the part 60 is loaded onto the AGV 50. The acquisition of the movement path 80 may be performed by another entity that receives image information from the image acquisition means 10. For this reason, the decision means 20 may analyze the movement path 80 acquired by means other than the image acquisition means 10. For example, the decision means 20 may determine the movement path by having a separate calculation unit analyze (track) the position of the AGV 50 in each of the acquired time-series images. Note that the timing for acquiring the movement path 80 is not limited to the timing when the part 60 is loaded onto the AGV 50. In other words, the timing when the part 60 is loaded onto the AGV 50 does not have to be used as the start trigger for loading.

[0067] Combinations of the configurations of Embodiments 1 and 2 are also included within the scope of the technical concept of this embodiment. Furthermore, an information output program that causes a computer to execute an information output method is also included within the scope of the technical concept of this embodiment.

[0068] As described above, the determination means 20 in this embodiment may be an information processing device such as a PC or server. The information processing device may include a processor, memory, and storage device. The storage device may store programs that perform the processing performed by each component of the determination means 20. The processor may also load a program from the storage device into memory and execute the program. In this way, the processor realizes the functions of each component in the determination means 20. The information processing device may realize the determination means 20 shown in Figure 3 by having the processor execute programs corresponding to the recognition unit 21, recognition determination unit 22, recognition image generation unit 23, projection image generation unit 24, and integration unit 25, etc., while referring to the memory and storage device.

[0069] Each component of the determination means 20 may be implemented with dedicated hardware. Furthermore, some or all of each component may be implemented by general-purpose or dedicated circuits, processors, etc., or combinations thereof. These may be implemented by a single chip or by multiple chips connected via a bus. Some or all of each component may be implemented by a combination of the aforementioned circuits, etc., and a program. Additionally, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field-programmable Gate Array), quantum processor (quantum computer control chip), etc., can be used as the processor PRC.

[0070] Furthermore, if some or all of the components of the determination means 20 are implemented by multiple information processing devices or circuits, these multiple information processing devices or circuits may be centrally located or distributed. For example, the information processing devices or circuits may be implemented in a form where each is connected via a communication network, such as a client-server system or a cloud computing system. Judgment means 20 The functionality may be provided in SaaS (Software as a Service) format.

[0071] The program, when loaded into a decision means 20 including a computer, includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium including electrical, optical, acoustic or other forms of propagating signals.

[0072] The information output method and information output program described below are also included within the scope of the technical concept of this embodiment.

[0073] (Note 1) The system captures images of the workspace, including a moving object, multiple parts, and at least one worker. Based on at least one of the following: first moving body information showing the moving body in the acquired image, second moving body information showing the time when the moving body appears in the workspace, and third moving body information showing the position of the moving body until it appears in the workspace, the system determines which component to be mounted on the moving body. A projection image is projected that includes at least one of the following: a first reference projection image showing the component to be mounted on the moving body; a second reference projection image showing which of the moving bodies appearing in the workspace will be mounted on; and a third reference projection image showing the storage position of the moving body. From the analysis of the movement path of the worker as they move through the workspace, a layout is generated that changes the position of the parts within the workspace. The projected image showing the changed position of the component in the aforementioned layout is projected. Information output method. (Note 2) When determining which components to mount on the moving body, the order in which multiple components are mounted is determined. When projecting a projection image to indicate that the components will be mounted, project a projection image showing the mounting order of the multiple components. The method for outputting information as described in Appendix 1. (Note 3) When the component is mounted on the moving body, multiple movement paths of multiple different workers moving through the workspace are acquired. When generating the aforementioned layout, the layout is generated in which the components are arranged in such a way that the differences in the movement paths of the multiple different workers are significant. The method of providing information as described in Appendix 2. (Note 4) When generating the aforementioned layout, if the difference in the movement paths of the multiple different workers from the second part to the third part is greater than the difference in the movement paths of the multiple different workers from the first part to the second part, the layout is generated with the position of the third part changed. The method for outputting information as described in Appendix 3. (Note 5) Images of the workspace, including a moving object, multiple parts, and at least one worker, are acquired from the image acquisition means. Based on at least one of the following: first moving body information showing the moving body in the acquired image, second moving body information showing the time when the moving body appears in the workspace, and third moving body information showing the position of the moving body until it appears in the workspace, the system determines which component to be mounted on the moving body. The image projection means projects a projection image that includes at least one of the following: a first reference projection image showing the component to be mounted on the moving body, a second reference projection image showing which of the moving bodies appearing in the workspace will be mounted on, and a third reference projection image showing the storage position of the moving body. From the analysis of the movement path of the worker as they move through the workspace, a layout is generated that changes the position of the parts within the workspace. The projected image showing the position of the modified component in the aforementioned layout is projected onto the image projection means. An information output program that instructs a computer to perform a specific action. (Note 6) When determining which components to mount on the moving body, the order in which multiple components are mounted is determined. When projecting a projection image to indicate that the components will be mounted, project a projection image showing the mounting order of the multiple components. An information output program described in Appendix 5 that causes a computer to perform the following actions. (Note 7) When the component is mounted on the moving body, multiple movement paths of multiple different workers moving through the workspace are acquired. When generating the aforementioned layout, the layout is generated in which the components are arranged in such a way that the differences in the movement paths of the multiple different workers are significant. An information output program described in Appendix 6 that causes a computer to perform the following actions. (Note 8) When generating the aforementioned layout, if the difference in the movement paths of the multiple different workers from the second part to the third part is greater than the difference in the movement paths of the multiple different workers from the first part to the second part, the layout is generated with the position of the third part changed. An information output program described in Appendix 7 that causes a computer to perform the following actions. [Explanation of Symbols]

[0074] 1.2 Information Output System 10 Image acquisition method 20 Judgment means 21 Recognition part 22 Recognition judgment part 23 Recognition Image Generation Unit 24 Projection Image Generation Unit 25. Integration Department 26 Memory Department 27. Representation Section 30 Image projection means 50, 51, 52, 53 AGV Parts 60, 61, 62, 63, 64 Operators 70, 71, 72, and 73 Movement routes 80, 81, 82, and 83 100 workspace

Claims

1. Image acquisition means for acquiring images of a workspace including a moving object, multiple parts, and at least one worker, A determination means for determining the component to be mounted on the moving body based on at least one of the following: first moving body information showing the moving body in the acquired image, second moving body information showing the time when the moving body appears in the workspace, and third moving body information showing the position of the moving body until it appears in the workspace; Image projection means for projecting a projection image that includes at least one of the following: a first reference projection image showing the component to be mounted on the moving body; a second reference projection image showing which of the moving bodies appearing in the workspace will be mounted on; and a third reference projection image showing the storage position of the moving body. An information output system equipped with, The determination means generates a layout that changes the position of the parts in the workspace based on an analysis of the movement path of the worker as they move through the workspace. The image projection means projects the projected image showing the position of the part that has been changed in the layout, The aforementioned determination means determines the mounting order of the multiple components, The image projection means projects the projection image showing the mounting order of the plurality of components, The image acquisition means acquires multiple movement paths of multiple different workers as they move through the workspace when the component is mounted on the moving body. The determination means generates a layout in which the components are arranged in such a way that the differences in the movement paths of the multiple different workers are large. Information output system.

2. The determination means generates the layout with the position of the third part changed if the difference in the movement paths of the multiple different workers from the second part to the third part is greater than the difference in the movement paths of the multiple different workers from the first part to the second part. The information output system according to claim 1.

3. The system captures images of the workspace, including a moving object, multiple parts, and at least one worker. Based on at least one of the following: first moving body information showing the moving body in the acquired image, second moving body information showing the time when the moving body appears in the workspace, and third moving body information showing the position of the moving body until it appears in the workspace, the system determines which component to be mounted on the moving body. A projection image is projected that includes at least one of the following: a first reference projection image showing the component to be mounted on the mobile body; a second reference projection image showing which of the mobile bodies appearing in the workspace will be mounted on; and a third reference projection image showing the storage position of the mobile body. From the analysis of the movement path of the worker as they move through the workspace, a layout is generated that changes the position of the parts within the workspace. The projected image showing the changed position of the component in the aforementioned layout is projected. A method of outputting information, When determining which components to mount on the moving body, the order in which multiple components are mounted is determined. When projecting a projection image that includes at least one of the first reference projection image, the second reference projection image, and the third reference projection image, the projection image showing the mounting order of the multiple components is projected. When the component is mounted on the moving body, multiple movement paths of multiple different workers moving through the workspace are acquired. When generating the aforementioned layout, the layout is generated in which the components are arranged in such a way that the differences in the movement paths of the multiple different workers are significant. Information output method.

4. When generating the layout, if the difference in the movement paths of the multiple different workers from the second part to the third part is greater than the difference in the movement paths of the multiple different workers from the first part to the second part, the layout is generated with the position of the third part changed. The information output method according to claim 3.

5. Images of the workspace, including a moving object, multiple parts, and at least one worker, are acquired from the image acquisition means. Based on at least one of the following: first moving body information showing the moving body in the acquired image, second moving body information showing the time when the moving body appears in the workspace, and third moving body information showing the position of the moving body until it appears in the workspace, the system determines which component to be mounted on the moving body. The image projection means projects a projection image that includes at least one of the following: a first reference projection image showing the component to be mounted on the moving body, a second reference projection image showing which of the moving bodies appearing in the workspace the component will be mounted on, and a third reference projection image showing the storage position of the moving body. From the analysis of the movement path of the worker as they move through the workspace, a layout is generated that changes the position of the parts within the workspace. The projected image showing the position of the modified component in the aforementioned layout is projected onto the image projection means. This is an information output program that causes a computer to perform the following actions: When determining which components to mount on the moving body, the order in which multiple components are mounted is determined. When projecting a projection image that includes at least one of the first reference projection image, the second reference projection image, and the third reference projection image, the projection image showing the mounting order of the multiple components is projected. When the component is mounted on the moving body, multiple movement paths of multiple different workers moving through the workspace are acquired. When generating the aforementioned layout, the layout is generated in which the components are arranged in such a way that the differences in the movement paths of the multiple different workers are significant. An information output program that instructs a computer to perform a specific action.

6. When generating the layout, if the difference in the movement paths of the multiple different workers from the second part to the third part is greater than the difference in the movement paths of the multiple different workers from the first part to the second part, the layout is generated with the position of the third part changed. The information output program according to claim 5, which causes a computer to perform the following action.

Images