Information processing equipment and work support system

Abstract

To provide an information processing device and a work support system which can appropriately perform support of a worker's work.SOLUTION: In a work support system 100, an information processing device 1 which generates work support information for supporting work comprises a processing unit 11 which acquires a worker's operation information. The processing unit 11, when determining that the worker is in an idle state, on the basis of photographic information acquired from a camera 23 of a worker terminal 2, generates the work support information, and displays work support display on a display device 24 of the worker terminal 2 based on the work support information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a work support system.

Background Art

[0002] There is known a measuring device that enables efficient grape bunch harvesting work (see, for example, Patent Document 1). In a conventional measuring device, an information processing apparatus performs a process of identifying a grape bunch to be measured based on detection of gripping of a grape bunch by a gripping body, and a process of starting measurement of the number of grains in the identified grape bunch. Since the measurement target is automatically identified by the operation of gripping an object and predetermined measurement is performed, it is convenient.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, for example, during agricultural work, a worker may hold a container for storing the harvested product and scissors. In addition, the worker may move the surrounding leaves aside with their hand in order to observe an object such as a fruit. That is, the worker's both hands may be occupied. For this reason, a conventional mechanism for supporting work triggered by an operation using a hand such as gripping an object may not be available, and a new mechanism is desired.

[0005] An object of the present invention is to provide a technique capable of appropriately supporting a worker's work.

Means for Solving the Problems

[0006] An exemplary information processing device of the present invention is an information processing device that generates work support information to assist in work, and comprises a processing unit that acquires worker's operation information, wherein the processing unit generates the work support information when it determines that the worker is in a stopped state.

[0007] Furthermore, an exemplary work support system of the present invention comprises an information processing device with the above configuration and a display device that displays a work support display based on the work support information. [Effects of the Invention]

[0008] According to exemplary examples of the present invention, it is possible to appropriately support the work of workers. [Brief explanation of the drawing]

[0009] [Figure 1] Block diagram showing the schematic configuration of the work support system according to the first embodiment. [Figure 2] Schematic diagram showing the configuration of the worker terminal. [Figure 3A] Schematic diagram to explain the ripeness determination model [Figure 3B] Schematic diagram to explain the ripeness determination model [Figure 4] Schematic diagram to explain the physiological disorder assessment model. [Figure 5] Schematic diagram to explain the flower thinning judgment model. [Figure 6] Schematic diagram to explain the leaf removal detection model. [Figure 7] A flowchart illustrating the process by which work support information is generated in an information processing system. [Figure 8] A diagram illustrating a specific example of movement / stop detection. [Figure 9A] Schematic diagram showing an example of a display on a display device. [Figure 9B] Schematic diagram showing an example of a display on a display device. [Figure 9C] Schematic diagram showing an example of a display on a display device. [Figure 10]Schematic diagram for explaining a preferred form of display on a transparent display unit [Figure 11] Diagram for explaining a detailed example of the generation process of work support information [Figure 12] Block diagram showing the schematic configuration of a work support system according to the second embodiment [Figure 13] Diagram showing an example of a work report generated by a work information generation device [Figure 14] Diagram showing an example of an abnormal area record generated by a work information generation device [Figure 15] Diagram showing a modified example of the abnormal area record shown in FIG. 14 [Figure 16] Diagram showing an example of a work history table generated by a work information generation device [Figure 17] Diagram showing an example of a cultivation environment monitoring table generated by a work information generation device

Mode for Carrying Out the Invention

[0010] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

[0011] <1. First Embodiment> (1-1. Outline of Work Support System) FIG. 1 is a block diagram showing the schematic configuration of a work support system 100 according to the first embodiment of the present invention. As shown in FIG. 1, the work support system 100 includes an information processing device 1 and an operator terminal 2. Note that the work support system 100 of the present embodiment is not particularly limited in terms of the application target, but is, for example, a system suitable for performing work support for agricultural work.

[0012] The information processing device 1 and the worker terminal 2 are configured to communicate with each other using either wireless or wired connections. In this embodiment, the information processing device 1 and the worker terminal 2 are configured to communicate with each other using a wireless LAN (Local Area Network), such as Wi-Fi (registered trademark). By using wireless communication, workers who carry the worker terminal 2, for example by wearing it, can move freely and perform their work without having to worry about the location of the information processing device 1.

[0013] The information processing device 1 may be included in the worker terminal 2. In other words, the work support system 100 may be configured as a single device. The worker terminal 2 may be the work support system 100. In this embodiment, there is one worker terminal 2 that is capable of communicating with the information processing device 1, but there may be multiple worker terminals 2 that are capable of communicating with the information processing device 1.

[0014] The information processing device 1 generates work support information to assist in the work. In this embodiment, the information processing device 1 is a computer device such as a personal computer located near the work site where the worker performs the work. However, the information processing device 1 may be a server device that can communicate via a communication network such as the Internet. This server device may be a cloud server.

[0015] As shown in Figure 1, the information processing device 1 includes a processing unit 11. The processing unit 11 is a so-called processor. Preferably, the processing unit 11 includes a GPU (Graphics Processing Unit). The information processing device 1 further includes a storage unit 12. The storage unit 12 stores or stores non-temporarily programs and data that can be read by a computer. The storage unit 12 is a storage medium composed of, for example, a semiconductor memory, a magnetic medium, and an optical medium. Details of the information processing device 1 will be described later.

[0016] As shown in Figure 1, in this embodiment, the worker terminal 2 comprises a control device 21, a sensor 22, a camera 23, and a display device 24. However, at least one of the sensor 22 and the camera 23 may not be included in the worker terminal 2, but may be connected to the worker terminal 2 via wired or wireless communication. Furthermore, the control device 21 may be included in the information processing device 1, or it may be included in the display device 24. In other words, the work support system 100 may be configured to include the information processing device 1 and the display device 24.

[0017] The display device 24 displays work support information based on the work support information generated by the information processing device 1. Therefore, according to the work support system 100, the worker can perform their work while viewing the work support display on the display device 24. As will be described in detail later, in the work support system 100 of this embodiment, the information processing device 1's functions allow the work support display to be automatically displayed at the appropriate time without the worker having to perform any manual operations. In other words, the work support system 100 can appropriately support the worker's work.

[0018] Furthermore, in this embodiment, the display device 24 is included in the worker terminal 2 carried by the worker. In other words, the display device 24 may be configured to be included in the terminal. This allows the worker to easily and immediately view the work support display. Note that "carried by the worker" broadly includes states in which the worker wears or holds the device, such as when the worker wears it on their face or when the worker holds it in their hand.

[0019] The control device 21 is a controller that controls the entire operator terminal 2. The control device 21 controls the sensor 22, camera 24, and display device 24. The control device 21 also performs processing to output information to the information processing device 1. The control device 21 also receives information from the information processing device 1. The control device 21 is composed of, for example, a CPU (Central Processing Unit), RAM (Random Access Memory), and ROM (Read Only Memory). The control device 21 performs various functions by performing calculation processing according to the computer program contained in the ROM, etc.

[0020] Sensor 22 detects the worker's movements. Sensor 22 outputs the detected information to the control device 21. Sensor 22 may consist of only one type of sensor or may include multiple sensors. For example, sensor 22 may include at least one of the following: an acceleration sensor, a gyroscope, a vibration sensor, and a camera. Sensor 22 may also include, for example, a speed sensor that detects speed information. This sensor may be a high-precision GPS (Global Positioning System), an ultrasonic sensor, an optical sensor, etc., capable of detecting distance information for calculating speed information.

[0021] Camera 23 has an optical system and an image sensor such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. Camera 23 outputs captured information to the control device 21. Camera 23 is used to generate work support information. In other words, the work support system 100 may be configured to include a camera 23 used to generate work support information. If the sensor 22 is configured to include a camera, camera 23 may be used interchangeably with the camera of sensor 22.

[0022] The display device 24 displays a work support display based on the work support information generated by the information processing device 1 as described above. Note that the work support information and the work support display may be the same or different, as will be explained in detail later. The display device 24 can be configured using, for example, a liquid crystal panel or an organic EL panel. Details of the display device 24 will be explained later.

[0023] As described above, worker terminal 2 is a terminal device carried by the worker. Worker terminal 2 is, for example, a smartphone, a tablet, or a wearable device. In this embodiment, worker terminal 2 is a wearable device. Figure 2 is a schematic diagram showing the configuration of worker terminal 2 according to an embodiment of the present invention.

[0024] As shown in Figure 2, the worker terminal 2 is configured as wearable glasses and is worn on the worker's face. When describing the structure of the worker terminal 2, which is configured as wearable glasses, the terms front, back, left, and right are used relative to the worker wearing the worker terminal 2.

[0025] The worker terminal 2 has a transparent display unit 2a that is positioned in front of the worker. In other words, the terminal may be configured to have a transparent display unit 2a that is positioned in front of the worker. The transparent display unit 2a is included in the display device 24 described above. That is, the display device 24 is more specifically composed of a transparent liquid crystal display or the like. A worker wearing the worker terminal 2 can see the scenery in front of them through the transparent display unit 2a. The worker can also see the work support display shown on the transparent display unit 2a.

[0026] The worker terminal 2 is positioned on the left and right sides of the transparent display unit 2a and further has a pair of frame units 2b extending behind the transparent display unit 2a. The worker wears the worker terminal 2 on their face by placing the rear ends of the pair of frame units 2b over their ears. The control device 21, sensor 22, and camera 23 are positioned, for example, in appropriate locations on the pair of frame units 2b. The camera 23 is positioned so that its optical axis is directed in front of the face of the worker wearing the worker terminal 2. In other words, the camera 23 photographs what is in front of the worker.

[0027] In this embodiment, the worker terminal 2, which is configured as a wearable device, is in the form of glasses, but it may also be in other configurations such as goggles, or it may be worn on the head, for example.

[0028] Furthermore, in this embodiment, the worker terminal 2 further includes a remote control device 25 configured as its operation unit. The remote control device 25 is electrically connected to a control device 21, which is located, for example, on the frame section 2b, by a cable 2c. The remote control device 25 may be configured to exchange information with the control device 21 using a wireless communication standard such as Bluetooth®. Alternatively, the remote control device 25 may not be provided; in this case, voice input or gesture input may be used, for example. Operation buttons may also be provided on the frame section 2b, etc.

[0029] (1-2. Details of the information processing device) The processing unit 11 of the information processing device 1 acquires information about the worker's movements. This information is, for example, based on a sensor 22 that is equipped to detect the worker's movements. Specifically, the processing unit 11 acquires the detection information from the sensor 22 from the worker terminal 2. The processing unit 11 may also be configured to acquire information after the control device 21, which has acquired the detection information from the sensor 22, has performed a predetermined process.

[0030] The processing unit 11 determines the worker's operating state based on the detection information from the sensor 22. The worker's operating state may include, for example, a moving state where the worker is moving, a stopped state where the worker has stopped moving, and a working state where the worker is performing a task. In this embodiment, the processing unit 11 determines whether the worker is in a moving state or a stopped state based on the detection information from the sensor 22. In addition to determining whether the worker is in a moving state or a stopped state, the processing unit 11 may also be configured to determine whether the worker is in a moving state, a working state, or a stopped state.

[0031] The processing unit 11 may determine the worker's operating state based on acceleration information, angle information, vibration information, velocity information, etc., obtained from the sensor 22 worn by the worker. If the sensor 22 includes a camera, the processing unit 11 may determine the worker's operating state based on changes in images that are continuous over time. In this embodiment, the processing unit 11 determines whether the worker is moving or stationary based on, for example, at least one of the acceleration information, angle information, vibration information, velocity information, and image information. Detailed examples related to the determination of the worker's operating state will be described later.

[0032] Furthermore, the determination of the worker's operating state based on the detection information from the sensor 22 may be performed by the control device 21 of the worker terminal 2. The processing unit 11 may be configured to obtain the determination result of the worker's operating state from the worker terminal 2 and process according to the determination result. In this configuration, the information based on the sensor 22 acquired by the processing unit 11 is the result of the control device 21 determining the operating state based on the detection information from the sensor 22. In other words, the information based on the sensor 22 acquired by the processing unit 11 may be either the detection information detected by the sensor 22 or the result information of the control device 21 determining the operating state based on the detection information from the sensor 22. To put it another way, the operating information acquired by the processing unit 11 may be the result of determining the worker's operating state based on the detection information from the sensor 22.

[0033] The processing unit 11 generates work support information when it determines that the operator is in a stopped state. This work support information may include, for example, text information, graphic information such as frames, image information, etc. Details of the work support information will be described later.

[0034] With this configuration, work support information can be automatically generated when the worker stops moving and focuses on the work object. In other words, work support information can be provided to the worker at the appropriate time without the worker having to perform any manual operations. The worker can easily obtain work support information even if, for example, both hands are occupied or their hands are dirty. Furthermore, the aforementioned manual operations by the worker include not only button operations but also gestures and other operational actions, and the worker can easily obtain work support information without performing any of these. In addition, with this configuration, since work support information is generated on the condition that the worker is stationary, the processing load on information processing can be reduced compared to when work support information is generated continuously.

[0035] In this embodiment, the processing unit 11 outputs work support information to the worker terminal 2 carried by the worker. In other words, the processing unit 11 may be configured to output work support information to the terminal. This configuration is suitable when the information processing device 1 and the worker terminal 2 are separate devices. With this configuration, the worker can obtain work support information at the appropriate time using the worker terminal 2 without having to perform any manual operations. Furthermore, since the work support information is output to the worker terminal 2 only when the worker is in a stopped state, the communication load can be reduced compared to when work support information is output to the worker terminal 2 at all times.

[0036] In detail, the processing unit 11 acquires image information from a camera 23, which is included in the sensor 22 or is provided separately from the sensor 22, and generates work support information based on said image information. In other words, the processing unit 11 may be configured to generate work support information based on image information acquired from the camera 23. With such a configuration, work support information can be generated using information similar to the information that the worker sees, so that the worker's work can be appropriately supported. In this embodiment, the camera 23 is positioned so that its optical axis is directed in front of the face of the worker wearing the worker terminal 2, and is provided to capture the worker's field of view.

[0037] In this embodiment, the memory unit 12 stores at least one trained model 121 obtained by machine learning. The information processing device 1 (processing unit 11) generates work support information using the trained model 121. The trained model 121 is a model that has been trained using a machine learning method such as deep learning. The processing unit 11 generates work support information by executing calculations according to the trained model 121.

[0038] In this embodiment, the number of trained models 121 stored in the memory unit 12 is multiple. That is, multiple types of trained models 121 are stored in the memory unit 12. Different types of trained models 121 result in different types of work support information. The processing unit 11 can execute only one trained model 121, or it can execute multiple trained models 121 simultaneously.

[0039] In this embodiment, the worker can select which type of pre-trained model 121 to use to obtain work support information. The worker can select which type of pre-trained model 121 to use, for example, by using the remote control device 25 described above. The worker can also select and execute multiple types of pre-trained models 121 simultaneously. Furthermore, in a configuration in which multiple types of pre-trained models 121 can be selected, if the information processing device 1 is connected to multiple worker terminals 2, a different type of pre-trained model 121 may be selected for each worker terminal 2.

[0040] The trained model 121 may be, for example, a ripeness determination model for determining ripeness, a physiological disorder determination model for determining physiological disorders, a fruit thinning determination model for determining whether fruit thinning is necessary, a flower thinning determination model for determining whether flower thinning is necessary, a leaf removal determination model for determining whether leaf removal is necessary, or a pest and disease detection model for detecting pests and diseases. Representative examples of these models are briefly described below.

[0041] Figures 3A and 3B are schematic diagrams illustrating the ripeness determination model. In detail, Figure 3A is a schematic diagram showing the growth stages of a crop. Figure 3B is a schematic diagram showing an example of determination using the ripeness determination model. In Figures 3A and 3B, the crop being determined for ripeness is a tomato.

[0042] The ripeness determination model extracts flowers (including buds) and fruits from images captured by camera 23 and classifies them according to their growth stage. The ripeness determination model in this example is a pre-trained model that has been machine-trained using 10 classifications of images, as shown in Figure 3A, which are classified according to their growth stages. The ripeness determination model classifies which of the 10 stages the flowers and fruits contained in the captured images belong to.

[0043] In the example shown in Figure 3B, the captured image contains three tomatoes. The ripeness determination model extracts these three tomatoes and assigns a classification number to each to determine which stage of growth it is in. Specifically, a rectangular bounding box is superimposed on the image to indicate the image area of ​​each tomato. Additionally, text indicating the growth stage classification is superimposed on the image. In the example shown in Figure 3B, two of the tomatoes are at growth stage "7," and one tomato is at growth stage "10." By looking at the image shown in Figure 3B, workers and cultivation managers can easily understand the growth stage of the tomatoes and easily determine when to harvest them.

[0044] Figure 4 is a schematic diagram illustrating the physiological disorder assessment model. More specifically, Figure 4 is a schematic diagram showing an example of assessment using the physiological disorder assessment model. In Figure 4, the crop targeted for physiological disorder assessment is the tomato. There are several types of physiological disorders in tomatoes. Examples of physiological disorders include hollow fruit, blossom-end rot, misshapen fruit, cracked fruit, netted fruit, and poorly colored fruit.

[0045] The physiological disorder detection model in this example is a pre-trained model that was trained using machine learning with training data that classified tomatoes into two categories: abnormal fruits (those with physiological disorders) and normal fruits (those without physiological disorders). The physiological disorder detection model classifies tomatoes into normal and abnormal fruits from captured images. In the example shown in Figure 4, normal tomatoes are extracted from the captured images. A bounding box indicating the image region of the extracted fruit is superimposed on the image. In addition, text indicating that it is a normal fruit is superimposed on the image. If a tomato is an abnormal fruit, text indicating that it is an abnormal fruit is superimposed on the image.

[0046] A worker or cultivation manager who sees the image shown in Figure 4 can determine that the tomato is normal and does not require any action such as removal. However, if the tomato is determined to be abnormal, the worker or manager can recognize that it is necessary to remove the tomato. In this example, the physiological disorder determination model is configured to determine whether the fruit is normal or abnormal, but it may also be configured to determine the type of physiological disorder present in the fruit included in the captured image.

[0047] Figure 5 is a schematic diagram illustrating the flower thinning judgment model. More specifically, Figure 5 is a schematic diagram showing an example of judgment made by the flower thinning judgment model. In Figure 5, the crop subject to flower thinning judgment is a tomato. The flower thinning judgment model in this example is a pre-trained model that has been trained using machine learning with training data that classifies images showing the position of tomato flowers on the stem into two categories: flowers to be thinned and flowers not to be thinned.

[0048] The fruit thinning determination model determines whether or not tomato flowers included in a captured image should be removed. In the example shown in Figure 5, the tomato flowers in the captured image have been determined to be removed. A bounding box indicating the image region of the fruit of the flower that was determined to be removed is superimposed on the image. In addition, text indicating that it should be removed is superimposed on the image. If the tomato flowers are not to be removed, text indicating that they are not to be removed may be superimposed on the image, or the text may not be displayed at all. Workers and cultivation managers looking at the image shown in Figure 5 can easily determine whether or not the tomato flowers should be removed.

[0049] Figure 6 is a schematic diagram illustrating a leaf pruning determination model. More specifically, Figure 6 is a schematic diagram showing an example of determination by the leaf pruning determination model. In Figure 6, the crop targeted for leaf pruning determination is a tomato. The leaf pruning determination model in this example is a pre-trained model that has been trained using machine learning with training data that classifies images of tomato cultivation sites into cases where leaf pruning is necessary and cases where it is not. The leaf pruning determination model is a pre-trained model that can classify whether or not leaf pruning is necessary.

[0050] As shown in Figure 6, when the leaf removal determination model determines from the captured image that leaf removal is necessary, it displays the point of focus in an identifiable manner based on contribution information to the decision, such as that obtained by a known method such as Grad-CAM (Gradient-weighted Class Activation Mapping). In the example shown in Figure 6, the point of focus is identifiable by changing the type of hatching superimposed on the image. Note that the hatching is an example, and other methods such as changing the color may be used. In the example shown in Figure 6, areas of high importance are displayed in a darker color. Workers and cultivation managers who look at the image shown in Figure 6 can easily determine which areas need leaf removal when necessary. If leaf removal is not necessary, for example, the system may display text indicating that there are no areas requiring leaf removal.

[0051] Next, the process by which work support information is generated in the information processing device 1 will be described. Figure 7 is a flowchart showing the process by which work support information is generated in the information processing device 1. In this embodiment, the process shown in Figure 7 is started when the worker puts the worker terminal 2, which is configured as wearable glasses, on their face and turns on the power.

[0052] In step S1, the processing unit 11 performs a movement / stop determination. The movement / stop determination determines whether the worker is in a moving state or a stopped state. Specifically, the processing unit 11 acquires information from the sensor 22 included in the worker terminal 2 worn by the worker and determines whether the worker is in a moving state or a stopped state. If the movement / stop determination determines whether the worker is in a moving state or a stopped state, the process proceeds to the next step S2. Before explaining the process in step S2, a specific example of the movement / stop determination will be explained.

[0053] Figure 8 is a diagram illustrating a specific example of movement / stop detection. Figure 8 includes the first graph G1, shown as black dots, and the second graph G2, shown as white dots. The horizontal axis of both graphs G1 and G2 is time and is common to both. The vertical axis of graph G1 is the acceleration sensor value. Although the acceleration sensor value exists in three directions (x, y, and z), for simplicity, Figure 8 only shows the value in the x-axis direction, assuming that there is no movement in the y-axis direction and movement in the z-axis direction. In addition, although graph G1 is plotted at one-second intervals, in reality, a predetermined number of samples (e.g., 50 to 100 times) are taken per second. The vertical axis of graph G2 is the percentage of times that "movement" was judged in the past second.

[0054] In the second graph G2, the determination of whether or not something is "moving" is made as follows: The difference between the acquired accelerometer value and the previously acquired value is calculated. More specifically, the difference is calculated as the sum of the squares of the difference values ​​for the x, y, and z axes. If the acquired values ​​include outliers, the difference may be calculated after performing outlier processing such as exclusion by a predetermined value, rounding to the mean, or smoothing. If the difference value exceeds a predetermined value, it is determined to be "moving". If the difference value is less than or equal to the predetermined value, it is determined to be "stopped". The predetermined value is an arbitrary value determined, for example, by conducting an experiment. As mentioned above, multiple accelerometer values ​​are obtained per second, so by determining whether or not each of them is "moving", it is possible to calculate the proportion of times that it was determined to be "moving" in the past second.

[0055] If the device is stationary, and the percentage of instances judged as "moving" in the past second exceeds the first threshold Th1, it is determined that it has changed from a stationary state to a moving state. If the device is moving, and the percentage of instances judged as "moving" in the past second falls below the second threshold Th2, it is determined that it has changed from a moving state to a stationary state. The first threshold Th1 and the second threshold Th2 are arbitrary values ​​determined, for example, by conducting an experiment. The first threshold Th1 and the second threshold Th2 may be the same value or different values. Although an acceleration sensor is given as an example, the system is not limited to this, and may consist of at least one of the following: a gyroscope sensor value, a vibration sensor value, a velocity sensor value, and a camera image. Furthermore, instead of relying on the difference from the previously acquired value, it may be determined that it is "moving" if, for example, the sensor value or its absolute value exceeds a predetermined value.

[0056] In step S2, the processing unit 11 determines whether the movement / stop determination indicates a stopped state. If it determines that the movement is stopped (Yes in step S2), the processing unit 11 proceeds to the next step S3. On the other hand, if it determines that the movement is in motion (No in step S2), the processing unit 11 decides not to generate work support information. In other words, it terminates the operation shown in Figure 7. After terminating the operation shown in Figure 7, the processing unit 11 resumes it at a predetermined timing.

[0057] In step S3, the processing unit 11 acquires the captured image taken by the camera 23. In this embodiment, the camera 23 takes a picture only when it is determined that the worker is stationary, and does not take a picture when it is determined that the worker is moving. This reduces the processing load of the captured information taken by the camera 23. For example, it can reduce the communication load between the information processing device 1 and the worker terminal 2. Alternatively, the camera 23 may be configured to take pictures continuously, and the captured images may be continuously acquired by the processing unit 11. With such a configuration, detailed information about the worker's movements can be acquired. Once the processing unit 11 acquires the captured image, it proceeds to the next step S4.

[0058] In step S4, the processing unit 11 performs inference using the trained model 121 with the acquired images. The processing unit 11 uses the acquired images to perform processes such as ripeness determination, physiological disorder determination, fruit thinning determination, flower thinning determination, leaf removal determination, and pest and disease detection. Which processing to perform is predetermined by the operator. The operator can select multiple types of processing. For example, the operator can have ripeness determination and physiological disorder determination performed simultaneously. Once the processing unit 11 completes the inference, it proceeds to the next step S5.

[0059] In step S5, the processing unit 11 determines whether or not there are objects necessary for generating work support information. In this embodiment, this determination is made using the results of inference by the trained model 121. For example, if a ripeness determination model is selected as the trained model 121, and the inference results do not extract crop flowers or fruits, the processing unit 11 determines that there are no objects because work support information cannot be created. On the other hand, if the inference results do extract crop flowers or fruits, the processing unit 11 determines that there are objects because work support information can be created. Note that a trained model other than the trained model 121 used to generate work support information may be used for the determination process of whether or not there are objects.

[0060] If it is determined that an object exists (Yes in step S5), the process proceeds to the next step, S6. If it is determined that there is no object (No in step S5), the processing unit 11 decides not to generate work support information and terminates the operation shown in Figure 7. With this configuration, when there is no object, no work support information is displayed on the transparent display unit 2a in front of the worker, ensuring the worker's field of view. When the worker stops in an area without crops, unnecessary information is not displayed on the transparent display unit 2a, allowing the worker to maintain a clear field of view.

[0061] In step S6, the processing unit 11 determines whether or not there are hazardous materials. Hazardous materials are, for example, work tools that have blades, such as scissors. In this embodiment, this determination is made using the results of inference by the trained model 121. For this purpose, the trained model 121 is specially equipped with a detection function for detecting hazardous materials. However, a trained model other than the trained model 121 used to generate work support information may be used for the determination process of whether or not there are hazardous materials.

[0062] If it is determined that there are no hazardous materials (No in step S6), the process proceeds to the next step S7. If it is determined that there are hazardous materials (Yes in step S6), the processing unit 11 decides not to generate work support information and terminates the process shown in Figure 7. With this configuration, if there are hazardous materials, the work support display will not be shown on the transparent display unit 2a in front of the worker. In other words, the worker's field of view is secured, making it easier for the worker to avoid hazardous materials.

[0063] Note that the processing order of steps S5 and S6 may be reversed. At least one of steps S5 and S6 may be omitted.

[0064] In step S7, the processing unit 11 generates work support information. The work support information is, for example, a work support image in which instruction information for the worker is added to an image captured by the camera 23. The instruction information is information obtained from the inference results of the trained model 121. The instruction information may be, for example, frame information such as a bounding box, or text information indicating the inference results. The work support information may consist only of instruction information. The work support display shown on the display device 24 may be the above-mentioned work support image, and the work support information may be the instruction information. In this case, the work support image may be generated on the worker terminal 2 side, which acquires the instruction information that is the work support information. With such a configuration, it is not necessary to send image information from the information processing device 1 to the worker terminal 2, thus reducing the communication burden between the information processing device 1 and the worker terminal 2. Once the processing unit 11 has generated the work support information, it proceeds to the next step S8.

[0065] In step S8, the processing unit 11 outputs work support information to the worker terminal 2. The worker terminal 2, having received the work support information from the processing unit 11, displays the work support information on the display device 24. Details of the work support information displayed on the display device 24 will be described later. Once the processing unit 11 has completed the output process, it proceeds to the next step S9.

[0066] In step S9, the processing unit 11 waits for a predetermined time. This predetermined time is, for example, a few seconds, but can be any time determined by experiments or other means. Once the predetermined time has elapsed, the processing unit 11 returns to step S1 and repeats the processing from step S1 onward.

[0067] Furthermore, by including the process in step S9, the display device 24 displays the next work support display after a predetermined time has elapsed since the previous work support display was shown. This suppresses the frequent updating of the work support displays shown by the display device 24. As a result, the burden on the worker to recognize the content of the work support displays can be reduced. In addition, it is possible to suppress the display delay of work support displays that occurs due to the accumulation of processing.

[0068] The predetermined waiting time mentioned above is preferably adjustable. This configuration makes it easier to accommodate both experienced workers who want the work support display to update more frequently and novice workers who want the work support display to update more slowly. In other words, the update frequency of the work support display can be changed according to the worker's request, resulting in a configuration that is convenient for the worker.

[0069] Furthermore, as can be seen from the above explanation, in this embodiment, the processing unit 11 generates work support information when it is determined that predetermined conditions are met based on the image information acquired from the camera 23, in addition to being in a stopped state. With this configuration, the timing of generating work support information can be narrowed to a more appropriate timing, and the processing load on the processing unit 11 can be reduced. In addition, the frequency of unnecessary information being displayed on the transparent display unit 2a can be reduced, making it easier to ensure the worker's field of view.

[0070] Furthermore, the specified conditions may be at least one of the following: the presence of an object and the absence of hazardous materials. The object may vary in detail depending on the type of trained model 121 selected for generating work support information. For example, if a physiological disorder detection model is selected, the object may be a fruit; if a flower thinning detection model is selected, the object may be a flower; and if a leaf removal detection model is selected, the object may be leaves and pathways. In addition, the specified conditions may also be conditions such as the worker observing the object for a certain period of time. Whether or not the worker is observing the object for a certain period of time may be determined, for example, from the changes in the captured image over time.

[0071] (1-3.Display device) Next, the details of the display modes of the display device 24 will be described. Figures 9A, 9B, and 9C are schematic diagrams showing examples of displays of the display device 24. Specifically, Figures 9A, 9B, and 9C are diagrams showing changes in the display mode on the transparent display unit 2a in response to changes in the operating state of a worker wearing the worker terminal 2, which is configured as wearable glasses.

[0072] Figure 9A shows the display configuration when the worker is in a moving state. As described above, when the worker is in a moving state, the information processing device 1 does not generate work support information. For this reason, as shown in Figure 9A, no work support information is displayed on the transparent display unit 2a. In other words, the worker can see the scenery through the transparent display unit 2a without being obstructed by work support information. The worker's field of view is wide, and the worker can move safely.

[0073] Figure 9B shows the display configuration immediately after the worker stops moving. The worker has stopped moving and is observing the crops. As described above, when the information processing device 1 detects that the worker is stopped, it performs inference using the trained model 121. The information processing device 1 also checks whether there are any target objects and hazardous materials. Figure 9B corresponds to the display configuration when these processes are being performed. At this point, no work support information has been generated, and as in the case of Figure 9B, no work support display is shown on the transparent display unit 2a.

[0074] Figure 9C shows the display configuration when the worker is stationary and the work support display is shown after fulfilling predetermined conditions. In this embodiment, the predetermined conditions are that both the target object is present and there are no hazardous materials. As shown in Figure 9C, the display device 24 displays the work support display (work support image 5 in this example) superimposed on the view through the transparent display unit 2a. With this configuration, the worker can easily see the work support display even when, for example, their hands are occupied. In addition, in this embodiment, the work support display is displayed only when the worker is stationary and is not displayed when the worker is moving. This reduces the update frequency of the work support display, making it easier to recognize the content of the work support display.

[0075] When the processing unit 11 generates work support information, this information is normally received by the worker terminal 2, and the display device 24 displays the work support information. However, if the communication status is poor, the work support information may not be received and the work support information may not be displayed. To address this, if the conditions for generating work support information are met but the work support information is not received for a certain period of time, the display device 24 may be configured to display a communication failure on the transparent display unit 2a. If the worker who recognizes the communication failure performs a specific action, such as nodding their head, the captured image may be reacquired and the work support information retransmitted.

[0076] In this embodiment, the work support display is a work support image in which instruction information for the worker is added to at least a portion of the captured image taken by the camera 23. In other words, the work support display is a work support image 5 in which instruction information 5a for the worker is added to at least a portion of the captured image of the worker's field of view. If the bounding box or other frame and text information constituting the instruction information 5a were to be directly superimposed on an object such as a crop visible through the transparent display unit 2a, it would be difficult to align the display position of the instruction information 5a, and the position of the instruction information 5a would likely be misaligned. In this embodiment, however, since the instruction information 5a for the worker is superimposed on the captured image of the worker's field of view and displayed on the transparent display unit 2a, it is possible to avoid misalignment from the position of the object. That is, it is possible to provide a display that is easy for the worker to understand.

[0077] In the example shown in Figure 9C, the instruction information 5a consists of a bounding box 5a1 and text information 5a2. However, the configuration of the instruction information 5a may be changed as appropriate depending on the inference results of the trained model 121, etc. In addition, other notification methods, such as voice notifications, may be combined to help the worker recognize and understand the work support image 5.

[0078] In this embodiment, the work support display (work support image 5 as an example) is displayed superimposed on a portion of the view through the transparent display unit 2a (the worker's view). Therefore, even when the work support display is displayed, the worker's view can be ensured, allowing the worker to perform their work safely. In this embodiment, a preferred configuration is to display the work support display at a position offset from the center of the transparent display unit 2a. This ensures that the worker's view is adequately maintained. Specifically, the work support display is displayed in the upper left corner of the transparent display unit 2a. However, this display position may be changed as appropriate.

[0079] Furthermore, it is preferable that the display range of the work support display in the transparent display unit 2a be adjustable. By configuring it in this way, the work support display can be displayed in a way that is easy for the worker to recognize and understand. For example, a remote control device 25 or an audio input device may be used to adjust the display range.

[0080] Furthermore, as shown in Figures 9A, 9B, and 9C, the display device 24 displays the marker 6 at predetermined coordinates on the transparent display section 2a. Placing the marker 6 in the transparent display section 2a in this way makes it easier for the worker to focus on the marker 6. In other words, it is easier to guide the worker's gaze to the position of the marker 6. This makes it easier to match the area the worker focuses on with the area shown by the work support display (work support image). In this embodiment, the predetermined coordinates are the center coordinates of the transparent display section 2a. This allows the worker's gaze to be guided to the center position of the transparent display section 2a. In this embodiment, the shape of the marker 6 is a plus (+) shape, but this is an example, and the display mode of the marker 6 may be changed as appropriate.

[0081] Furthermore, in this embodiment, the display device 24 displays a common marker superimposed on both the view through the transparent display unit 2a and the work support display, which corresponds the positional relationship between the view through the transparent display unit 2a and the work support display. This configuration makes it easier for the worker to recognize the correspondence between their view and the work support display.

[0082] In this embodiment, the first marker 6a, which is displayed superimposed on the field of view through the transparent display unit 2a, is the same as the marker 6 displayed at the center coordinates of the transparent display unit 2a as described above. The second marker 6b, which is displayed superimposed on the work support display (work support image 5), has the same shape as the first marker 6. By making them the same shape, the correspondence can be made clearer. The position of the second marker 6b is positioned at a location corresponding to the center position of the transparent display unit 2a in the captured image. Note that the markers may be replaced with, for example, grid lines instead of the configuration (+ display) of this embodiment.

[0083] Furthermore, as shown in Figures 9A, 9B, and 9C, the display device 24 indicates which of several operating states the worker is in, including a stopped state. This configuration allows the worker to act while being aware of what processing the information processing device 1 is performing. For example, the worker can be aware that the information processing device 1 has determined the worker is in a stopped state and focus on the object or remain still to obtain an appropriate work support display. This allows the camera 23 to capture images with less blur, making the work support display easier to see.

[0084] In this example, the operating status indicator icon 7 is displayed on the lower right side of the transparent display unit 2a. Figure 9A shows an icon indicating that the unit is in a moving state as the operating status indicator icon 7. Figures 9B and 9C show an icon indicating that the unit is stopped as the operating status indicator icon 7. The position, size, and display manner of the operating status indicator icon 7 are illustrative examples and may be changed as appropriate.

[0085] In this example, a colored frame 241 is displayed around the periphery of the transparent display unit 2a. The color of the frame 241 is changed depending on whether the operator is stationary or moving. For example, in the moving state shown in Figure 9A, the frame 241 is red, and in the stationary state shown in Figures 9B and 9C, the frame 241 is blue. This configuration provides the same effect as displaying the operation status indicator icon 7. Through both the color change of the frame 241 and the display of the operation status indicator icon 7, the operator can reliably recognize what operation state the information processing device 1 has determined.

[0086] Figure 10 is a schematic diagram illustrating a preferred display configuration in the transparent display unit 2a. As shown in Figure 10, it is preferable that the display device 24 displays the type of trained model 121 currently in use in an identifiable manner. With this configuration, when there are multiple types of work support displays that are displayed depending on the use of the trained model 121, the worker can recognize which work support display is being displayed and perform the work accordingly. As a result, it becomes easier to obtain the appropriate work support display.

[0087] In the example shown in Figure 10, a model display icon 8 indicating the type of pre-trained model 121 currently in use is displayed on the lower left side of the transparent display unit 2a. In the example shown in Figure 10, three types of pre-trained models 121 are used. Specifically, a ripeness judgment model, a flower thinning judgment model, and a leaf removal judgment model are used. The number of models used, the types of models, and the way the models are displayed are merely examples and may be changed as appropriate.

[0088] Figure 11 is a diagram illustrating a detailed example of the process for generating work support image 5. The upper part of Figure 11 shows the inference results by the proficiency determination model. As shown in this figure, the inference by the proficiency determination model is performed on the entire shooting range of camera 23. The lower part of Figure 11 shows work support image 5.

[0089] In the example shown in Figure 11, the work support image 5 is not configured to directly display the inference results from the proficiency assessment model. The work support image 5 is generated using an image extracted from a portion of the image captured by camera 23. By generating the work support image 5 by extracting a portion of the image captured by camera 23 in this way, the work support image 5 can be brought closer to the user's field of view.

[0090] Furthermore, it is preferable that the image range R extracted from the captured image used to generate the work support image 5 be adjustable. By configuring it in this way, the work support image 5 can be brought closer to the user's field of view, making it easier to understand the work support image 5. For example, a remote control device 25 or an audio input device may be used to adjust the image range R.

[0091] Furthermore, in the example shown in Figure 11, when generating the work support image 5, some of the results inferred by the proficiency assessment model are intentionally excluded. Specifically, the system is configured to intentionally not display inference results that are outside a certain range from the center of the work support image 5. Here, the system is configured to exclude the display of inference results when they fall outside a certain range from the center, but it is not limited to this. For example, the system could also exclude the display of inference results when they exceed a certain quantity, belong to an unnecessary classification class, or have a low judgment score.

[0092] In other words, in the work support image 5, the instruction information 5a may be configured to be displayed only for a specific range, a specific number, a specific classification class, and a specific judgment score. That is, in the work support image 5, the instruction information 5a may be configured to be displayed only for predetermined conditions. With such a configuration, the information in the work support image 5 can be narrowed down to an appropriate amount, reducing the burden on the worker when recognizing the work support image 5. In other words, the worker's work efficiency can be improved. The specific range, specific number, specific classification class, and specific judgment score may be determined appropriately through experiments, etc.

[0093] <2. Second Embodiment> Figure 12 is a block diagram illustrating the schematic configuration of a work support system 200 according to a second embodiment of the present invention. As shown in Figure 12, the work support system 200 comprises an information processing device 1A, a worker terminal 2A, a work information generation device 3, and an administrator terminal 4. The work support system 200 of the second embodiment is, like the work support system 100 of the first embodiment, a system suitable for providing work support, for example, in agricultural work.

[0094] The information processing device 1A has a configuration that is generally the same as that of the information processing device 1 in the first embodiment. For this reason, the explanation of the information processing device 1A will be limited to the differences, and the parts that overlap with the first embodiment will be omitted. The worker terminal 2A has a configuration that is the same as that of the worker terminal 2 in the first embodiment. For this reason, the explanation of the worker terminal 2A will be omitted unless it is particularly necessary. Note that if the information processing device 1A and the worker terminal 2A are configured as separate devices, the number of worker terminals 2A that are communicatively connected to the information processing device 1A is not limited to one, but may be multiple. In this embodiment as well, the number of learned models 121A stored in the storage unit 12A is multiple, but it may be one.

[0095] The information processing device 1A is connected to the work information generation device 3 and the administrator terminal 4 via a communication network 300 such as the Internet. The work information generation device 3 and the administrator terminal 4 are connected to each other via the communication network 300. Alternatively, the information processing device 1A and the administrator terminal 4 may be connected to each other via a LAN such as a wireless LAN.

[0096] The work information generation device 3 is, for example, a server device such as a cloud server. The work information generation device 3 is composed of, for example, a processing unit such as a CPU, RAM, and ROM. The administrator terminal 4 is, for example, a terminal device owned by the administrator such as a personal computer, tablet terminal, or smartphone. The work information generation device 3 may be included in the administrator terminal 4. In this case, the information processing device 1A and the administrator terminal 4 may be configured to communicate via a wireless LAN such as Wi-Fi.

[0097] The processing unit 11A of the information processing device 1A outputs work support information to the worker terminal 2A carried by the worker and to the work information generation device 3 that generates the worker's work information. The part in which the processing unit 11A outputs work support information to the worker terminal 2A is the same as in the first embodiment. In this embodiment, the processing unit 11A outputs work support information not only to the worker terminal 2A but also to the work information generation device 3. With this configuration, it becomes possible to share work support information, including the inference results from the trained model 121A, between the worker and managers or experts.

[0098] Furthermore, in this embodiment, the processing unit 11A outputs information of the trained model 121A used to generate the work support information, in addition to the work support information, to the work information generation device 3. With this configuration, the work information generation device 3 can appropriately infer what kind of work the worker performed and appropriately generate the worker's work record.

[0099] In detail, the processing unit 11A outputs information on the trained model 121A used by the worker, such as a ripeness determination model, a physiological disorder determination model, a flower thinning determination model, a leaf removal determination model, a pest and disease detection model, etc. The work information generation device 3, which has acquired the information on the trained model 121A used, estimates the work content based on this information and records the work.

[0100] For example, if the work information generator 3 receives information that the ripeness determination model was used, it will estimate that harvesting work was performed. For example, if the work information generator 3 receives information that the physiological disorder determination model was used, it will estimate that fruit thinning work was performed. For example, if the work information generator 3 receives information that the flower thinning determination model was used, it will estimate that flower thinning work was performed. For example, if the work information generator 3 receives information that the leaf removal determination model was used, it will estimate that leaf removal and training work was performed. For example, if the work information generator 3 receives information that the ripeness determination model, the physiological disorder determination model, and the disease and pest detection model were used simultaneously, it will estimate that inspection work was performed.

[0101] In this embodiment, the processing unit 11A outputs to the work information generation device 3 not only work support information and information from the learned model 121A used, but also shooting information and work time information acquired from the worker terminal 2A. In addition, various information acquired by sensors 22, etc., may also be output to the work information generation device 3. The work time information may include the work start time and work end time. With this configuration, the work information generation device 3 can generate more detailed work records.

[0102] The processing unit 11A may be configured to output work support information, information on the trained model 121A used, shooting information, and work time information to the work information generation device 3 all at once, for example, when the day's work is completed. Alternatively, the processing unit 11A may be configured to periodically output work support information, etc., to the work information generation device 3 during the day's work. Furthermore, the information processing device 1A may be configured to directly send the work time information to the administrator terminal 4 via email or the like. For example, the work time information may be automatically sent from the information processing device 1A to the administrator terminal 4 when the work is completed. With this configuration, the administrator can immediately know when the worker has completed their work.

[0103] The administrator can use the administrator terminal 4 to view the work information generated by the work information generation device 3 on a screen or by printing it out. The administrator can also save the work information generated by the work information generation device 3 to the administrator terminal 4. Furthermore, the administrator can process the work information generated by the work information generation device 3 using the administrator terminal 4.

[0104] Figure 13 shows an example of a work report 31 generated by the work information generation device 3. The work report 31 is an example of the work information described above. In the example shown in Figure 13, the work report includes the work date, work start time, work end time, worker, work location, crop to be worked on, and work details. All of this information can be generated from information acquired from the information processing device 1A. The worker listed in the work report 31 is identified, for example, using a user ID transmitted from the worker terminal 2A to the information processing device 1A. The work location is identified, for example, using the worker's location information transmitted from the worker terminal 2A to the information processing device 1A. The crop to be worked on and the breakdown of work are determined, for example, based on the inference results of the trained model 121A. In the breakdown of work, "JD" means ripeness, for example, JD10 means that the ripeness is 10 out of 10 levels and that it is ready for harvest.

[0105] Figure 14 shows an example of an abnormal area record 32 generated by the work information generation device 3. The abnormal area record 32 is an example of the work information described above. In the example shown in Figure 14, six cultivation shelves 322 are shown, which are located inside a greenhouse 321 where crops are cultivated. In Figure 14, the black dots shown above or below the cultivation shelves 322 indicate the worker's movement trajectory 323. The white circle 324 displayed superimposed on the work trajectory 323 indicates that an abnormality was detected. More specifically, the circle 324 indicates that an abnormality was detected near the location indicated by the circle 324 on the cultivation shelf 322.

[0106] An abnormality is one detected by the trained model 121A. If the physiological disorder detection model is used as the trained model 121A, the circle 324 indicates that an abnormal fruit, such as blossom-end rot, has been detected. If the pest and disease detection model is used as the trained model 121A, the circle 324 indicates that a pest or disease has been detected. If, in response to the detection of an abnormality, the worker takes action such as removing the abnormal fruit, this action may be included in the record.

[0107] Figure 15 shows a modified version of the abnormal area record 32 shown in Figure 14. In the modified abnormal area record 32A, the degree of abnormality is classified into multiple categories according to the number of abnormalities per predetermined area, and a mapping format is adopted in which the distribution of the degree of abnormality is shown for each cultivation shelf 322A in greenhouse 321A. In the example shown in Figure 15, the degree of abnormality is classified into three stages, but this is an example, and the degree of abnormality may be classified into two stages or four or more stages. The example shown in Figure 15 is suitable for showing, for example, the occurrence of physiologically impaired fruit or pests and diseases, or the occurrence of areas where leaf removal is necessary.

[0108] Figure 16 shows an example of a work history table 33 generated by the work information generation device 3. The work history table 33 is an example of the work information described above. The work history table 33 is a table that shows the work performed on each day for a certain number of days. What kind of work was performed on each day is estimated by the type of trained model 121A used as described above. The parts shown in bold lines indicate that the corresponding work was performed.

[0109] In the example shown in Figure 16, there are five types of work: leaf removal, training, fruit thinning, pest control, and harvesting. However, this is merely an example, and the types of work can be changed as appropriate. In the example shown in Figure 16, leaf removal and training are performed on date X. On date Y, leaf removal, training, and harvesting are performed. By looking at the work history table 33, managers can easily grasp what kind of work was done on each day and understand the trends in when and what kind of work is done. In other words, by looking at the work history table 33, managers can easily create work plans.

[0110] Figure 17 shows an example of a cultivation environment monitoring table 34 generated by the work information generation device 3. The cultivation environment monitoring table 34 is an example of the work information described above. The cultivation environment monitoring table 34 can be created, for example, using the inference results obtained by the leaf pruning judgment model. In the example shown in Figure 17, every week, the degree of leaf pruning required for each cultivation shelf is determined based on the inference results of the leaf pruning judgment model, and hatching corresponding to the required degree is applied to the corresponding date and cultivation shelf.

[0111] In the example shown in Figure 17, the degree of need for leaf removal is classified into five levels. For the sake of explanation, the levels of need for leaf removal, from least to most, are designated as "Level 1," "Level 2," "Level 3," "Level 4," and "Level 5." Level "5" represents the state where leaf removal is most necessary and corresponds to "Leaf Removal Required" in Figure 17.

[0112] As an example, let's focus on cultivation shelves A, B, and C. On date S, all cultivation shelves A-C have a leaf pruning requirement of "1". One week later, on date T, all cultivation shelves A-C have a leaf pruning requirement of "2". Two weeks later, on date U, cultivation shelf A has a leaf pruning requirement of "2", while cultivation shelves B and C have a leaf pruning requirement of "3". Three weeks later, on date V, cultivation shelf A has a leaf pruning requirement of "4", while cultivation shelves B and C have a leaf pruning requirement of "5".

[0113] By reviewing the cultivation environment monitoring table 34, managers can understand the time-series changes in the need for leaf pruning. This allows managers to predict whether leaf pruning will be necessary in the future. This prediction enables them to appropriately determine the work schedule and prepare the necessary personnel.

[0114] <3. Things to keep in mind> Various technical features disclosed herein can be modified in various ways without departing from the spirit of the technical creation. Furthermore, the multiple embodiments and modifications shown herein may be combined as possible. [Explanation of Symbols]

[0115] 1. 1A... Information Processing Device 2, 2A... Worker terminal 2a...Transparent display section 3...Work information generation device 5. Work support images 5a...Instruction information 6. Marker 6a...First marker (common marker) 6b...Second marker (common marker) 11, 11A... Processing Unit 12, 12A...Storage section 22...Sensor 23...Camera 24...Display device 31. Work Report (Work Information) 32, 32A... Abnormal area record (work information) 33. Work History Table (Work Information) 34. Cultivation Environment Monitoring Chart (Work Information) 100, 200... Work support system 121, 121A... Pre-trained models

Claims

1. An information processing device that generates work support information to assist in work, It includes a processing unit that acquires worker movement information and camera image information. The processing unit is an information processing device that generates work support information when it determines that the worker is in a stopped state and further determines that predetermined conditions are met based on the photographic information.

2. The information processing apparatus according to claim 1, wherein the processing unit generates the work support information based on the shooting information.

3. The information processing apparatus according to claim 1 or 2, wherein the processing unit outputs the work support information to a terminal.

4. The information processing apparatus according to claim 3, wherein the processing unit outputs the work support information to a work information generation device that generates the worker's work information.

5. It further includes a memory unit that stores at least one pre-trained model created using machine learning, The information processing apparatus according to claim 4, wherein the processing unit outputs information of the trained model used to generate the work support information to the work information generation device.

6. An information processing device according to any one of claims 1 to 5, A display device that displays work support information based on the aforementioned work support information, A work support system equipped with the following features.

7. Equipped with the aforementioned camera, The work support system according to claim 6, wherein the work support display is a work support image in which instruction information for the worker is attached to at least a portion of the image captured by the camera.

8. The work support system according to claim 7, wherein the instruction information in the work support image is displayed only under predetermined conditions.

9. The work support system according to claim 7 or 8, wherein the image range extracted from the captured image used for generating the work support image is provided to be adjustable.

10. The display device is included in the terminal, and is part of the work support system according to any one of claims 6 to 9.

11. The terminal has a transparent display unit that is positioned in front of the worker's eyes. The work support system according to claim 10, wherein the display device displays the work support display superimposed on the field of view through the transparent display unit.

12. The work support system according to claim 11, wherein the work support display is displayed superimposed on a portion of the field of view through the transparent display unit.

13. The work support system according to claim 12, wherein the display range of the work support display in the transparent display section is adjustable.

14. The work support system according to any one of claims 11 to 13, wherein the display device displays a common marker that corresponds the positional relationship between the view through the transparent display unit and the work support display, superimposed on the view through the transparent display unit and the work support display, respectively.

15. The work support system according to any one of claims 11 to 13, wherein the display device displays a marker at predetermined coordinates of the transparent display portion.

16. The work support system according to any one of claims 6 to 15, wherein the display device indicates which of the multiple operating states, including the stopped state, the worker is in.

17. The information processing device further comprises a storage unit that stores at least one trained model obtained by machine learning, and generates the work support information using the trained model. The work support system according to any one of claims 6 to 16, wherein the display device displays the type of the trained model currently in use in an identifiable manner.

18. The display device displays the next work support display after a predetermined time has elapsed since the previous work support display was displayed. The work support system according to any one of claims 6 to 17, wherein the predetermined time is provided to be changeable.

Images