Information processing apparatus and agricultural work support system

Abstract

To provide a technique for appropriately supporting the work of a farm worker.SOLUTION: An exemplary information processing apparatus for generating farm work support information for supporting farm work is provided with a processing unit for acquiring operation information of a farm worker and generating the farm work support information when determining that the farm worker is in a stopped state.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an agricultural work support system. [Background technology]

[0002] A measuring device that enables efficient grape picking work is known (see, for example, Patent Document 1). In conventional measuring devices, an information processing device performs the process of identifying a bunch of grapes to be measured based on the detection of the gripping of the bunch of grapes by a gripper, and the process of starting to count the number of grapes in the identified bunch of grapes. This is convenient because the measurement target is automatically identified by the action of gripping an object, and the specified measurement is performed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6744898 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, when working in the fields, a worker may need to carry a container for harvested produce and scissors. Furthermore, the worker may need to use his or her hands to move surrounding leaves aside in order to observe an object, such as fruit. In other words, the worker's hands may be full. For this reason, conventional mechanisms that assist work by using hands to grasp an object, for example, may not be useful, and a new mechanism is desired.

[0005] The present invention aims to provide a technique that can appropriately support the work of agricultural workers. [Means for solving the problem]

[0006] An exemplary information processing device of the present invention is an information processing device that generates agricultural work support information that supports agricultural work, and is equipped with a processing unit that acquires agricultural worker movement information, and the processing unit generates the agricultural work support information when it determines that the agricultural worker is in a stopped state.

[0007] Moreover, an exemplary farm work support system of the present invention includes an information processing device configured as described above, and a display device that displays a farm work support display based on the farm work support information. [Effects of the Invention]

[0008] According to the exemplary embodiment of the present invention, farm workers can be appropriately assisted in their work. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a work assistance system according to a first embodiment. [Figure 2] Schematic diagram showing the configuration of a worker terminal [Figure 3A] Schematic diagram for explaining the ripeness determination model [Figure 3B] Schematic diagram for explaining the ripeness determination model [Figure 4] Schematic diagram for explaining the physiological disorder determination model [Figure 5] Schematic diagram for explaining the flower thinning judgment model [Figure 6] Schematic diagram for explaining the leaf missing detection model [Figure 7] A flowchart showing a process for generating work support information in an information processing device. [Figure 8] A diagram to explain a specific example of movement / stop determination [Figure 9A] Schematic diagram showing a display example of a display device [Figure 9B] Schematic diagram showing a display example of a display device [Figure 9C] Schematic diagram showing a display example of a display device [Figure 10]Schematic diagram for explaining a preferred form of display on the transparent display portion. [Figure 11] FIG. 10 is a diagram illustrating a detailed example of a process for generating work support information. [Figure 12] FIG. 10 is a block diagram showing a schematic configuration of a work support system according to a second embodiment. [Figure 13] FIG. 10 is a diagram showing an example of a work report generated by the work information generation device. [Figure 14] FIG. 10 is a diagram showing an example of an abnormal area record generated by the work information generation device. [Figure 15] FIG. 15 is a diagram showing a modification of the abnormal area recording shown in FIG. 14. [Figure 16] FIG. 10 is a diagram showing an example of a work history table generated by a work information generation device. [Figure 17] FIG. 10 is a diagram showing an example of a cultivation environment monitoring table generated by the work information generating device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

[0011] 1. First Embodiment (1-1. Overview of the work support system) Fig. 1 is a block diagram showing a schematic configuration of a work support system 100 according to a first embodiment of the present invention. As shown in Fig. 1, the work support system 100 includes an information processing device 1 and a worker terminal 2. The work support system 100 of this embodiment is not intended to be particularly limited to specific applications, but is a system suitable for supporting agricultural work, for example.

[0012] The information processing device 1 and the worker terminal 2 are capable of communicating with each other wirelessly or via a wired connection. In this embodiment, the information processing device 1 and the worker terminal 2 are capable of communicating with each other using a wireless LAN (Local Area Network) such as Wi-Fi (registered trademark). By using wireless technology, a worker who carries the worker terminal 2 by wearing it on their body or the like can move freely and perform work without worrying about the location of the information processing device 1.

[0013] The information processing device 1 may be included in the worker terminal 2. That is, the work support system 100 may be configured as one device. The worker terminal 2 may be the work support system 100. In addition, in this embodiment, the number of worker terminals 2 that are provided to be able to communicate with the information processing device 1 is one, but the number of worker terminals 2 that are provided to be able to communicate with the information processing device 1 may be multiple.

[0014] The information processing device 1 generates work support information that supports work. In this embodiment, the information processing device 1 is a computer device such as a personal computer that is placed near a work site where a worker performs work. However, the information processing device 1 may also be a server device that can communicate via a communication network such as the Internet. The server device may also be a cloud server.

[0015] As shown in FIG. 1, the information processing device 1 includes a processing unit 11. The processing unit 11 is a so-called processor. In a preferred embodiment, the processing unit 11 includes a GPU (Graphics Processing Unit). The information processing device 1 further includes a storage unit 12. The storage unit 12 non-temporarily stores or memorizes computer-readable programs, data, and the like. The storage unit 12 is a storage medium configured, for example, by a semiconductor memory, a magnetic medium, an optical medium, and the like. Details of the information processing device 1 will be described later.

[0016] 1 , in this embodiment, the worker terminal 2 includes a control device 21, a sensor 22, a camera 23, and a display device 24. However, at least one of the sensor 22 and the camera 23 may not be included in the worker terminal 2, but may be connected to the worker terminal 2 so as to be able to communicate with it via a wired or wireless connection. Furthermore, the control device 21 may be included in the information processing device 1, or may be included in the display device 24. In other words, the work support system 100 may be configured to include the information processing device 1 and the display device 24.

[0017] The display device 24 displays a work support display based on the work support information generated by the information processing device 1. Therefore, according to the work support system 100, the worker can perform work while looking at the work support display displayed on the display device 24. As will be described in detail later, in the work support system 100 of this embodiment, the function of the information processing device 1 allows the worker to automatically display the work support display at an appropriate timing without performing any operation using their hands. In other words, according to the work support system 100, the worker's work can be appropriately supported.

[0018] In this embodiment, the display device 24 is included in the worker terminal 2 carried by the worker. In other words, the display device 24 may be configured to be included in the terminal. This allows the worker to easily and immediately view the work support display. Note that "carried by the worker" broadly includes a state in which the worker carries it on their body, and includes, for example, a state in which the worker wears it on their face or a state in which the worker holds it in their hand.

[0019] The control device 21 is a controller that controls the entire worker terminal 2. The control device 21 controls the sensor 22, the camera 24, and the display device 24. The control device 21 also performs a process of outputting information to the information processing device 1. The control device 21 also receives information input from the information processing device 1. The control device 21 is configured to include, for example, an arithmetic unit such as a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory). The control device 21 performs various functions by performing arithmetic processing in accordance with a computer program stored in the ROM or the like.

[0020] The sensor 22 detects the movement of the worker. The sensor 22 outputs the detected information to the control device 21. The sensor 22 may be configured with only one type of sensor, or may include multiple sensors. For example, the sensor 22 may be configured to include at least one of an acceleration sensor, a gyro sensor, a vibration sensor, and a camera. The sensor 22 may also be configured to include, for example, a speed sensor that detects speed information. The sensor may be a high-precision GPS (Global Positioning System), an ultrasonic sensor, an optical sensor, or the like that can detect distance information for calculating speed information.

[0021] The camera 23 has an optical system and an imaging element such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The camera 23 outputs photographic information to the control device 21. The camera 23 is used to generate work support information. In other words, the work support system 100 may be configured to include a camera 23 used to generate work support information. Note that, if the sensor 22 is configured to include a camera, the camera 23 may also serve as the camera of the sensor 22.

[0022] The display device 24 displays a work support display based on the work support information generated by the information processing device 1 as described above. Note that, as will be described in detail later, the work support information and the work support display may be the same or different. The display device 24 can be configured using, for example, a liquid crystal panel or an organic EL panel. Details of the display device 24 will be described later.

[0023] As described above, the worker terminal 2 is a terminal device carried by a worker. The worker terminal 2 is, for example, a smartphone, a tablet terminal, or a wearable device. In this embodiment, the worker terminal 2 is a wearable device. FIG. 2 is a schematic diagram showing the configuration of the worker terminal 2 according to the embodiment of the present invention.

[0024] 2, the worker terminal 2 is configured as wearable glasses and is worn on the worker's face. In describing the structure of the worker terminal 2 configured as wearable glasses, the expressions front, back, left, and right are used based on the worker wearing the worker terminal 2.

[0025] The worker terminal 2 has a transparent display unit 2a that is placed in front of the worker's eyes. In other words, the terminal may be configured to have the transparent display unit 2a that is placed in front of the worker's eyes. The transparent display unit 2a is included in the display device 24 described above. In other words, the display device 24 is specifically configured by a transparent liquid crystal display or the like. A worker wearing the worker terminal 2 can see the scenery ahead through the transparent display unit 2a. The worker can also see the work support display displayed on the transparent display unit 2a.

[0026] The worker terminal 2 is disposed on the left and right sides of the transparent display unit 2a and further has a pair of frame units 2b extending rearward from the transparent display unit 2a. The worker places the rear ends of the pair of frame units 2b over their ears and wears the worker terminal 2 on their face. The control device 21, sensor 22, and camera 23 described above are disposed, for example, in appropriate positions on the pair of frame units 2b. The camera 23 is disposed so that its optical axis faces forward of the face of the worker wearing the worker terminal 2. In other words, the camera 23 captures an image in front of the worker.

[0027] In this embodiment, the worker terminal 2 configured as a wearable device is in the form of glasses, but may be in the form of goggles or other configurations, and may be worn on the head, for example.

[0028] In this embodiment, the worker terminal 2 further includes a remote control device 25 configured as its operation unit. The remote control device 25 is electrically connected to the control device 21, which is disposed on the frame portion 2b or the like, via a cable 2c. The remote control device 25 may be configured to exchange information with the control device 21 using a wireless communication standard such as Bluetooth (registered trademark). The remote control device 25 may not be provided, and in this case, voice input or gesture input may be used. An operation button may also be provided on the frame portion 2b or the like.

[0029] (1-2. Details of the information processing device) The processing unit 11 of the information processing device 1 acquires motion information of the worker. The motion information is, for example, information based on a sensor 22 that is provided so as to be able to detect the movement of the worker. In detail, the processing unit 11 acquires the detection information of the sensor 22 from the worker terminal 2. Note that the processing unit 11 may be configured to acquire information after the control device 21, which has acquired the detection information from the sensor 22, has performed a predetermined process.

[0030] The processing unit 11 determines the motion state of the worker based on the detection information of the sensor 22. The motion state of the worker may include, for example, a moving state in which the worker is moving, a stationary state in which the worker has stopped moving, and a working state in which the worker is working. In this embodiment, the processing unit 11 determines whether the worker is in a moving state or a stationary state based on the detection information of the sensor 22. Note that the processing unit 11 may be configured to determine not only whether the worker is in a moving state or a stationary state, but also whether the worker is in a moving state, a working state, or a stationary state, for example.

[0031] The processing unit 11 may determine the motion state of the worker based on, for example, acceleration information, angle information, vibration information, speed information, etc. obtained from the sensor 22 worn by the worker. Furthermore, if the sensor 22 includes a camera, the processing unit 11 may determine the motion state of the worker based on changes in images that are continuous over time. In this embodiment, the processing unit 11 determines whether the worker is in a moving state or a stationary state based on, for example, at least one of acceleration information, angle information, vibration information, speed information, and photographic information. A detailed example of determining the motion state of the worker will be described later.

[0032] The determination of the worker's motion state based on the detection information of the sensor 22 may be performed by the control device 21 of the worker terminal 2. The processing unit 11 may be configured to acquire a determination result of the worker's motion state from the worker terminal 2 and perform processing according to the determination result. In this configuration, the information based on the sensor 22 acquired by the processing unit 11 is the result of the control device 21 determining the motion state based on the detection information of the sensor 22. In other words, the information based on the sensor 22 acquired by the processing unit 11 may be either detection information detected by the sensor 22 or result information of the control device 21 determining the motion state based on the detection information of the sensor 22. In other words, the motion information acquired by the processing unit 11 may be the result of determining the worker's motion state based on the detection information of the sensor 22.

[0033] When it is determined that the worker is in a stopped state, the processing unit 11 generates work support information. The work support information may be, for example, text information, graphic information such as a frame, image information, etc. Details of the work support information will be described later.

[0034] With this configuration, work support information can be automatically generated when the worker stops moving and focuses on the work object. In other words, work support information can be provided to the worker at an appropriate timing without the worker having to perform any operations using their hands. For example, the worker can easily obtain work support information even when both hands are full or their hands are dirty. Note that the above-mentioned operations using the worker's hands include not only button operations, etc., but also gestures and other operational movements, and the worker can easily obtain work support information without performing these operations. Furthermore, with this configuration, work support information is generated only when the worker is stopped, thereby reducing the processing load of information processing compared to when work support information is generated constantly.

[0035] In this embodiment, the processing unit 11 outputs the work support information to a worker terminal 2 carried by the worker. That is, the processing unit 11 may be configured to output the work support information to a terminal. This configuration is suitable for the case where the information processing device 1 and the worker terminal 2 are separate devices. With this configuration, the worker can obtain the work support information at an appropriate timing using the worker terminal 2 without performing an operation with their hands. Furthermore, since the configuration is such that the work support information is output to the worker terminal 2 on the condition that the worker is in a stopped state, the communication load can be reduced compared to when the work support information is constantly output to the worker terminal 2.

[0036] In detail, the processing unit 11 acquires image capture information from the camera 23 that is included in the sensor 22 or that is provided separately from the sensor 22, and generates work support information based on the image capture information. In other words, the processing unit 11 may be configured to generate work support information based on image capture information acquired from the camera 23. With such a configuration, it is possible to generate work support information using information similar to the information seen by the worker, thereby enabling appropriate support for the worker's work. In this embodiment, the camera 23 is disposed so that the optical axis of the camera 23 faces in front of the face of the worker wearing the worker terminal 2, and is provided so as to be able to capture images of the worker's field of view.

[0037] In this embodiment, the storage unit 12 stores at least one trained model 121 obtained by machine learning. The information processing device 1 (processing unit 11) generates work support information by using the trained model 121. The trained model 121 is a model that has been trained by machine learning using a machine learning method such as deep learning. The processing unit 11 generates work support information by executing calculation processing in accordance with the trained model 121.

[0038] In this embodiment, the memory unit 12 stores a plurality of trained models 121. That is, the memory unit 12 stores a plurality of types of trained models 121. Different types of trained models 121 result in different types of work assistance information. The processing unit 11 can execute only one trained model 121, or can execute multiple trained models 121 simultaneously.

[0039] In this embodiment, the worker can select which type of trained model 121 to use to acquire work assistance information. The worker can select which type of trained model 121 to use, for example, by using the above-mentioned remote control device 25. The worker can also select and execute multiple types of trained models 121 simultaneously. Furthermore, in a configuration in which multiple types of trained models 121 can be selected, if the information processing device 1 is connected to multiple worker terminals 2, a different type of trained model 121 may be selected for each worker terminal 2.

[0040] The trained model 121 may be, for example, a ripeness determination model for determining ripeness, a physiological disorder determination model for determining physiological disorders, a fruit thinning determination model for determining fruit thinning, a flower thinning determination model for determining flower thinning, a leaf loss determination model for determining whether leaf loss is necessary, a pest detection model for detecting pests, etc. Representative examples of these models will be briefly described below.

[0041] 3A and 3B are schematic diagrams for explaining a ripeness determination model. In detail, FIG. 3A is a schematic diagram showing the growth stages of a crop. FIG. 3B is a schematic diagram showing an example of determination using the ripeness determination model. In FIGS. 3A and 3B, the crop to be subjected to ripeness determination is a tomato.

[0042] The ripeness determination model extracts flowers (including buds) and fruits contained in the captured image taken by the camera 23 and classifies them according to their growth stage. The ripeness determination model in this example is a trained model that has undergone machine learning using as training data images of 10 categories classified according to the growth stage shown in Fig. 3A. The ripeness determination model classifies the flowers and fruits contained in the captured image into one of 10 stages.

[0043] In the example shown in FIG. 3B, three tomatoes are included in the captured image. The ripeness determination model extracts these three tomatoes and assigns a classification number to each fruit to classify it into one of the growth stages. In detail, a rectangular frame (bounding box) indicating the image area of ​​the tomato is superimposed on the image. Also, a text display indicating the growth stage classification is superimposed on the image. In the example shown in FIG. 3B, two tomatoes are at growth stage "7" and one tomato is at growth stage "10." By viewing the image shown in FIG. 3B, workers and cultivation managers can easily understand the growth stages of the tomatoes and easily determine when to harvest them.

[0044] Fig. 4 is a schematic diagram for explaining a physiological disorder determination model. In detail, Fig. 4 is a schematic diagram showing an example of determination using the physiological disorder determination model. In Fig. 4, the crop to be determined for physiological disorders is a tomato. There are multiple types of physiological disorders in tomatoes. Examples of physiological disorders include hollow fruit, fruit with end rot, irregular fruit, cracked fruit, netted fruit, and poorly colored fruit.

[0045] The physiological disorder determination model in this example is a trained model that has undergone machine learning using images as training data, in which tomatoes with physiological disorders are classified into two categories: abnormal fruits, and tomatoes without physiological disorders are classified into normal fruits. The physiological disorder determination model classifies tomatoes from a photographed image into normal fruits and abnormal fruits. In the example shown in Figure 4, normal tomatoes are extracted from the photographed image. A bounding box indicating the image area of ​​the extracted fruit is superimposed on the image. In addition, text indicating that the fruit is normal is superimposed on the image. Note that if the tomato is abnormal, text indicating that the fruit is abnormal is superimposed on the image.

[0046] A worker or cultivation manager who sees the image shown in Fig. 4 can determine that the tomato fruit is normal and that no action such as removal is necessary. If the tomato fruit is determined to be abnormal, the worker or the like can recognize that action such as removal of the tomato is necessary. In this example, the physiological disorder determination model is configured to determine whether the fruit is normal or abnormal, but it may also be configured to determine the type of physiological disorder the fruit contained in the captured image has.

[0047] Fig. 5 is a schematic diagram for explaining the flower thinning determination model. In detail, Fig. 5 is a schematic diagram showing an example of determination using the flower thinning determination model. In Fig. 5, the crop that is the target of flower thinning determination is a tomato. The fruit thinning determination model in this example is a trained model that has undergone machine learning using training data in which images showing the positions of tomato flowers relative to the stem are classified into two categories: flowers that should be picked and flowers that should not be picked.

[0048] The fruit thinning determination model determines whether or not a tomato flower included in a photographed image is a target for picking. In the example shown in FIG. 5, the tomato flower included in the photographed image is determined to be a target for picking. A bounding box indicating the image area of ​​the flower that is the target for picking is superimposed on the image. In addition, a text display indicating that the tomato flower is a target for picking is superimposed on the image. Note that if the tomato flower is not a target for picking, a text display indicating that the tomato flower is not a target for picking may be superimposed on the image, or no text display may be displayed. A worker or cultivation manager looking at the image shown in FIG. 5 can easily determine that the tomato flower is a target for picking.

[0049] FIG. 6 is a schematic diagram for explaining the missing leaf determination model. In detail, FIG. 6 is a schematic diagram showing an example of determination using the missing leaf determination model. In FIG. 6, the crop that is the target of missing leaf determination is a tomato. The missing leaf determination model in this example is a trained model that has been subjected to machine learning using training data that classifies images taken of a tomato cultivation site into cases where leaf removal work is necessary and cases where leaf removal work is not necessary. The missing leaf determination model is a trained model that can classify whether leaf removal work is necessary or not.

[0050] As shown in FIG. 6, when the leaf-cutting determination model determines from the captured image that leaf-cutting work is necessary, it displays a identifiable point of interest based on information about the degree of contribution to the determination, such as information obtained using the well-known Gradient-weighted Class Activation Mapping (Grad-CAM). In the example shown in FIG. 6, the type of hatching superimposed on the image makes the point of interest identifiable. Note that the hatching is an example, and other methods, such as changing the color, may be used. In the example shown in FIG. 6, areas that attract a high level of attention are displayed in dark colors. When leaf-cutting work is necessary, workers or cultivation managers who view the image shown in FIG. 6 can easily determine the areas that require leaf-cutting. Note that when leaf-cutting work is not necessary, a configuration may be adopted in which, for example, a text message indicating that there are no areas that require leaf-cutting work is displayed.

[0051] Next, a description will be given of the flow of generating work support information in the information processing device 1. Fig. 7 is a flowchart showing the flow of generating work support information in the information processing device 1. In this embodiment, the processing shown in Fig. 7 starts when a worker puts on the worker terminal 2 configured as wearable glasses on his / her face and turns on the power.

[0052] In step S1, the processing unit 11 performs a moving / stopping determination. In the moving / stopping determination, it is determined whether the worker is in a moving state or a stopped state. Specifically, the processing unit 11 acquires information from the sensor 22 included in the worker terminal 2 worn by the worker, and determines whether the worker is in a moving state or a stopped state. When the moving / stopping determination determines whether the worker is in a moving state or a stopped state, the process proceeds to the next step S2. Before describing the process of step S2, a specific example of the moving / stopping determination will be described.

[0053] FIG. 8 is a diagram illustrating a specific example of a movement / stop determination. FIG. 8 includes a first graph G1 represented by black dots and a second graph G2 represented by white dots. The horizontal axes of both the first graph G1 and the second graph G2 represent time and are common to both. The vertical axis of the first graph G1 represents the acceleration sensor value. Note that the acceleration sensor value has values ​​along the x-axis, y-axis, and z-axis. However, for simplicity, FIG. 8 shows only the x-axis value, assuming no movement along the y-axis or z-axis. Furthermore, although the first graph G1 is plotted every second, in reality, sampling is performed a predetermined number of times (e.g., 50 to 100 times) per second. The vertical axis of the second graph G2 represents the percentage of times determined to be "moving" in the past second.

[0054] The determination of whether or not the second graph G2 indicates "movement" is performed as follows: The difference between the acquired acceleration sensor value and the previously acquired value is calculated. Specifically, the difference is calculated as the sum of the squares of the difference values ​​on the x-axis, y-axis, and z-axis. If the acquired values ​​contain abnormal values, the difference value may be calculated after performing abnormal value processing such as excluding them using a predetermined value or rounding off the average value, or smoothing processing. If the difference value exceeds the predetermined value, it is determined to be "movement." If the difference value is equal to or less than the predetermined value, it is determined to be "stop." The predetermined value is an arbitrary value determined, for example, by conducting experiments. As described above, multiple acceleration sensor values ​​are obtained per second. By determining whether or not each of these values ​​indicates "movement," the proportion of times the value was determined to be "movement" in the past second can be calculated.

[0055] When a state is stopped and the proportion of times a state is determined to be "moving" in the past second exceeds a first threshold Th1, the state is determined to have changed from a stopped state to a moving state. When a state is moving and the proportion of times a state is determined to be "moving" in the past second falls below a second threshold Th2, the state is determined to have changed from a moving state to a stopped state. The first threshold Th1 and the second threshold Th2 are arbitrary values ​​determined, for example, by conducting experiments. The first threshold Th1 and the second threshold Th2 may be the same value or different values. Although an acceleration sensor is shown as an example, the present invention is not limited to this, and the threshold may be configured using at least one of a gyro sensor value, a vibration sensor value, a speed sensor value, and a camera image. Furthermore, the state may be determined to be "moving" when the sensor value or its absolute value exceeds a predetermined value, regardless of the difference from the previously acquired value.

[0056] In step S2, the processing unit 11 determines whether or not the moving / stopped state has been determined to be a stopped state in the moving / stopped state determination. If it has been determined to be a stopped state (Yes in step S2), the processing unit 11 proceeds to the next step S3. On the other hand, if it has been determined to be a moving state (No in step S2), the processing unit 11 determines not to generate work support information. That is, the processing unit 11 temporarily ends the work shown in FIG. 7. After temporarily ending the processing shown in FIG. 7, the processing unit 11 resumes it at a predetermined timing.

[0057] In step S3, the processing unit 11 acquires the captured image taken by the camera 23. In this embodiment, the camera 23 captures images only when it is determined that the worker is stationary, and does not capture images when it is determined that the worker is moving. This reduces the processing load of the captured information captured by the camera 23. For example, it reduces the communication load between the information processing device 1 and the worker terminal 2. Note that the camera 23 may be configured to capture images constantly, and the captured images may be constantly acquired by the processing unit 11. With this configuration, detailed information about the worker's movements can be acquired. After acquiring the captured images, the processing unit 11 proceeds to the next step S4.

[0058] In step S4, the processing unit 11 performs inference using the trained model 121, using the acquired photographed image. The processing unit 11 performs processes such as ripeness determination, physiological disorder determination, fruit thinning determination, flower thinning determination, leaf loss determination, and pest and disease detection, using the acquired photographed image. Which process is to be performed is predetermined by the selection of the operator. The operator can select multiple types of process. For example, the operator can perform ripeness determination and physiological disorder determination simultaneously. When the processing unit 11 completes the inference, it proceeds to the next step S5.

[0059] In step S5, the processing unit 11 determines whether or not there is an object necessary for generating work support information. In this embodiment, this determination is made using the result of inference by the trained model 121. For example, if a ripeness determination model is selected as the trained model 121 and the inference result shows that no flowers or fruits of the crop are extracted, the processing unit 11 determines that there is no object because work support information cannot be created. On the other hand, if the inference result shows that flowers or fruits of the crop are extracted, the processing unit 11 determines that there is an object because work support information can be created. Note that a trained model other than the trained model 121 used to generate the work support information may be used in the process of determining whether or not there is an object.

[0060] If it is determined that the target object is present (Yes in step S5), the process proceeds to the next step S6. If it is determined that the target object is not present (No in step S5), the processing unit 11 determines that work support information will not be generated, and temporarily ends the work shown in FIG. 7. With this configuration, if there is no target object, work support information is not displayed on the transparent display unit 2a in front of the worker, and the worker's field of vision can be secured. If the worker stops in a place where there are no crops, unnecessary information is not displayed on the transparent display unit 2a, and the worker can properly secure his or her field of vision.

[0061] In step S6, the processing unit 11 determines whether or not there is a dangerous object. The dangerous object is, for example, a work tool with a blade such as scissors. In this embodiment, the determination is made using the result of inference by the trained model 121. For this purpose, a detection function for detecting dangerous objects is specially added to the trained model 121. However, the process of determining whether or not there is a dangerous object may use a trained model other than the trained model 121 used to generate the work support information.

[0062] If it is determined that there is no dangerous object (No in step S6), the process proceeds to the next step S7. If it is determined that there is a dangerous object (Yes in step S6), the processing unit 11 determines that work support information will not be generated, and temporarily ends the work shown in FIG. 7. With this configuration, if there is a dangerous object, it is possible to configure the transparent display unit 2a in front of the worker not to display the work support display. In other words, it is possible to ensure the worker's field of vision and make it easier for the worker to avoid the dangerous object.

[0063] The processing order of steps S5 and S6 may be reversed, and at least one of steps S5 and S6 may not be provided.

[0064] In step S7, the processing unit 11 generates work support information. The work support information is, for example, a work support image in which instruction information for the worker is added to an image captured by the camera 23. The instruction information is information obtained by the inference result using the trained model 121. The instruction information may be, for example, frame information such as a bounding box, or text information indicating the inference result. The work support information may consist of only instruction information. A configuration may be adopted in which the work support display displayed on the display device 24 is the above-mentioned work support image, and the work support information is instruction information. In this case, the work support image may be generated on the worker terminal 2 side that acquires the instruction information, which is work support information. With such a configuration, it is not necessary to transmit image information from the information processing device 1 to the worker terminal 2, thereby reducing the communication burden between the information processing device 1 and the worker terminal 2. After generating the work support information, the processing unit 11 proceeds to the next step S8.

[0065] In step S8, the processing unit 11 outputs the work support information to the worker terminal 2. The worker terminal 2 that has acquired the work support information from the processing unit 11 displays a work support display on the display device 24. Details of the work support display on the display device 24 will be described later. When the processing unit 11 completes the output process, the process proceeds to the next step S9.

[0066] In step S9, the processing unit 11 waits for a predetermined time. The predetermined time is, for example, several seconds, but may be any time determined through experiments, etc. After the predetermined time has elapsed, the processing unit 11 returns to step S1 and performs the processes from step S1 onwards again.

[0067] By providing the process of step S9, the display device 24 displays the next work support display a predetermined time after displaying the previous work support display. This makes it possible to prevent the work support display displayed by the display device 24 from being updated too frequently. As a result, it is possible to reduce the burden on the worker to recognize the display contents of the work support display. It is also possible to prevent delays in displaying the work support display caused by the accumulation of processes.

[0068] It is preferable that the predetermined time given as the waiting time be changeable. By configuring it in this way, it is possible to easily accommodate both an experienced worker who wants the work support display to be updated more frequently and a novice worker who wants the work support display to be updated more slowly. In other words, the update frequency of the work support display can be changed according to the worker's request, making it a convenient configuration for the worker.

[0069] Furthermore, as can be seen from the above description, in this embodiment, the processing unit 11 generates work support information when it is determined that a predetermined condition is met based on the photographic information acquired from the camera 23, in addition to the stopped state. With this configuration, the timing for generating work support information can be narrowed down to a more appropriate timing, thereby reducing the processing load on the processing unit 11. Furthermore, the frequency with which unnecessary information is displayed on the transparent display unit 2a can be reduced, making it easier to ensure the worker's field of vision.

[0070] As described above, the predetermined condition may be at least one of the presence of an object and the absence of a dangerous object. Specifically, the object may differ depending on the type of trained model 121 selected for generating the work support information. For example, if a physiological disorder determination model is selected, the object may be fruit; if a flower thinning determination model is selected, the object may be flowers; and if a leaf loss determination model is selected, the object may be leaves and a path. The predetermined condition may also be a condition in which the worker gazes at the object for a certain period of time. Whether or not the worker has gazed at the object for a certain period of time may be determined, for example, from the change in the captured image over time.

[0071] (1-3.Display device) Next, we will explain in detail the display mode of the display device 24. Figures 9A, 9B, and 9C are schematic diagrams showing display examples of the display device 24. In detail, Figures 9A, 9B, and 9C are diagrams showing changes in the display mode on the transparent display unit 2a in response to changes in the operating state of the worker wearing the worker terminal 2 configured as wearable glasses.

[0072] FIG. 9A is a diagram showing a display mode when the worker is moving. As described above, when the worker is moving, the information processing device 1 does not generate work support information. For this reason, as shown in FIG. 9A, no work support display is displayed on the transparent display unit 2a. In other words, the worker can view the scenery through the transparent display unit 2a without being obstructed by the work support display. A wide field of view is ensured for the worker, allowing the worker to move safely.

[0073] FIG. 9B is a diagram showing the display mode immediately after the worker has changed from a moving state to a stationary state. The worker has stopped moving and is observing the crops. As described above, when the information processing device 1 detects that the worker is in a stationary state, it performs inference using the trained model 121. The information processing device 1 also checks whether there are any objects or dangerous objects. FIG. 9B corresponds to the display mode when these processes are being performed. At this point, work support information has not been generated, and as in the case of FIG. 9B, no work support display is displayed on the transparent display unit 2a.

[0074] FIG. 9C is a diagram showing a display mode when the worker is in a stopped state and a work support display is displayed when predetermined conditions are met. In this embodiment, the predetermined conditions are that both the presence of an object and the absence of a dangerous object are met. As shown in FIG. 9C, the display device 24 displays the work support display (in this example, the work support image 5) superimposed on the field of view through the transparent display unit 2a. With this configuration, the worker can easily see the work support display even when, for example, their hands are full. Furthermore, in this embodiment, the work support display is displayed only when the worker is in a stopped state, and is not displayed when the worker is moving. This reduces the frequency with which the work support display is updated, making it easier to recognize the content of the work support display.

[0075] When the processing unit 11 generates work support information, the work support information is normally received by the worker terminal 2, and the display device 24 displays a work support display. However, if the communication state is poor, the work support information may not be received and the work support display may not be displayed. In anticipation of such a case, if the work support information is not received for a certain period of time even though the conditions for generating the work support information are met, the display device 24 may be configured to display on the transparent display unit 2a that there is a communication failure. If the worker, recognizing the communication failure, takes a specific action such as nodding his head, the captured image may be reacquired and the work support information retransmitted.

[0076] In this embodiment, the work support display is a work support image in which instruction information for the worker is added to at least a portion of an image captured by the camera 23. In other words, the work support display is a work support image 5 in which instruction information 5a for the worker is added to at least a portion of an image captured of the worker's field of vision. If a bounding box or other frame or text information constituting the instruction information 5a is to be displayed superimposed directly on an object such as a crop seen through the transparent display unit 2a, it is difficult to align the display position of the instruction information 5a, and the position of the instruction information 5a is likely to be shifted. In this regard, in this embodiment, the instruction information 5a for the worker is displayed on the transparent display unit 2a superimposed on an image captured of the worker's field of vision, so that it is possible to avoid shifting from the position of the object. In other words, a display that is easy for the worker to understand can be provided.

[0077] 9C, the instruction information 5a is composed of a bounding box 5a1 and text information 5a2. However, the composition of the instruction information 5a may be changed as appropriate depending on the inference result of the trained model 121. In addition, to help the worker recognize and understand the work support image 5, other notification methods such as audio notification may be combined.

[0078] In this embodiment, the work support display (for example, the work support image 5) is displayed superimposed on a part of the field of view (the worker's field of view) through the transparent display unit 2a. As a result, even when the work support display is displayed, the worker's field of view can be ensured, and the worker can work safely. In this embodiment, as a preferred form, the work support display is displayed at a position shifted from the center of the transparent display unit 2a. This makes it possible to appropriately ensure the worker's field of view. In detail, the work support display is displayed in the upper left part of the transparent display unit 2a. However, this display position may be changed as appropriate.

[0079] It is preferable that the display range of the work assistance display on the transparent display unit 2a is adjustable. By configuring it in this way, the work assistance display can be displayed in a state that is easy for the worker to recognize and understand. To adjust the display range, for example, a remote control device 25 or a voice input device may be used.

[0080] 9A, 9B, and 9C, the display device 24 displays a marker 6 at predetermined coordinates on the transparent display unit 2a. Placing the marker 6 on the transparent display unit 2a in this manner makes it easier for the worker to focus on the marker 6. That is, it is possible to easily guide the worker's gaze to the position of the marker 6. This makes it easier to align the range that the worker focuses on with the range shown in the work support display (work support image). In this embodiment, the predetermined coordinates are the center coordinates of the transparent display unit 2a. This makes it possible to guide the worker's gaze to the center position of the transparent display unit 2a. In this embodiment, the shape of the marker 6 is a plus (+) shape, but this is merely an example, and the display mode of the marker 6 may be changed as appropriate.

[0081] Furthermore, in this embodiment, the display device 24 displays a common marker that corresponds the positional relationship between the field of view through the transparent display unit 2a and the work support display, superimposed on both the field of view through the transparent display unit 2a and the work support display. This configuration makes it easier for the worker to recognize the correspondence between his or her field of view and the work support display.

[0082] In this embodiment, among the common markers, the first marker 6a displayed superimposed in the field of view through the transparent display unit 2a is the same as the marker 6 displayed at the center coordinates of the transparent display unit 2a described above. The second marker 6b displayed superimposed on the work support display (work support image 5) has the same shape as the first marker 6. By making them the same shape, the correspondence can be made easier to understand. The position of the second marker 6b is arranged at a position corresponding to the center position of the transparent display unit 2a in the captured image. Note that the marker may be, for example, a grid line instead of the configuration (+ display) of this embodiment.

[0083] 9A, 9B, and 9C, the display device 24 displays whether the worker is in one of multiple operating states, including a stopped state. This configuration allows the worker to act while recognizing what processing the information processing device 1 will perform. For example, the worker can focus on an object or remain still so that an appropriate work assistance display can be obtained, while being aware that the information processing device 1 has determined that the worker is in a stopped state. This allows the camera 23 to capture an image with minimal blur, making the work assistance display easier to see.

[0084] In this example, an operation status display icon 7 indicating the operation status is displayed at the lower right side of the transparent display unit 2a. In Fig. 9A, an icon indicating a moving state is displayed as the operation status display icon 7. In Figs. 9B and 9C, an icon indicating a stopped state is displayed as the operation status display icon 7. The position, size, and display mode of the operation status display icon 7 are examples, and may be changed as appropriate.

[0085] In this example, a colored frame 241 is displayed around the periphery of the transparent display unit 2a. The color of the frame 241 is changed depending on whether the worker is in a stationary state or a moving state. For example, the color of the frame 241 is red in the moving state shown in FIG. 9A, and the color of the frame 241 is blue in the stationary state shown in FIGS. 9B and 9C. This configuration can achieve the same effect as the display of the operation state display icon 7. By both the color change of the frame 241 and the display of the operation state display icon 7, the worker can reliably recognize what operation state the information processing device 1 has determined.

[0086] Fig. 10 is a schematic diagram illustrating a preferred form of display on the transparent display unit 2a. As shown in Fig. 10, the display device 24 preferably displays the type of trained model 121 in use in an identifiable manner. With this configuration, when there are multiple types of work assistance displays that are displayed by using the trained model 121, the worker can perform the work while recognizing which work assistance display will be displayed. As a result, it is possible to more easily obtain an appropriate work assistance display.

[0087] In the example shown in FIG. 10, a usage model display icon 8 indicating the type of trained model 121 currently being used is displayed in the lower left corner of the transparent display unit 2a. In the example shown in FIG. 10, three types of trained models 121 are used. In detail, a ripeness determination model, a flower thinning determination model, and a leaf loss determination model are used. The number of models used, the types of models, and the display modes of the models are merely examples and may be changed as appropriate.

[0088] FIG. 11 is a diagram for explaining a detailed example of the generation process of the work support image 5. The upper part of FIG. 11 is a diagram showing the inference result using the ripeness determination model. As shown in the figure, the inference using the ripeness determination model is performed for the entire shooting range of the camera 23. The lower part of FIG. 11 shows the work support image 5.

[0089] 11, the work support image 5 is not configured to directly display the inference result of the maturity assessment model. The work support image 5 is configured to be generated using an image that is an extracted portion of the image captured by the camera 23. By generating the work support image 5 by extracting a portion of the image captured by the camera 23 in this way, the work support image 5 can be brought closer to the user's field of vision.

[0090] It is preferable that the image range R extracted from the captured image used to generate the work support image 5 is adjustable. This configuration allows the work support image 5 to be brought closer to the user's field of vision, making it easier to understand the work support image 5. The image range R may be adjusted using, for example, a remote control device 25 or a voice input device.

[0091] 11, some of the results inferred by the maturity assessment model are intentionally excluded when generating the work support image 5. In detail, the inference results that exist outside a certain range from the center of the work support image 5 are intentionally not displayed. Here, the inference results that are outside a certain range from the center are excluded from display. However, the present invention is not limited to this. For example, the inference results may be excluded from display when they exceed a certain quantity, belong to an unnecessary classification class, or have a low assessment score.

[0092] That is, the instruction information 5a in the work support image 5 may be configured to be displayed by narrowing it down to a specific range, a specific number, a specific classification class, and a specific judgment score. That is, the instruction information 5a in the work support image 5 may be configured to be displayed by narrowing it down to a predetermined condition. With such a configuration, the information in the work support image 5 can be narrowed down to an appropriate amount, and the burden on the worker when recognizing the work support image 5 can be reduced. That is, the work efficiency of the worker can be improved. The specific range, the specific number, the specific classification class, and the specific judgment score may be appropriately determined through experiments, etc.

[0093] 2. Second Embodiment Fig. 12 is a block diagram showing a schematic configuration of a work support system 200 according to a second embodiment of the present invention. As shown in Fig. 12, the work support system 200 includes an information processing device 1A, a worker terminal 2A, a work information generating device 3, and a manager terminal 4. Note that, like the work support system 100 according to the first embodiment, the work support system 200 according to the second embodiment is a system suitable for providing work support for, for example, agricultural work.

[0094] The information processing device 1A has a configuration generally similar to that of the information processing device 1 of the first embodiment. For this reason, the description of the information processing device 1A will be omitted for parts that overlap with the first embodiment, and will focus on the differences. The worker terminal 2A has a configuration similar to that of the worker terminal 2 of the first embodiment. For this reason, a description of the worker terminal 2A will be omitted unless particularly necessary. Note that when the information processing device 1A and the worker terminal 2A are configured as separate devices, the number of worker terminals 2A communicatively connected to the information processing device 1A is not limited to one, and may be multiple. In this embodiment as well, the number of trained models 121A stored in the storage unit 12A is multiple, but may be single.

[0095] The information processing device 1A is communicably connected to the work information generation device 3 and the manager terminal 4 via a communication network 300 such as the Internet. The work information generation device 3 and the manager terminal 4 are communicably connected to each other via the communication network 300. The information processing device 1A and the manager terminal 4 may also be communicably connected to each other via a LAN such as a wireless LAN.

[0096] The work information generation device 3 is, for example, a server device such as a cloud server. The work information generation device 3 is configured to include, for example, a computing device such as a CPU, RAM, and ROM. The administrator terminal 4 is, for example, a terminal device owned by the administrator, such as a personal computer, tablet terminal, or smartphone. The work information generation device 3 may be included in the administrator terminal 4. In this case, the information processing device 1A and the administrator terminal 4 may be configured to be able to communicate with each other via a wireless LAN such as Wi-Fi.

[0097] The processing unit 11A of the information processing device 1A outputs work support information to a worker terminal 2A carried by the worker and to a work information generation device 3 that generates work information for the worker. As in the first embodiment, the processing unit 11A outputs the work support information to the worker terminal 2A. In this embodiment, the processing unit 11A outputs the work support information not only to the worker terminal 2A but also to the work information generation device 3. This configuration makes it possible for the worker to share work support information including the inference results from the trained model 121A with a manager or an expert.

[0098] Furthermore, in this embodiment, the processing unit 11A outputs, in addition to the work support information, information on the trained model 121A used to generate the work support information to the work information generation device 3. With this configuration, the work information generation device 3 can appropriately estimate what work the worker has performed, and can appropriately generate a work record of the worker.

[0099] In detail, the processing unit 11A outputs, for example, information on the trained model 121A used by the worker, such as a ripeness determination model, a physiological disorder determination model, a flower thinning determination model, a leaf loss determination model, a disease and pest detection model, etc. The work information generation device 3, which has acquired the information on the trained model 121A used, estimates the work content based on the information and records the work.

[0100] For example, when the work information generating device 3 receives information that a ripeness determination model was used, it infers that harvesting work was performed. For example, when the work information generating device 3 receives information that a physiological disorder determination model was used, it infers that fruit thinning work was performed. For example, when the work information generating device 3 receives information that a flower thinning determination model was used, it infers that flower thinning work was performed. For example, when the work information generating device 3 receives information that a leaf loss determination model was used, it infers that leaf thinning and pruning work was performed. For example, when the work information generating device 3 receives information that a ripeness determination model, a physiological disorder determination model, and a pest detection model were used simultaneously, it infers that patrol work was performed.

[0101] Furthermore, in this embodiment, the processing unit 11A outputs the photographing information and work time information acquired from the worker terminal 2A to the work information generation device 3, in addition to the work support information and information on the used trained model 121A. In addition, various types of information acquired by the sensor 22 or the like may also be output to the work information generation device 3. The work time information may include the work start time and the work end time. With this configuration, the work information generation device 3 can generate a more detailed work record.

[0102] The processing unit 11A may be configured to collectively output the work support information, information on the used trained model 121A, the photography information, and the work time information to the work information generation device 3, for example, when the work for the day is completed. The processing unit 11A may also be configured to periodically output the work support information, etc. to the work information generation device 3 during the work for the day. The work time information may also be directly sent from the information processing device 1A to the administrator terminal 4 using email, etc. For example, the work time information may be automatically sent from the information processing device 1A to the administrator terminal 4 when the work is completed. This configuration allows the administrator to immediately know when the worker has completed the work.

[0103] By using the administrator terminal 4, the administrator can view the work information generated by the work information generation device 3 on a screen or by printing it out. The administrator can also save the work information generated by the work information generation device 3 in the administrator terminal 4. The administrator can also process the work information generated by the work information generation device 3 using the administrator terminal 4.

[0104] FIG. 13 is a diagram showing an example of a work report 31 generated by the work information generating device 3. The work report 31 is an example of the above-mentioned work information. In the example shown in FIG. 13, the work report includes the work date, work start time, work end time, worker, work location, crop to be worked on, and work content. All of this information can be generated using information acquired from the information processing device 1A. The worker listed in the work report 31 is identified, for example, using a user ID or the like transmitted from the worker terminal 2A to the information processing device 1A. The work location is identified, for example, using the worker's location information transmitted from the worker terminal 2A to the information processing device 1A. The crop to be worked on and the breakdown of the work are determined, for example, based on the inference results of the trained model 121A. The "JD" shown in the work breakdown refers to ripeness. For example, JD10 indicates that the ripeness is 10 out of 10 and that the crop is ready for harvesting.

[0105] FIG. 14 is a diagram showing an example of an abnormal area record 32 generated by the work information generating device 3. The abnormal area record 32 is an example of the above-mentioned work information. The example shown in FIG. 14 shows six cultivation shelves 322 arranged in a greenhouse 321 for cultivating crops. In FIG. 14, black dots shown above or below the cultivation shelves 322 indicate the movement trajectory 323 of the worker. Furthermore, a hollow circle 324 displayed superimposed on the work trajectory 323 indicates that an abnormality has been found. In detail, the circle 324 indicates that an abnormality has been found in the vicinity of the location of the cultivation shelf 322 where the circle 324 is indicated.

[0106] The abnormality is an abnormality detected by the trained model 121A. When a physiological disorder determination model is used as the trained model 121A, the circle 324 indicates that an abnormal fruit such as end rot has been detected. When a pest detection model is used as the trained model 121A, the circle 324 indicates that a pest has been detected. Note that when an abnormality is detected, if an operator performs work such as removing the abnormal fruit, the fact that the removal work has been performed may be included in the record.

[0107] FIG. 15 is a diagram showing a modified example of the abnormal area record 32 shown in FIG. 14. The modified abnormal area record 32A uses a mapping format in which the abnormality level is classified into multiple levels according to the number of abnormalities occurring per specified area, and the distribution of the abnormality level is shown for each cultivation shelf 322A in the greenhouse 321A. In the example shown in FIG. 15, the abnormality level is classified into three levels, but this is merely an example, and the abnormality level may be classified into two levels, four levels, or more. The example shown in FIG. 15 is suitable for showing, for example, the occurrence of physiologically disordered fruit or pests, or the occurrence of areas requiring leaf chipping.

[0108] FIG. 16 is a diagram showing an example of a work history table 33 generated by the work information generation device 3. The work history table 33 is an example of the work information described above. The work history table 33 is a table that shows the work performed on each day for a certain number of days. The type of work performed on each day is estimated based on the type of trained model 121A used, as described above. The parts indicated by bold lines indicate that the corresponding work was performed.

[0109] In the example shown in FIG. 16, there are five types of work: defoliation, training, fruit thinning, pest control, and harvesting. However, this is merely an example, and the types of work may be changed as appropriate. In the example shown in FIG. 16, defoliation and training are performed on date X. On date Y, defoliation, training, and harvesting are performed. By looking at work history table 33, managers and the like can easily understand what work was performed on each day and can also grasp trends regarding what work is performed at what time. In other words, by looking at work history table 33, managers and the like can easily create work plans.

[0110] FIG. 17 is a diagram showing an example of a cultivation environment monitoring table 34 generated by the work information generating device 3. The cultivation environment monitoring table 34 is an example of the above-mentioned work information. The cultivation environment monitoring table 34 can be created, for example, by using the inference results obtained from the leaf loss determination model. In the example shown in FIG. 17, the degree of need for leaf loss is determined for each cultivation shelf every week based on the inference results of the leaf loss determination model, and the corresponding date and cultivation shelf are hatched according to the determined degree of need.

[0111] In the example shown in Figure 17, the degree of necessity for leaf cutting is classified into five levels. For ease of explanation, the levels are listed in order of decreasing leaf cutting necessity as "1," "2," "3," "4," and "5." Level "5" represents the state in which leaf cutting is most necessary, and corresponds to "leaves to be cut" in Figure 17.

[0112] As an example, we will focus on cultivation shelves A, B, and C. On date S, cultivation shelves A to C all have a leaf thinning necessity level of "1." On date "T," one week after date S, cultivation shelves A to C all have a leaf thinning necessity level of "2." On date "U," two weeks after date T, cultivation shelf A has a leaf thinning necessity level of "2," and cultivation shelves B and C have a leaf thinning necessity level of "3." On date V, three weeks after date U, cultivation shelf A has a leaf thinning necessity level of "4," and cultivation shelves B and C have a leaf thinning necessity level of "5."

[0113] By looking at the cultivation environment monitoring table 34, managers can grasp the time-series changes in the need for leaf thinning. Managers can then predict whether leaf thinning will be necessary in the future. By being able to predict whether leaf thinning will be necessary, they can appropriately decide on the date for work and prepare workers.

[0114] <3. Things to keep in mind> Various modifications can be made to the various technical features disclosed in this specification without departing from the spirit of the technical creation. Furthermore, multiple embodiments and modifications shown in this specification can be combined to the extent possible. [Explanation of symbols]

[0115] 1, 1A... Information processing device 2, 2A... Operator terminal 2a Transparent display section 3...Work information generation device 5. Work support images 5a...Instruction information 6 Markers 6a First marker (common marker) 6b Second marker (common marker) 11, 11A Processing section 12, 12A...Storage section 22 Sensor 23. Camera 24...Display device 31. Work report (work information) 32, 32A: Abnormal area record (work information) 33 Work history table (work information) 34. Cultivation environment monitoring table (work information) 100, 200... Work support system (agricultural work support system) 121, 121A···Pre-trained model

Claims

1. An information processing device that generates agricultural work support information that supports agricultural work, a processing unit for acquiring movement information of a farm worker, The information processing device wherein the processing unit generates the farm work support information when it is determined that the farm worker is in a stopped state.

2. The information processing device according to claim 1 , wherein the processing unit generates the farm work support information based on photographic information acquired from a camera.

3. The information processing device according to claim 2 , wherein the processing unit generates the farm work support information when it is determined that a predetermined condition is met based on the photographic information acquired from the camera in addition to the stopped state.

4. The information processing device according to claim 1 , wherein the processing unit outputs the farm work support information to a terminal.

5. The information processing device according to claim 4 , wherein the processing unit outputs the farm work support information to a farm work information generating device that generates work information for the farm worker.

6. Further comprising a storage unit that stores at least one trained model by machine learning, The information processing device according to claim 5 , wherein the processing unit outputs information about the trained model used to generate the farm work support information to the farm work information generation device.

7. An information processing device according to any one of claims 1 to 6; a display device that displays a farm work support display based on the farm work support information; A farming support system equipped with:

8. a camera used to generate the farm work support information; The farm work support system according to claim 7 , wherein the farm work support display is a farm work support image in which instruction information for the farm worker is added to at least a part of an image captured by the camera.

9. The farm work support system according to claim 8 , wherein the instruction information displayed on the farm work support image is limited to a predetermined condition.

10. 10. The farm work support system according to claim 8, wherein an image range extracted from the captured image used for generating the farm work support image for use in generating the farm work support image is set to be adjustable.

11. The farm work support system according to claim 7 , wherein the display device is included in a terminal.

12. The terminal has a transparent display unit that is placed in front of the farm worker, The farm work support system according to claim 11, wherein the display device displays the farm work support display superimposed on the field of view through the transparent display unit.

13. The farm work support system according to claim 12, wherein the farm work support display is displayed superimposed on a part of the field of view through the transparent display unit.

14. The farm work support system according to claim 13 , wherein a display range of the farm work support display on the transparent display unit is adjustable.

15. 15. The agricultural work support system according to claim 12, wherein the display device displays a common marker that corresponds to the positional relationship between the field of view through the transparent display unit and the agricultural work support display, superimposed on each of the field of view through the transparent display unit and the agricultural work support display.

16. The farm work support system according to claim 12 , wherein the display device displays a marker at a predetermined coordinate on the transparent display portion.

17. The farm work support system according to claim 7 , wherein the display device displays whether the farm worker is in a plurality of operating states including the stopped state.

18. The information processing device further includes a storage unit that stores at least one trained model by machine learning, and generates the farm work support information using the trained model; The agricultural work support system according to claim 7 , wherein the display device identifiably displays the type of the trained model being used.

19. the display device displays the next farm work support display after a predetermined time has elapsed since the farm work support display was displayed, The farm work support system according to claim 7 , wherein the predetermined time is set to be changeable.

Images