Processing system, server, and terminal device

Abstract

To provide a processing system, a server, and a terminal device for improving convenience of the processing system.SOLUTION: A processing system 1 includes an image acquisition section 141, a first grouping section 100, and a representative image acquisition section 131. The image acquisition section 141 acquires a plurality of first target images in which a plurality of treatment targets is respectively captured. The first grouping section 100 performs treatment target grouping for dividing the plurality of treatment targets into a plurality of treatment target groups on the basis of the plurality of first target images. The representative image acquisition section 131 acquires a plurality of representative images respectively representing the plurality of treatment target groups.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to processing systems. [Background technology]

[0002] Patent Documents 1 and 2 disclose systems for processing images containing crops. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-157767 [Patent Document 2] Japanese Patent Application Publication No. 2022-62294 Summary of the Invention [Problem to be solved by the invention]

[0004] There is room for improvement in the processing system.

[0005] Therefore, the present disclosure has been made in consideration of the above points, and aims to provide a technique that can improve the convenience of a processing system. [Means for solving the problem]

[0006] One aspect of the processing system includes a target image acquisition unit, a first grouping unit, and a representative image acquisition unit. The target image acquisition unit acquires a plurality of first target images, each of which depicts a plurality of treatment targets. The grouping unit performs treatment target grouping, dividing the plurality of treatment targets into a plurality of treatment target groups, based on the plurality of first target images. The representative image acquisition unit acquires a plurality of representative images, each of which represents a plurality of treatment target groups.

[0007] Furthermore, one aspect of the processing system is the above processing system, and includes a terminal device having an image acquisition unit, and a server having a representative image acquisition unit.

[0008] One aspect of the server is a server included in the above processing system.

[0009] One aspect of the terminal device is a terminal device included in the processing system. [Effects of the Invention]

[0010] The convenience of the processing system is improved. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of the configuration of a processing system. [Figure 2] FIG. 2 is a schematic diagram illustrating an example of the configuration of a terminal device. [Figure 3] FIG. 2 is a schematic diagram illustrating an example of a configuration of a server. [Figure 4] FIG. 1 is a schematic diagram illustrating an example of the configuration of a processing system. [Figure 5] FIG. 1 is a schematic diagram illustrating an example of the operation of a processing system. [Figure 6] FIG. 2 is a schematic diagram showing an example of a captured image. [Figure 7] FIG. 2 is a schematic diagram illustrating an example of a target image. [Figure 8] FIG. 2 is a schematic diagram illustrating an example of the configuration of a feature amount acquisition unit. [Figure 9] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 10] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 11] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 12] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 13] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 14] FIG. 10 is a schematic diagram illustrating an example of a treatment target map. [Figure 15] FIG. 10 is a schematic diagram illustrating an example of a treatment target map. [Figure 16]FIG. 10 is a schematic diagram illustrating an example of a treatment target map. [Figure 17] FIG. 10 is a schematic diagram illustrating an example of a treatment target map. [Figure 18] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 19] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 20] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 21] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 22] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 23] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 24] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 25] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 26] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 27] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 28] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 29] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 30] FIG. 10 is a schematic diagram illustrating an example of a feature amount map. [Figure 31] FIG. 1 is a schematic diagram illustrating an example of the configuration of a processing system. [Figure 32] FIG. 1 is a schematic diagram illustrating an example of the configuration of a processing system. [Figure 33] FIG. 1 is a schematic diagram illustrating an example of the operation of a processing system. [Figure 34] FIG. 1 is a schematic diagram illustrating an example of the configuration of a processing system. [Figure 35] FIG. 1 is a schematic diagram illustrating an example of the operation of a processing system. [Figure 36] FIG. 10 is a schematic diagram illustrating an example of a cautionary feature map. [Figure 37] FIG. 10 is a schematic diagram illustrating an example of a cautionary feature map. [Figure 38] 10 is a schematic diagram for explaining an example of the operation of a caution-required determination unit. FIG. [Figure 39] 10 is a schematic diagram for explaining an example of the operation of a caution-required determination unit. FIG. [Figure 40] 10 is a schematic diagram for explaining an example of the operation of a caution-required determination unit. FIG. [Figure 41] FIG. 1 is a schematic diagram illustrating an example of the configuration of a processing system. [Figure 42] FIG. 1 is a schematic diagram illustrating an example of the configuration of a processing system. [Figure 43] FIG. 1 is a schematic diagram illustrating an example of the configuration of a processing system. [Figure 44] FIG. 1 is a schematic diagram illustrating an example of the operation of a processing system. DETAILED DESCRIPTION OF THE INVENTION

[0012] <Example of processing system configuration> FIG. 1 is a schematic diagram showing an example of a processing system 1. As shown in FIG. 1, the processing system 1 includes, for example, a server 3 and a terminal device 4. The terminal device 4 is located, for example, in a remote location relative to the server 3. The server 3 and the terminal device 4 are connected to, for example, a network 2. The network 2 may include, for example, the Internet or a LAN. LAN is an abbreviation for Local Area Network. The server 3 and the terminal device 4 can communicate with each other through the network 2. The terminal device 4 is also called, for example, an edge device. The processing system 1 may include multiple terminal devices 4.

[0013] The terminal device 4 can acquire, for example, a target image showing a treatment target on which some treatment is to be performed. The server 3 can perform processing using the target image acquired by the terminal device 4.

[0014] The terminal device 4 is used, for example, in a farm where a large number of crops (also referred to as agricultural products) of the same variety are grown. The terminal device 4 may be used in a farm where fruits of the same variety (e.g., grapes of the same variety or strawberries of the same variety) are grown, or in a farm where vegetables of the same variety (e.g., Chinese cabbage of the same variety or cabbage of the same variety) are grown, or in a farm where other crops of the same variety are grown. The terminal device 4 is capable of acquiring a target image of a crop to be treated in the farm. The crop shown in the target image acquired by the terminal device 4 may be a fruit such as grapes or strawberries, a vegetable such as Chinese cabbage or cabbage, or another agricultural product. The terminal device 4 is capable of acquiring multiple target images, each showing multiple crops of the same variety grown in the farm. Hereinafter, the farm where the terminal device 4 is used may be referred to as the target farm.

[0015] The terminal device 4 is connected to, for example, a camera 5 and a position information acquisition unit 6 which are separate components from the processing system 1. The terminal device 4, the camera 5, and the position information acquisition unit 6 are mounted, for example, on a vehicle 10 which travels within the target farm. Therefore, the terminal device 4, the camera 5, and the position information acquisition unit 6 can move together with the vehicle 10. The vehicle 10 may be capable of autonomous driving, may be remotely controlled, or may be directly operated by a person. An autonomously driving vehicle is also called, for example, a rover. The camera 5 and the position information acquisition unit 6 may constitute part of the processing system 1.

[0016] The camera 5 is capable of photographing crops in the target farm. The camera 5, for example, can continuously photograph while the vehicle 10 is traveling, and photograph all of the crops in the target farm. The vehicle 10 travels within the target farm so that the camera 5 can photograph all of the crops in the target farm. Each photographed image 50 generated by the camera 5 is input to the terminal device 4. The multiple photographed images 50 input to the terminal device 4 capture all of the crops in the target farm. The photographed image 50 may be, for example, a color image. As will be described later, the terminal device 4 cuts out the portion of the photographed image 50 that captures the crops, and acquires a target image in which the crops are captured in a large size.

[0017] The position information acquisition unit 6 can acquire position information 60 (also referred to as vehicle position information 60) that indicates the absolute position of the vehicle 10. The position information acquisition unit 6 repeatedly acquires the vehicle position information 60 while the vehicle 10 is traveling. The position information acquisition unit 6 can acquire the vehicle position information 60 based on, for example, a received signal received from a GNSS satellite (for example, a GPS satellite). GNSS is an abbreviation for Global Navigation Satellite System. GPS is an abbreviation for Global Positioning System.

[0018] Each vehicle position information 60 acquired by the position information acquisition unit 6 is input to the terminal device 4. Based on each vehicle position information 60 from the position information acquisition unit 6 and each captured image 50 from the camera 5, the terminal device 4 acquires position information (also referred to as crop position information) indicating the absolute position of each crop in the target farm. For example, the terminal device 4 transmits to the server 3 the position information of multiple crops in the target farm and multiple target images each capturing the multiple crops in the target farm. The server 3 performs processing using the position information of multiple crops in the target farm and the multiple target images each capturing the multiple crops.

[0019] The server 3 can also accept input of treatment details for crops from a user. The user of the server 3 may be, for example, an expert in growing crops. The server 3 notifies the terminal device 4 of the treatment details input by the user (for example, the expert).

[0020] The user of the terminal device 4 is, for example, a worker who performs treatment on crops in a target farm. The terminal device 4 displays the treatment content notified from the server 3. This allows the worker to perform treatment on each crop in accordance with the treatment content displayed on the terminal device 4. The crops or treatment targets can also be referred to as, for example, samples.

[0021] The terminal device 4, camera 5, and location information acquisition unit 6 do not have to be mounted on the vehicle 10. In this case, the camera 5 may be able to photograph all of the crops in the target farm by having a person carrying the terminal device 4, camera 5, and location information acquisition unit 6 move around the target farm.

[0022] <Example of terminal device configuration> Fig. 2 is a schematic diagram showing an example of the configuration of the terminal device 4. As shown in Fig. 2, the terminal device 4 includes a control unit 40, a storage unit 41, interfaces 42, 45, and 46, a display unit 47, and an input unit 48. The terminal device 4 is, for example, a computer device.

[0023] The interface 42 is capable of communicating with the network 2. The interface 42 may be referred to as, for example, an interface circuit, a communication unit, or a communication circuit. The interface 42 is capable of communicating with the server 3 through the network 2. The interface 42 may perform wired communication with the network 2 or wireless communication. The interface 42 may be capable of performing wireless communication conforming to Wi-Fi, for example, or may be capable of performing communication conforming to other standards. The interface 42 inputs information received from the network 2 to the control unit 40. The interface 42 also transmits information from the control unit 40 to the network 2.

[0024] The interface 45 is capable of communicating with the camera 5. The interface 45 may also be referred to as, for example, an interface circuit, a communication unit, or a communication circuit. The interface 45 may communicate with the camera 5 via wired or wireless communication. The interface 45 receives the captured image 50 from the camera 5 and outputs the received captured image 50 to the control unit 40.

[0025] The interface 46 is capable of communicating with the position information acquisition unit 6. The interface 46 may also be referred to as, for example, an interface circuit, a communication unit, or a communication circuit. The interface 46 may communicate with the position information acquisition unit 6 via wired or wireless communication. The interface 46 receives vehicle position information 60 from the position information acquisition unit 6 and outputs the received vehicle position information 60 to the control unit 40.

[0026] The control unit 40 can comprehensively manage the operation of the terminal device 4 by controlling the other components of the terminal device 4. The control unit 40 can also be called, for example, a control circuit. The control unit 40 includes, for example, at least one processor. The control unit 40 may include, for example, a CPU (Central Processing Unit).

[0027] The storage unit 41 may include a non-transitory recording medium readable by the CPU of the control unit 40, such as a read-only memory (ROM) and a random access memory (RAM). The storage unit 41 stores, for example, a program 41a for controlling the terminal device 4. Various functions of the control unit 40 are realized, for example, by the CPU of the control unit 40 executing the program 41a in the storage unit 41.

[0028] The configuration of the control unit 40 is not limited to the above example. For example, the control unit 40 may include multiple CPUs. The control unit 40 may also include at least one DSP (Digital Signal Processor). All or some of the functions of the control unit 40 may be realized by a hardware circuit that does not require software to realize the function. The storage unit 41 may also include a computer-readable non-transitory recording medium other than ROM and RAM. The storage unit 41 may also include, for example, a small hard disk drive and an SSD (Solid State Drive).

[0029] The input unit 48 can accept various inputs from the user. The input unit 48 may include, for example, a mouse and a keyboard. The input unit 48 may also include a touch sensor that accepts touch operations by the user. The input unit 48 may also include an audio input unit that accepts audio input from the user. The audio input unit may include, for example, a microphone. The control unit 40 can identify the content of the input accepted by the input unit 48 based on the output signal from the input unit 48.

[0030] The display unit 47 is capable of displaying various types of information under the control of the control unit 40. The display unit 47 has a display surface that displays various types of information. The display unit 47 may be, for example, a liquid crystal display, an organic electro-luminescence (EL) display, or a plasma display. Furthermore, if the input unit 48 is equipped with a touch sensor, the touch sensor and the display surface of the display unit 47 may constitute a touch panel display having a display function and a touch detection function. In this case, the input unit 48 can detect a touch operation on the display surface of the display unit 47. The input unit 48 and the display unit 47 constitute a user interface.

[0031] The terminal device 4 may be a portable device that can be carried by a user, or a stationary device that is installed in the vehicle 10. The terminal device 4 may be a tablet device, a mobile phone such as a smartphone, a notebook or desktop personal computer, or a wearable device such as smart glasses. At least one of the camera 5 and the location information acquisition unit 6 may be included in the terminal device 4.

[0032] <Server configuration example> Fig. 3 is a schematic diagram showing an example of the configuration of the server 3. As shown in Fig. 3, the server 3 includes a control unit 30, a storage unit 31, an interface 32, a display unit 37, and an input unit 38. The server 3 is, for example, a computer device.

[0033] The interface 32 is capable of communicating with the network 2. The interface 32 may also be referred to as, for example, an interface circuit, a communication unit, or a communication circuit. The interface 32 is capable of communicating with the interface 42 of the terminal device 4 through the network 2. The interface 32 may perform wired communication with the network 2 or wireless communication. The interface 32 may be capable of performing wireless communication conforming to Wi-Fi, for example, or may be capable of performing communication conforming to other standards. The interface 32 inputs information received from the network 2 to the control unit 30. The interface 32 also transmits information from the control unit 30 to the network 2.

[0034] The control unit 30 can comprehensively manage the operation of the server 3 by controlling the other components of the server 3. The control unit 30 can also be called, for example, a control circuit. The control unit 30 includes, for example, at least one processor. The control unit 30 may include, for example, a CPU.

[0035] The storage unit 31 may include a non-transitory recording medium such as a ROM and a RAM that can be read by the CPU of the control unit 30. The storage unit 31 stores, for example, a program 31a for controlling the server 3. Various functions of the control unit 30 are realized, for example, by the CPU of the control unit 30 executing the program 31a in the storage unit 31.

[0036] The configuration of the control unit 30 is not limited to the above example. For example, the control unit 30 may include multiple CPUs or at least one DSP. All or some of the functions of the control unit 30 may be realized by a hardware circuit that does not require software to realize the function. Similarly to the storage unit 41, the storage unit 31 may include a computer-readable non-transitory recording medium other than ROM and RAM.

[0037] The input unit 38 can accept various inputs from the user. The input unit 38 may include, for example, a mouse and a keyboard. The input unit 38 may also include a touch sensor that accepts touch operations by the user. The input unit 38 may also include an audio input unit that accepts audio input from the user. The audio input unit may include, for example, a microphone. The control unit 30 can recognize the content of the input accepted by the input unit 38 based on the output signal from the input unit 38.

[0038] The display unit 37 is capable of displaying various types of information under the control of the control unit 30. The display unit 37 has a display surface that displays various types of information. The display unit 37 may be, for example, a liquid crystal display, an organic EL display, or a plasma display. Furthermore, if the input unit 38 includes a touch sensor, the touch sensor and the display surface of the display unit 37 may form a touch panel display having a display function and a touch detection function. In this case, the input unit 38 can detect a touch operation on the display surface of the display unit 37.

[0039] <Example of processing system operation> The processing system 1 performs an information collection process for collecting information about crops in a target farm, and a monitoring process for monitoring the state of the crops. The information collection process is performed, for example, once for each of multiple growth stages of the crop. For example, if the crop is grapes or strawberries, the information collection process is performed once for each of the germination period, flowering period, fruit growth period (also referred to as the fruit growth period), and harvest period. The information collection process is performed, for example, at the beginning of each growth stage. The monitoring process is performed repeatedly, for example, after the information collection process is performed once for each of multiple growth stages. The monitoring process is performed, for example, every day or every few days for each growth stage.

[0040] <Example of information collection process> Fig. 4 is a schematic diagram showing an example of a plurality of functional blocks included in a processing system 1 that performs information collection processing. As shown in Fig. 4, the processing system 1 includes, for example, the following functional blocks: a first grouping unit 100, a representative image acquisition unit 131, a feature map generation unit 132, a treatment content identification unit 133, a treatment information generation unit 134, an image processing unit 141, and a treatment target information generation unit 142. The first grouping unit 100 includes, for example, a feature acquisition unit 140 and a second grouping unit 130.

[0041] The control unit 30 of the server 3 includes, for example, functional blocks including a second grouping unit 130, a representative image acquisition unit 131, a feature map generation unit 132, a treatment content identification unit 133, and a treatment information generation unit 134. The second grouping unit 130, the representative image acquisition unit 131, the feature map generation unit 132, the treatment content identification unit 133, and the treatment information generation unit 134 are each functional blocks formed by the CPU of the control unit 30 executing a program 31a in the storage unit 31. Note that all or some of the functions of the second grouping unit 130 may be realized by a hardware circuit that does not require software to realize the function. The same applies to the representative image acquisition unit 131, the feature map generation unit 132, the treatment content identification unit 133, and the treatment information generation unit 134, as well as to the functional blocks described below that the server 3 and the terminal device 4 include.

[0042] The control unit 40 of the terminal device 4 includes, for example, a feature acquisition unit 140, an image processing unit 141, and a processing target information generation unit 142. Each of the feature acquisition unit 140, the image processing unit 141, and the processing target information generation unit 142 is a functional block formed by the CPU of the control unit 40 executing a program 41a in the storage unit 41.

[0043] 5 is a schematic diagram showing an example of information collection processing. In the information collection processing, first, in step s1, the image processing unit 141 of the terminal device 4 acquires a target image 150 showing a crop and crop position information indicating the position of the crop. The image processing unit 141 functions as an image acquisition unit (see FIG. 4) that acquires the target image 150, and also functions as a position information acquisition unit that acquires the crop position information. Since the crop position information can also be considered as position information that indicates the absolute position of the treatment target, the crop position information can also be considered as treatment target position information.

[0044] The image processing unit 141 acquires a target image 150 and crop position information based on the captured image 50 from the camera 5. Based on the captured images 50 obtained by the camera 5, the image processing unit 141 acquires a plurality of target images 150 each showing a plurality of crops in the target farm, and a plurality of pieces of crop position information indicating the positions of the plurality of crops in the target farm.

[0045] 6 and 7 are schematic diagrams respectively showing an example of a photographed image 50 and a target image 150 acquired during the fruit growing season and the harvest season when the crop to be treated is grapes.

[0046] In step s1, the image processing unit 141 performs object recognition processing (also referred to as object detection processing) on ​​the captured image 50 using machine learning, template matching, or the like, to identify partial images 51 in which crops appear in the captured image 50. In the example of Fig. 6, two partial images 51 are identified from the captured image 50. Each of the two partial images 51 shows a large image of a bunch of grapes.

[0047] Next, the image processing unit 141 enlarges each of the acquired partial images 51 to generate a target image 150 of a predetermined size. This results in the acquisition of a target image 150 in which the crop is shown in a large size. In the example of Fig. 7, a bunch of grapes is shown in the target image 150.

[0048] The range of the crop that appears in the target image 150 and the partial image 51 that is the basis of the target image 150 is determined by object recognition processing executed by the image processing unit 141. If the crop is a strawberry, for example, a single strawberry appears in the partial image 51 and the target image 150 acquired during the fruit growth period or harvest time.

[0049] The partial image 51 and the target image 150 acquired during the germination period may depict, for example, a germinated crop. The partial image 51 and the target image 150 acquired during the flowering period may depict, for example, a flowering crop. However, since there is variation in the growth conditions among multiple crops in the target farm, the partial image 51 and the target image 150 acquired during the germination period may not necessarily depict a germinated crop. Similarly, the partial image 51 and the target image 150 acquired during the flowering period may not necessarily depict a flowering crop.

[0050] Furthermore, in step s1, the image processing unit 141 generates treatment target position information indicating the absolute position of the treatment target shown in the partial image 51 based on vehicle position information 60 indicating the absolute position of the vehicle 10 when the photographed image 50 was acquired by the camera 5, the relative position of the partial image 51 in the photographed image 50, and various parameters related to the camera 5 (for example, the relative position of the camera 5 with respect to the vehicle 10 and the angle of view of the camera 5). When the photographed image 50 shown in Fig. 6 is obtained, the image processing unit 141 generates crop position information indicating the absolute position of the crop shown in the partial image 51 for each of the two partial images 51 identified from the photographed image 50.

[0051] In this way, in step s1, the image processing unit 141 acquires the target image 150 and crop position information for all crops in the target farm based on the multiple captured images 50 that capture all crops in the target farm. That is, for each treatment target, the image processing unit 141 acquires the target image 150 that captures the treatment target and treatment target position information that indicates the position of the treatment target.

[0052] When the image processing unit 141 acquires treatment target position information indicating the position of the treatment target, it associates the treatment target position information with identification information (also called treatment target ID) of the treatment target. ID is an abbreviation for Identifier. The treatment target ID is identification information of the crop, and can also be called a crop ID. In the processing system 1, each treatment target is managed by a treatment target ID.

[0053] The first grouping unit 100 included in the processing system 1 performs treatment target grouping, based on a plurality of target images acquired by the image processing unit 141, to divide a plurality of treatment targets (in other words, a plurality of crops) appearing in each of the plurality of target images into a plurality of treatment target groups. The first grouping unit 100 includes, for example, a feature amount acquisition unit 140 that acquires feature amounts of the plurality of target images acquired by the image processing unit 141, and a second grouping unit 130 that performs treatment target grouping based on the feature amounts of the plurality of target images. The treatment target groups can also be referred to as crop groups. The groups can also be referred to as classifications, for example.

[0054] After step s1, in step s2, the feature amount acquiring unit 140 acquires the feature amount of each target image acquired in step s1. FIG.

[0055] 8, the feature acquisition unit 140 includes, for example, an autoencoder 240. The autoencoder 240 may be configured with, for example, a neural network such as a convolutional neural network.

[0056] The autoencoder 240 includes, for example, an encoder 241 to which the target image 150 is input, and a decoder 242. The encoder 241 converts the target image 150 into a feature of the target image 150. As a result, the feature acquisition unit 140 acquires the feature of the target image 150.

[0057] The feature amount is represented, for example, by a vector. If the target image 150 has, for example, 128 pixels vertically and 128 pixels horizontally, the encoder 241 outputs, for example, a feature amount represented by a 400-dimensional vector. Hereinafter, a vector representing a feature amount may be referred to as a feature vector. Furthermore, a feature amount of a target image that shows a processing target may be referred to as a feature amount corresponding to the processing target or a feature amount of the processing target. Similarly, a feature amount of a target image that shows a crop may be referred to as a feature amount corresponding to the crop or a feature amount of the crop.

[0058] The decoder 242 restores the target image 150 input to the encoder 241 based on the feature amount (in other words, the feature vector) output from the encoder 241. For example, the decoder 242 converts the 400-dimensional feature vector into an image having 128 pixels vertically and 128 pixels horizontally, which corresponds to the target image 150, and outputs the image. Ideally, the decoder 242 outputs an image identical to the target image 150 input to the encoder 241.

[0059] In this way, the feature amount acquiring unit 140 acquires the feature amount (in other words, the feature vector) of each target image acquired in step s1. The feature amount of the target image can be said to be, for example, a value representing the characteristics of the appearance of the treatment target shown in the target image. Note that the feature amount acquiring unit 140 may include only the encoder 241 out of the encoder 241 and the decoder 242.

[0060] After step s2, in step s3, the treatment target information generation unit 142 generates information about each treatment target (also referred to as first treatment target information). The first treatment target information includes, for example, a target image, feature amounts of the target image, treatment target position information of the treatment target appearing in the target image, and a treatment target ID of the treatment target appearing in the target image. The treatment target information generation unit 142 generates first treatment target information for all crops in the target farm. The first treatment target information can also be referred to as first crop information.

[0061] After step s3, in step s4, the terminal device 4 transmits the plurality of pieces of first processing target information generated by the processing target information generating unit 142 to the server 3.

[0062] Step s11 is executed in the server 3 that receives multiple pieces of first processing target information from the terminal device 4. In step s11, the second grouping unit 130 performs feature grouping to divide multiple feature amounts included in each piece of first processing target information into multiple feature amount groups. Then, for each of the multiple feature amount groups, the second grouping unit 130 includes multiple processing targets (i.e., multiple crops) that appear in multiple target images each having the multiple feature amounts that make up that feature amount group into one processing target group. This results in multiple processing target groups (i.e., multiple crop groups) that correspond to the multiple feature amount groups. The feature amounts of multiple target images in which multiple processing targets that make up one processing target group appear constitute one feature amount group. The second grouping unit 130 divides the multiple feature amounts into multiple feature amount groups, thereby dividing multiple processing targets that appear in multiple target images each having the multiple feature amounts into multiple processing target groups. It can be said that multiple processing targets that make up one processing target group are similar in appearance to each other. Furthermore, since the appearance of a crop depends on the growing conditions of the crop, when the treatment target is a crop, it can be said that the multiple crops that make up one treatment target group have similar growing conditions to each other.

[0063] The second grouping unit 130 may, for example, reduce the number of dimensions of the features of multiple target images and then classify the feature quantities into multiple feature groups. For example, suppose that the features are represented by 400-dimensional feature vectors, and the number of dimensions of the features is 400. In this case, the second grouping unit 130 may, for example, reduce the number of dimensions of the multiple feature quantities from 400 to three or two, and then classify the three- or two-dimensional feature quantities into multiple feature groups. This simplifies the feature grouping process. Reducing the number of dimensions of the features can also be considered compressing the number of dimensions of the features. The second grouping unit 130 can reduce the number of dimensions of the features by multiplying the feature vectors representing the features by a transformation formula (or transformation matrix) for dimensionality reduction (or dimensionality compression). Hereinafter, the transformation formula for dimensionality reduction may be referred to as a dimensionality reduction transformation formula. The second grouping unit 130 may reduce the number of dimensions of the features using, for example, principal component analysis. The second grouping section 130 may divide a plurality of feature amounts into a plurality of feature amount groups while keeping the number of dimensions the same.

[0064] The second grouping unit 130 may group a plurality of feature amounts using, for example, the k-means method. Grouping is also called, for example, clustering, and a group is also called, for example, a cluster. Below, an example of the operation of the second grouping unit 130 when the second grouping unit 130 divides a plurality of feature amounts into N feature amount groups using the k-means method will be described.

[0065] The second grouping unit 130 first arranges a plurality of features in a feature space. The dimension of the feature space is the same as the dimension of the features to be grouped. For example, if the dimension of the features to be grouped (i.e., the features with reduced dimensionality) is two-dimensional, the dimension of the feature space will be two-dimensional.

[0066] Next, the second grouping unit 130 sets the initial centroids of the N feature vector groups in the feature vector space. Next, the second grouping unit 130 identifies the initial centroid that is closest to the position of a certain feature vector among the N initial centroids in the feature vector space, and includes the certain feature vector in the feature vector group that has the identified initial centroid. The second grouping unit 130 performs the same process for each feature vector to provisionally determine the feature vector group to which each feature vector belongs.

[0067] Next, the second grouping unit 130 finds the center of gravity of each of the N feature groups. Then, the second grouping unit 130 identifies the center of gravity closest to the position of a certain feature among the N center of gravity in the feature space, and includes the certain feature in the feature group having the identified center of gravity. The second grouping unit 130 performs the same process for each feature, and again tentatively determines to which feature group each feature belongs.

[0068] The second grouping unit 130 repeats the above process and ends the process when the affiliation of each feature no longer changes. The N feature groups after the process are the final N feature groups. In this way, the multiple features are grouped into N feature groups. For each of the N feature groups, the second grouping unit 130 includes multiple processing targets that appear in multiple target images each having multiple feature values ​​that make up that one feature group into one processing target group. In this way, N processing target groups corresponding to the N feature groups are obtained.

[0069] After step s11, in step s12, the feature amount map generating unit 132 generates a feature amount map 300 that represents the distribution of multiple feature amounts in feature amount space. Next, in step s13, the display unit 37 of the server 3 displays the feature amount map 300.

[0070] Fig. 9 is a schematic diagram showing an example of display of a feature amount map 300 on the display unit 37. In the example of Fig. 9, the number of dimensions of the feature amounts and the number of dimensions of the feature amount space are two. The feature amount map 300 shown in Fig. 9 represents the distribution of multiple feature amounts in the feature amount space of the feature amounts whose number of dimensions has been reduced to two. Displaying the feature amount map 300 for the feature amounts whose number of dimensions has been reduced makes it easier for the user to grasp the overview of the distribution of multiple feature amounts in the feature amount space.

[0071] In the feature amount map 300, the position of each feature amount in the feature amount space is indicated by, for example, a circle 305. In the feature amount map 300, the positions of multiple feature amounts are indicated in different ways for each feature amount group, making it easier for the user to visually distinguish between the multiple feature amount groups.

[0072] 9, the multiple feature amounts are divided into three feature amount groups: a first feature amount group, a second feature amount group, and a third feature amount group. Circles 305a representing the positions of feature amounts belonging to the first feature amount group are hatched with horizontal lines. Circles 305b representing the positions of feature amounts belonging to the second feature amount group are hatched with sandy lines. Circles 305c representing the positions of feature amounts belonging to the third feature amount group are hatched with left-leaning lines.

[0073] Note that the method of showing the positions of multiple feature amounts in different ways for each feature amount group in the feature amount map 300 is not limited to the example in Fig. 9. For example, in the feature amount map 300, the positions of feature amounts belonging to the first feature amount group, the positions of feature amounts belonging to the second feature amount group, and the positions of feature amounts belonging to the third feature amount group may be shown by circles 305 of different colors.

[0074] 9, in the feature map 300, a representative point 301 in the feature space of each feature group is shown. In the example of Fig. 9, in the feature map 300, a representative point 301a in the feature space of the first feature group, a representative point 301b in the feature space of the second feature group, and a representative point 301c in the feature space of the third feature group are shown by circles larger than the circles 305 that indicate the positions of the features. The representative point 301 of the feature group may be, for example, the center of gravity of the feature group (that is, the average value of the positions of the multiple feature amounts that make up the feature group).

[0075] When the input unit 38 of the server 3 receives a selection input from the user to select a representative point 301 of a certain feature group on the feature map 300, the display unit 37 may display a representative image 310 of a processing target group corresponding to the certain feature group. The representative image 310 is an image that represents the processing target group. The selection input to select an object displayed on the display surface of the display unit 37, such as the representative point 301, may be a predetermined touch operation on the object, or a predetermined mouse operation with the pointer positioned on the object.

[0076] 10 to 12 are schematic diagrams showing display examples of the representative image 310. When the input unit 38 receives a selection input for selecting the representative point 301a of the first feature amount group, the display unit 37, under the control of the control unit 30, displays the representative image 310a of the first processing target group corresponding to the first feature amount group, together with the feature amount map 300, as shown in Fig. 10. The feature amounts of the multiple target images, each of which shows the multiple processing targets that make up the first processing target group, make up the first feature amount group.

[0077] When the input unit 38 receives a selection input for selecting the representative point 301b of the second feature amount group, the display unit 37 displays a representative image 310b of the second processing target group corresponding to the second feature amount group, together with the feature amount map 300, as shown in Fig. 11. The feature amounts of the target images in which the multiple processing targets constituting the second processing target group are respectively captured constitute the second feature amount group.

[0078] When the input unit 38 receives a selection input for selecting the representative point 301c of the third feature amount group, the display unit 37 displays a representative image 310c of the third processing target group corresponding to the third feature amount group, together with the feature amount map 300, as shown in Fig. 12. The feature amounts of the target images in which the multiple processing targets constituting the third processing target group are respectively captured constitute the third feature amount group.

[0079] The representative image acquisition unit 131 of the server 3 can acquire N representative images 310 each representing one of the N feature amount groups. When the input unit 38 receives a selection input for selecting a representative point 301 of a certain feature amount group on the feature amount map 300, the representative image acquisition unit 131 acquires a representative image 310 of the processing target group corresponding to the certain feature amount group. The representative image 310 acquired by the representative image acquisition unit 131 is then displayed on the display unit 37.

[0080] The representative image acquiring unit 131 may set a target image corresponding to the representative point 301 selected by the user in the feature space as the representative image 310. For example, the representative image acquiring unit 131 may set a target image included in the first processing target information including a feature at the same position in the feature space as the representative point 301 selected by the user as the representative image 310. That is, the representative image acquiring unit 131 may set a target image having a feature at the same position in the feature space as the representative point 301 selected by the user, among the multiple target images transmitted from the terminal device 4 in step s4, as the representative image 310. Furthermore, if the multiple target images transmitted from the terminal device 4 in step s4 do not include a feature at the same position as the representative point 301 selected by the user, the representative image acquiring unit 131 may set a target image having a feature at a position closest to the selected representative point, among the multiple target images, as the representative image 310.

[0081] In this way, the processing system 1 acquires a representative image of the treatment target group, and therefore the processing system 1 can perform processing using the representative image of the treatment target group. This improves the convenience of the processing system. For example, as in the examples of FIGS. 10 to 12, when the server 3 displays the representative image 310, the user of the server 3 can visually confirm the representative image of the treatment target group. Therefore, the user can easily grasp the outline of the state of the treatment targets in the treatment target group from the representative image. In the case where the treatment targets are crops as in this example, the user can easily grasp the outline of the growth status of the crops in the crop group from the representative image.

[0082] When the input unit 38 receives a selection input for selecting a feature position 315 in the feature map 300, the display unit 37 may display a target image 150 having the feature, as shown in Fig. 13. When the feature position 315 for each feature is selected, the display unit 37 may display the target image 150 having the feature. This allows the user to select the feature position 315 in the feature map 300 and check the target image 150 having the feature.

[0083] A feature amount map 300 may show a predetermined range 302 from a representative point 301 in a feature amount space. In the examples of FIGS. 9 to 13, the feature amount map 300 shows a predetermined range 302a from a representative point 301a of a first feature amount group, a predetermined range 302b from a representative point 301b of a second feature amount group, and a predetermined range 302c from a representative point 301c of a third feature amount group. As shown in FIGS. 9 to 13, the predetermined range 302 may be shown as a circle centered on the representative point 301. Hereinafter, the predetermined range 302 of a feature amount group refers to the predetermined range 302 from the representative point 301a of the feature amount group.

[0084] When the input unit 38 receives a selection input for selecting the position of a feature that is inside a predetermined range 302 in the feature map 300, the display unit 37 displays a target image 150 having the feature. This target image 150 is a target image 150 having a feature that is located inside the predetermined range 302 from the representative point 301 in the feature space.

[0085] Furthermore, when the input unit 38 receives a selection input for selecting the position of a feature that is outside the predetermined range 302 in the feature map 300, the display unit 37 displays the target image 150 having the feature (see FIG. 13). This target image 150 is a target image 150 having a feature that is located outside the predetermined range 302 from the representative point 301 in the feature space.

[0086] The predetermined range 302 is automatically set by, for example, the server 3. The predetermined range 302 for one feature group may be set based on, for example, the deviation σ (in other words, the standard deviation σ) of the distances from the representative point 301 of the multiple feature amounts constituting the feature group in the feature space. For example, the predetermined range 302 may be set so that the radial distance is 3σ or 4σ. Alternatively, the predetermined range 302 for one feature group may be set so that a predetermined proportion of the multiple feature amounts constituting the feature group is included within the predetermined range 302 relative to the total number of the multiple feature amounts. The predetermined proportion may be, for example, 70%, 80%, 90%, or some other value. For example, consider a case where the total number of multiple feature amounts constituting one feature group is 1,000 and the predetermined proportion is 90%. In this case, the predetermined range 302 for the feature group is set so that 900 of the 1,000 feature amounts constituting the feature group are included within the predetermined range 302.

[0087] In this way, by showing the predetermined range 302 from the representative point 301 in the feature map 300, the user can easily identify feature quantities that are located far from the representative point 301. For example, if the predetermined range 302 is called the allowable range, the user can easily identify feature quantities that are outside the allowable range. It can also be said that the user can easily identify feature quantities that are located far from the position of the representative feature quantity of the feature group. Furthermore, the user can easily understand the distribution of multiple feature quantities in the feature group using the predetermined range 302 as a reference.

[0088] Furthermore, when the display unit 37 displays a target image 150 having a feature outside the predetermined range 302 in the feature space, the user can check the state of crops in the treatment target group that are in a growth state significantly different from the growth state of a representative crop by checking the target image 150. The growth state of the representative crop in the treatment target group can also be said to be, for example, the average growth state of the crops in the treatment target group.

[0089] After step s13, that is, after the display of the feature amount map 300 starts, the input unit 38 of the server 3 can receive from the user, for each of the N treatment target groups, treatment contents common to the plurality of treatment targets constituting one treatment target group. That is, the input unit 38 can receive from the user, for each of the N treatment target groups, treatment contents common to one treatment target group.

[0090] In step s14, the input unit 38 accepts from the user the treatment content for each of the N treatment target groups. In step s15, the treatment information generation unit 134 generates treatment information indicating N types of treatment content corresponding to the N treatment target groups accepted by the input unit 38. When the treatment information is generated, the information collection process ends. The generated treatment information is stored in the storage unit 31 as information collection result information, together with the result of the feature grouping (in other words, the result of the treatment target grouping), the feature map 300, a representative image of each feature group, and information indicating the execution time of the information collection process.

[0091] By executing the information collection process for the target farm as described above at each growing stage each year, the storage unit 31 stores information on the results of information collection for the target farm at each growing stage each year.

[0092] For example, when the input unit 38 receives a selection input from the user to select the representative point 301 of a certain feature group included in the displayed feature map 300, it can receive from the user a treatment content common to the certain treatment target group. This selection input is different from the selection input when the representative image 310 is displayed.

[0093] In this example, a representative image 310 of the treatment target group is displayed, and the user can check the representative image 310 of the treatment target group and then input the treatment content common to the treatment target group into the input unit 38. Therefore, the user can check the outline of the state of the treatment targets of the treatment target group from the representative image and then input the treatment content common to the treatment target group into the input unit 38. As in this example, when the treatment targets are crops, a skilled crop grower can check the outline of the growth status of the crops in the crop group from the representative image and then input the treatment content common to the crop group into the input unit 38. The input unit 38 may accept the treatment content common to the treatment target group while the representative image 310 of the treatment target group is displayed on the display unit 37.

[0094] The input unit 38 has, for example, a voice input unit that accepts a natural language spoken by a user and expresses a treatment content. The treatment content identification unit 133 identifies the treatment content expressed by the natural language accepted by the voice input unit. The treatment information generation unit 134 generates treatment information indicating N types of treatment content corresponding to the N treatment target groups identified by the treatment content identification unit 133.

[0095] In this way, the user can input the details of the process to the processing system 1 in natural language, which improves the convenience of the processing system 1.

[0096] The treatment content identification unit 133, for example, performs speech recognition on the natural language received by the voice input unit and converts the natural language into a character string (also referred to as text). Next, the treatment content identification unit 133 extracts, from the acquired character string, a plurality of pieces of information (also referred to as a plurality of pieces of necessary identification information) required to identify the treatment content. This process may be performed using, for example, machine learning. For example, a character string may be input to a trained neural network, and the trained neural network may output a plurality of pieces of necessary identification information. Then, the treatment content identification unit 133 identifies the treatment content expressed in the natural language received by the voice input unit based on the acquired plurality of pieces of necessary identification information. In this example, for example, a plurality of basic treatment contents are stored in the storage unit 31 in advance. The treatment content identification unit 133 converts the acquired plurality of pieces of necessary identification information into one of the plurality of basic treatment contents. The one basic treatment content obtained by this conversion becomes the treatment content expressed in the natural language received by the voice input unit, and the treatment content identification unit 133 can identify the treatment content expressed in the natural language received by the voice input unit.

[0097] The plurality of basic treatment details are appropriately set depending on the treatment target. For example, consider a case where the treatment target is fruit such as grapes. In this case, the plurality of basic treatment details may include a treatment to increase fertilizer by 10%, a treatment to increase fertilizer by 20%, a treatment to discard the crop, a treatment to take no treatment, and a treatment to bag the fruit.

[0098] When the plurality of pieces of specified necessary information acquired by the treatment content identification unit 133 contains insufficient information and the plurality of pieces of specified necessary information acquired by the treatment content identification unit 133 cannot be converted into basic treatment contents, the server 3 may inquire of the user about the insufficient information. For example, consider a case where the user inputs the natural language phrase "increase fertilizer" into the input unit 38. In this case, since information on how much fertilizer to increase is insufficient, the server 3 may inquire of the user about by what percentage the fertilizer should be increased. The inquiry to the user may be made, for example, using the display unit 37. In this case, the display unit 37 may display a character string such as "By what percentage should the fertilizer be increased?". It can be said that the display unit 37 functions as an output unit that inquires of the user about the insufficient information.

[0099] When a user inquires about missing information, the user inputs natural language describing the missing information into the voice input unit in response to the inquiry. As a result, all information on the treatment contents common to the treatment target group is input from the user to the input unit 38. The treatment content identification unit 133 performs voice recognition on the natural language received by the voice input unit and converts the natural language into a character string. The treatment content identification unit 133 then extracts the missing information from the acquired character string. The treatment content identification unit 133 converts a plurality of pieces of information consisting of the extracted missing information and the plurality of pieces of specific necessary information acquired up to that point into one of a plurality of basic treatment contents. As a result, the server 3 recognizes the treatment contents common to the treatment target group.

[0100] <Example of treatment information> The treatment information may include a treatment target map 350 that shows the distribution of multiple treatment targets. In the treatment target map 350, for example, the position of each treatment target is shown in a different manner depending on the treatment content. If the treatment target is a crop, the treatment target map 350 shows how multiple crops are distributed in the target farm. In other words, the treatment target map 350 shows the distribution of multiple crops in the target farm.

[0101] Fig. 14 is a schematic diagram showing an example of a treatment target map 350. In the treatment target map 350 shown in Fig. 14, the position of each treatment target is represented by a figure 351. When the treatment target is a strawberry, the figure 351 representing the position of the treatment target may be a figure imitating the outer shape of a strawberry fruit, as shown in Fig. 14.

[0102] 14, there are three treatment target groups: a first treatment target group, a second treatment target group, and a third treatment target group. A first treatment content exists as a treatment content common to the first treatment target group, a second treatment content exists as a treatment content common to the second treatment target group, and a third treatment content exists as a treatment content common to the third treatment target group.

[0103] 14, a graphic 351a representing the position of each of the treatment targets constituting the first treatment target group is hatched with diagonal lines slanting upward to the right. A graphic 351b representing the position of each of the treatment targets constituting the second treatment target group is hatched with sandy lines. A graphic 351c representing the position of each of the treatment targets constituting the third treatment target group is hatched with diagonal lines slanting downward to the right.

[0104] In this way, for example, the position of each treatment target is shown in a different manner depending on the treatment content in the treatment target map 350. By displaying such treatment target map 350 on the display unit 47 of the terminal device 4, the worker who is the user of the terminal device 4 can easily identify the position of each treatment target and the treatment content for each treatment target.

[0105] Note that the method of showing the position of each treatment target in a different manner for each treatment content in the treatment target map 350 is not limited to the example in Fig. 14. For example, in the treatment target map 350, the positions of the treatment targets in the first treatment target group, the positions of the treatment targets in the second treatment target group, and the positions of the treatment targets in the third treatment target group may be shown by figures 351 of different colors.

[0106] In the treatment information, information about the treatment target (also referred to as second treatment target information) is associated with the position of the treatment target on the treatment target map 350. In the treatment information, the second treatment target information is associated individually with the position of each treatment target on the treatment target map 350.

[0107] The second treatment target information for a certain treatment target includes, for example, a target image in which the certain treatment target appears, a treatment target ID for the certain treatment target, treatment target location information for the certain treatment target, and explanatory information that explains the treatment content for the certain treatment target.The second treatment target information for a certain treatment target also includes feature amounts of the target image in which the certain treatment target appears and information indicating the time when the feature amounts were acquired.The treatment information includes second treatment target information for each crop in the target farm.The second treatment target information can also be called second crop information.

[0108] <Example of display of treatment information on a terminal device> The server 3 transmits the treatment information generated by the treatment information generation unit 134 to the terminal device 4. For example, when the input unit 38 receives a predetermined input, the interface 32 may transmit the treatment information in the storage unit 31 to the terminal device 4 under the control of the control unit 30. Furthermore, when the server 3 receives an instruction to transmit the treatment information from the terminal device 4, the interface 32 may transmit the treatment information in the storage unit 31 to the terminal device 4 under the control of the control unit 30. In the terminal device 4, when the input unit 48 receives a predetermined input, the interface 42 may transmit information indicating an instruction to transmit the treatment information to the server 3 under the control of the control unit 40.

[0109] The terminal device 4 stores the treatment information received from the server 3 in the memory unit 41. The control unit 40 displays the treatment information in the memory unit 41 on the display unit 47. The worker, who is the user of the terminal device 4, performs treatment on each crop in the target farm in accordance with the treatment content indicated in the treatment information displayed on the display unit 47.

[0110] When the input unit 48 receives a predetermined input from the user, the control unit 40 causes the display unit 47 to display a treatment target map 350 included in the treatment information. When the input unit 38 receives a selection input selecting a position 355 of the treatment target on the treatment target map 350 displayed on the display unit 47, the control unit 40 displays a screen 360 showing second treatment target information included in the treatment information and associated with the selected position 355, as shown in FIG. 15 . The selection of the position 355 of the treatment target on the treatment target map 350 can also be seen as the selection of the treatment target. Hereinafter, the crop corresponding to the selected position 355, i.e., the selected crop, will be referred to as the selected crop.

[0111] The screen 360 includes the target image 150 showing the selected crop, identification information 361 of the selected crop, crop position information 362 of the selected crop, and explanatory information 363 that explains the treatment details for the selected crop. The explanatory information 363 is, for example, composed of an explanatory text of the treatment details.

[0112] In this example, as will be described later, when the treatment information is updated in the monitoring process, the second treatment target information included in the treatment information may include an evaluation value (also referred to as an effect evaluation value) 364 of the effect of the treatment content. The evaluation value can also be considered a score. The example in Figure 15 shows a screen 360 in which the second treatment target information for the selected crop includes the effect evaluation value 364.

[0113] Screen 360 includes an instruction area 367 for instructing that screen 360 be hidden. When input unit 48 receives a selection input for selecting instruction area 367, display unit 47 hides screen 360.

[0114] In this way, in this example, when the input unit 48 receives a selection input for selecting the position of a treatment target on the treatment target map 350, the display unit 47 displays the treatment content for the treatment target. This allows the worker to check the treatment content for the treatment target by selecting the position of the treatment target on the treatment target map 350. Furthermore, as in the example of FIG. 15 , when the screen 360 includes the effect evaluation value 364, the worker can check the evaluation value of the effect of the treatment content for the treatment target by selecting the position of the treatment target on the treatment target map 350.

[0115] The treatment information generation unit 134 may generate second treatment information that indicates treatment details in more detail than the above-mentioned treatment information (also referred to as first treatment information). For example, the second treatment information includes a treatment target map 350 and second treatment target information for each crop, similar to the first treatment information. Furthermore, in the second treatment information, detailed explanation information indicating a detailed explanation of the treatment details for the treatment target is associated with the position of the treatment target on the treatment target map 350. In the second treatment information, detailed explanation information is associated individually with the position of each treatment target on the treatment target map 350. The second treatment information includes second treatment target information and detailed explanation information for each crop in the target farm.

[0116] The detailed explanation information includes, for example, a detailed explanation that explains the procedure in more detail than the explanation that constitutes the explanation information. The detailed explanation information also includes, for example, an image (also called a procedure image) that shows the procedure of the procedure. The procedure image may be composed of multiple still images or may be a moving image.

[0117] The server 3 may, for example, send first treatment information to a terminal device 4 held by a worker who is familiar with the work (i.e., treating crops), and send second treatment information to a terminal device 4 held by a worker who is not familiar with the work.

[0118] In the terminal device 4 that has received the second action information, the second action information is stored in the storage unit 41. The control unit 40 causes the display unit 47 to display the second action information in the storage unit 41. When the input unit 48 receives a predetermined input from the user, the control unit 40 causes the display unit 47 to display an action target map 350 included in the second action information. When the input unit 48 receives a selection input for selecting a position 355 of the action target in the action target map 350 displayed on the display unit 47, the control unit 40 displays a screen 360 showing the second action target information included in the second action information and associated with the selected position 355, as in FIG. 15 described above. Hereinafter, the screen 360 showing the second action target information included in the second action information will be referred to as a screen 360a.

[0119] When the input unit 48 receives a selection input selecting the area on the screen 360a where the explanatory information 363 is displayed, the display unit 47 displays a screen 370 showing a detailed explanatory text 371 contained in the detailed explanatory information of the selected crop included in the second treatment information, as shown in Figure 16.

[0120] Screen 370 includes an instruction area 375 for instructing the display of a procedure image. When input unit 48 receives a selection input for selecting instruction area 375, display unit 47 displays procedure image 378 included in the detailed explanation information for the selected crop included in the second treatment information, as shown in Fig. 17. In the example of Fig. 17, procedure image 378 showing the procedure for bagging fruit is displayed as a plurality of still images.

[0121] In this way, the terminal device 4 displays the second treatment information showing the treatment content in detail, so that even an operator who is not familiar with the work can easily perform the treatment on the crops.

[0122] The feature amount map 300 may show a locus 308 of points that are equidistant from the representative points 301 of the two feature amount groups. The locus 308 is identified by the control unit 30. Fig. 18 is a schematic diagram showing an example of how the feature amount map 300 shows a locus 308 (also referred to as a second locus 308) of points that are equidistant from the representative point 301b of the second feature amount group and the representative point 301c of the third feature amount group.

[0123] When the input unit 38 receives a predetermined input while the display unit 37 is displaying the feature amount map 300, the display unit 37 may display a second locus 308 on the feature amount map 300. Furthermore, when the input unit 38 receives a predetermined input while the display unit 37 is displaying the feature amount map 300, the display unit 37 may display a locus 308 of points that are equidistant from the representative point 301a of the first feature amount group and the representative point 301b of the second feature amount group (also referred to as the first locus 308) on the feature amount map 300. Furthermore, when the input unit 38 receives a predetermined input while the display unit 37 is displaying the feature amount map 300, the display unit 37 may display a locus 308 of points that are equidistant from the representative point 301a of the first feature amount group and the representative point 301c of the third feature amount group (also referred to as the third locus 308). The display unit 37 may simultaneously display at least two trajectories 308, namely, the first trajectory 308, the second trajectory 308, and the third trajectory 308. Furthermore, the display unit 37 may always display at least one of the first trajectory 308, the second trajectory 308, and the third trajectory 308 on the feature amount map 300.

[0124] In this way, when the feature map 300 shows the locus 308 of points equidistant from the representative points 301 of the two feature groups, a user looking at the display of the feature map 300 can easily identify the features equidistant from the representative points 301 of the two feature groups in the feature space.

[0125] When the input unit 38 receives a selection input for selecting a feature position on the trajectory 308 in the feature map 300, the display unit 37 displays a target image 150 having the selected feature, as shown in Fig. 19. In the example of Fig. 19, the position of a feature included in the third feature group on the second trajectory 308 is selected, and the target image 150 having the feature corresponding to the selected position is displayed. The target image 150 shown in Fig. 19 is a target image 150 having a feature that is equidistant from the representative point 301b of the second feature group and the representative point 301c of the third feature group.

[0126] In this way, when the display unit 37 displays a target image 150 having features equidistant from the representative points 301 of the two feature groups in the feature space, the user can confirm, for example, the state of a crop that is in an intermediate growth state between the growth states of representative crops in the two treatment target groups by checking the target image 150. In other words, the user can confirm the state of a crop that is in doubt as to which of the two treatment target groups it should belong to.

[0127] 20 , the display unit 37 may display a group information screen 330 showing group information related to feature groups together with the feature map 300. For example, when the display unit 37 is displaying the feature map 300, if the input unit 38 receives a predetermined input (for example, a selection input for selecting the outer shape (in other words, the outline) of the predetermined range 302), the display unit 37 may display the group information screen 330.

[0128] The group information screen 330 may include, for example, a name 331 of the feature amount group and a value 332 of the radial distance of a predetermined range 302 from a representative point 301 of the feature amount group. In FIG. 20, "first group" refers to the first feature amount group, and "allowable distance" refers to the radial distance of the predetermined range 302. Hereinafter, the first feature amount group, the second feature amount group, and the third feature amount group may be referred to as the first group, the second group, and the third group, respectively.

[0129] The group information screen 330 may include an average value 333 of the distances from the representative point 301 in the feature space for the multiple feature amounts constituting the feature group, and a variance 334 of the distances. The group information screen 330 may also include a number 335 of the multiple feature amounts constituting the feature group that are located inside a predetermined range 302 in the feature space, and a number 336 of the feature amounts that are located outside the predetermined range 302. In the example of FIG. 20 , the group information screen 330 for the first feature amount group is displayed, but a group information screen 330 showing group information for the second feature amount group may also be displayed, or a group information screen 330 showing group information for the third feature amount group may also be displayed.

[0130] 20, the feature amount map 300 may also show a group number value 340, which is the number of feature amount groups. The group number can also be said to be the number of groups to be processed.

[0131] The representative point 301 may be specified by a user of the server 3. For example, when the display unit 37 is displaying the feature map 300, if the input unit 38 receives a movement instruction input from the user instructing to move the position of a certain representative point 301 to a certain position on the feature map 300, the feature map generation unit 132 updates the feature map 300 so that the position of the certain representative point 301 moves to the certain position. The display unit 37 displays the updated feature map 300. When moving the position of a certain representative point 301 on the feature map 300, the feature map generation unit 132 changes the predetermined range 302 from the certain representative point 301 in accordance with the movement of the position of the certain representative point 301. The movement instruction input may be, for example, drag-and-drop of the representative point 301. It can also be said that the feature map generation unit 132 that updates the feature map 300 functions as the feature map update unit 132.

[0132] Fig. 21 is a schematic diagram showing an example of the state after representative point 301b of the second group shown in Fig. 20 has been moved in response to a movement instruction input from the user. In the example of Fig. 21, representative point 301b before the movement and predetermined range 302b from representative point 301b before the movement are shown by dashed lines, but the dashed lines may or may not be displayed on display unit 37.

[0133] The predetermined range 302 may be specified (in other words, settable) by a user of the server 3. For example, when the display unit 37 displays the group information screen 330, the user may be able to change the radial distance value 332 of the predetermined range 302 included in the group information screen 330. In this case, for example, the input unit 38 may be able to accept a selection input for selecting an area on the display surface of the display unit 37 where the radial distance value 332 is displayed, and then accept a designation input from the user that specifies the value 332 of the radial distance 332. Note that the user may be able to change the predetermined range 302 by directly manipulating the outline of the predetermined range 302.

[0134] Fig. 22 is a schematic diagram showing an example of what the predetermined range 302a shown in Fig. 20 looks like after it has been changed by the user. In the example of Fig. 22, the predetermined range 302a before the change is shown by a dashed line, but the dashed line may or may not be displayed on the display unit 37. In response to the change to the predetermined range 302a, the group information screen 330 for the first group is updated.

[0135] The user may also be able to set whether to enable or disable the automatic setting function of the predetermined range 302 in the server 3. Fig. 23 is a schematic diagram showing an example of the display of the group information screen 330 in this case.

[0136] 23 includes a setting area 337 for setting whether to enable or disable the automatic setting function of the predetermined range 302 in the server 3. After receiving a selection input for selecting the setting area 337, the input unit 38 may receive a designation input from the user instructing whether to enable or disable the automatic setting function of the predetermined range 302 in the server 3.

[0137] When the automatic setting function of the predetermined range 302 in the server 3 is enabled, "Auto Setting: ON" is displayed in the setting area 337, as shown in FIG. 23. Then, setting content information 338 indicating the content of the automatic setting is displayed on the group information screen 330. In the example of FIG. 23, setting content information 338 is displayed in the case where the radial distance of the predetermined range 302 is automatically set to 3σ, as described above. On the other hand, when the automatic setting function of the predetermined range 302 in the server 3 is disabled, "Auto Setting: OFF" is displayed in the setting area 337. Then, setting content information 338 is not displayed on the group information screen 330. When the automatic setting function of the predetermined range 302 in the server 3 is disabled, the predetermined range 302 can be specified by the user.

[0138] The result of the feature grouping by the second grouping unit 130 may be adjustable by a user of the server 3. Since the result of the feature grouping determines the result of the treatment target grouping, adjusting the result of the feature grouping can also be said to be adjusting the result of the treatment target grouping.

[0139] For example, the feature group to which a feature belongs may be changeable by the user. In other words, the process target group to which a process target belongs may be changeable by the user. FIG. 24 is a schematic diagram illustrating an example of a display on the display unit 37 in this case. For example, when the display unit 37 is displaying a feature map 300, if the input unit 38 receives a selection input for selecting the position of a feature on the feature map 300, the display unit 37 may display a setting screen 380 for changing the feature group to which the feature belongs, as shown in FIG. 24. The selection of the position of a feature on the feature map 300 can also be considered as the selection of the feature. Hereinafter, a feature whose position is selected by the user on the feature map 300 may be referred to as a selected feature.

[0140] The setting screen 380 includes, for example, a target image 150 having a selected feature. The setting screen 380 includes a designation area 381 for instructing that the selected feature be assigned to the first group. The designation area 381 can also be said to be a designation area for specifying the first group as the group to which the selected feature will belong. The setting screen 380 includes a designation area 382 for instructing that the selected feature be assigned to the second group. The setting screen 380 includes a designation area 383 for instructing that the selected feature be assigned to the third group.

[0141] In the example of FIG. 24 , the position of a feature included in the first group is selected, and the position is on a first locus 308 of points equidistant from the representative point 301a of the first group and the representative point 301b of the second group. In the example of FIG. 24 , the selected feature is included in the first group. When the input unit 38 receives a selection input selecting the designation area 382, ​​the second grouping unit 130 changes the feature group to which the selected feature belongs from the first group to the second group. Furthermore, when the input unit 38 receives a selection input selecting the designation area 383, the second grouping unit 130 changes the feature group to which the selected feature belongs from the first group to the third group. The selection of the designation area 382 and the designation area 383 can also be considered an instruction to change the feature group to which the selected feature belongs. When the input unit 38 receives an instruction to change the feature group to which the selected feature belongs, the second grouping unit 130 changes the feature group to which the selected feature belongs in accordance with the change instruction. When the input unit 38 receives a selection input for selecting the designation area 381, the second grouping unit 130 does not change the affiliation of the selected feature amount, but keeps it in the first group.

[0142] When the feature amount group to which the selected feature amount belongs is changed, the feature amount map generation unit 132 updates the feature amount map 300. Then, the display unit 37 displays the updated feature amount map 300. In this example, since the treatment content is set for each treatment target group, in other words, for each feature amount group, when the feature amount group to which the selected feature amount belongs is changed, the treatment content for the treatment target appearing in the target image 150 having the selected feature amount changes. Therefore, when the feature amount group to which the selected feature amount belongs is changed, the treatment information generation unit 134 updates the treatment information in the storage unit 31. The treatment information generation unit 134 that updates the treatment information can also be said to function as the treatment information update unit 134. The updated treatment information is transmitted from the server 3 to the terminal device 4.

[0143] The setting screen 380 may not include a designation area corresponding to the feature group to which the selected feature currently belongs, among the designation areas 381, 382, ​​and 383. In the example of Fig. 24, the setting screen 380 may not include the designation area 381.

[0144] Furthermore, when the position of a feature amount on locus 308 of points equidistant from representative point 301 of a certain feature amount group and representative point 301 of another feature amount is selected, setting screen 380 may include only the designation area corresponding to the certain feature amount group and the designation area corresponding to the other feature amount group out of designation areas 381, 382, ​​and 383. In the example of Fig. 24, setting screen 380 may include only designation areas 381 and 382 out of designation areas 381, 382, ​​and 383.

[0145] Furthermore, when the position of a feature that is included in a certain feature group and that is on locus 308 of points equidistant from representative point 301 of the certain feature group and representative point 301 of another feature is selected, setting screen 380 may include only the designation area corresponding to the other feature group out of designation areas 381, 382, ​​and 383. In the example of Fig. 24, setting screen 380 may include only designation area 382 out of designation areas 381, 382, ​​and 383.

[0146] Selecting the position of a feature in the feature map 300 can also be seen as selecting a processing target that appears in a target image having that feature. In this example, when the feature group to which the selected feature belongs is changed, the processing target group to which the processing target (also referred to as the selected processing target) appears in the target image having the selected feature belongs is changed. Therefore, the setting screen 380 can also be said to be a screen for changing the processing target group to which the selected processing target belongs. Furthermore, the instruction area 381 included in the setting screen 380 can be said to be an instruction area for instructing to set the selected processing target to the first processing target group corresponding to the first group. Similarly, the instruction area 382 included in the setting screen 380 can be said to be an instruction area for instructing to set the selected processing target to the second processing target group corresponding to the second group. Furthermore, the instruction area 383 included in the setting screen 380 can be said to be an instruction area for instructing to set the selected processing target to the third processing target group corresponding to the third group.

[0147] When the selected process target belongs to the first process target group, the selection in the designation area 382 and the designation area 383 can be said to be an instruction to change the process target group to which the selected process target belongs. When the input unit 38 receives an instruction to change the process target group to which the selected process target belongs, the second grouping unit 130 changes the process target group to which the selected process target belongs in accordance with the change instruction.

[0148] The number of feature amount groups (i.e., the value of N) may be changeable by the user. In other words, the number of groups to be processed may be changeable by the user. For example, when the display unit 37 displays the feature amount map 300 indicating the number of groups value 340, the user may be able to change the number of groups value 340. In this case, for example, the input unit 38 may be able to accept a selection input for selecting an area in the feature amount map 300 indicating the number of groups value 340, and then accept a designation input from the user specifying the number of groups value 340. For example, the expert user may change the number of groups after checking, on the display surface of the display unit 37, target images 150 having features outside the predetermined range 302.

[0149] When the number of groups is changed by the user, the second grouping unit 130 performs feature grouping again based on the changed number of groups. That is, the changed number of groups is used and the above-mentioned step s11 is executed again. Since the number of groups is also the number of groups to be processed, it can be said that the first grouping unit 100 performs grouping to be processed again based on the changed number of groups.

[0150] FIG. 25 is a schematic diagram showing an example of feature map 300 (also referred to as feature map 300a) when the number of groups is changed from "3" to "5" and the multiple feature amounts acquired by feature acquisition unit 140 are divided into a first feature amount group, a second feature amount group, a third feature amount group, a fourth feature amount group, and a fifth feature amount group.

[0151] In the feature amount map 300a, a circle 305d representing the position of a feature amount included in the fourth feature amount group is hatched upward to the right. A circle 305e representing the position of a feature amount included in the fifth feature amount group is hatched in a grid pattern. The feature amount map 300a also shows a representative point 301d of the fourth feature amount group and a predetermined range 302d from the representative point 301d. The feature amount map 300a also shows a representative point 301e of the fifth feature amount group and a predetermined range 302e from the representative point 301e.

[0152] When the number of groups is changed and feature grouping is performed again, the above steps s12 to s15 are performed again to generate new treatment information. The generation of new treatment information can also be said to be an update of the treatment information.

[0153] When the feature grouping is re-run, the user can adjust the results of the re-run feature grouping in the same manner as described above. Each time the feature grouping results are adjusted, the treatment information is updated.

[0154] Parameters (also simply referred to as necessary parameters) required for feature grouping by the second grouping unit 130 may be user-specifiable. In this case, in the server 3, the input unit 38 accepts user specification of the necessary parameters between steps s4 and s11 in FIG. 5. For example, when feature grouping is performed using the k-means method, the necessary parameters include the number of groups (i.e., the value of N) and the initial centroids of the feature groups. Between steps s4 and s11, the input unit 38 accepts specification of the number of groups and the initial centroids of each feature group. An example of the operation of the server 3 when the input unit 38 accepts the necessary parameters will be described below.

[0155] When step s4 is executed and the server 3 receives a plurality of pieces of first process target information from the terminal device 4, the feature amount map generation unit 132 of the server 3 generates a feature amount map 400 indicating the distribution in feature amount space of a plurality of feature amounts included in each of the plurality of pieces of first process target information. Then, the display unit 37 displays the feature amount map 400.

[0156] FIG. 26 is a schematic diagram showing an example of a feature map 400 displayed on the display unit 37. The feature map 400 shown in FIG. 26 represents the distribution of multiple feature amounts in a feature space where the number of dimensions has been reduced to two. In the feature map 400, the position of each feature amount in the feature space is indicated by, for example, a circle 405. At the stage where the feature map 400 is generated, feature grouping has not been performed, and therefore the positions of multiple feature amounts are indicated in the same manner in the feature map 400. In the example of FIG. 26, the circles 405 in the feature map 400 are not hatched.

[0157] The feature amount map 400 indicates a value 440 of the number of groups specified by the user. In the example of Fig. 26, the number of groups has not yet been specified by the user, so the value 440 of the number of groups is indicated by "X", which means that it is undetermined.

[0158] Additionally, initial centroid candidates 401 of the feature group (also referred to as initial centroid candidates 401) are shown in the feature map 400. The initial centroid candidates 401 are shown, for example, by larger circles than the circles 405 that represent the positions of the feature amounts. In the example of Fig. 26, three initial centroid candidates 401 are shown.

[0159] Here, the information gathering process of interest is called the information gathering process of interest. The information gathering process of interest can also be said to be the information gathering process to be explained.

[0160] If the time when the attention information collection process is executed (also referred to as the attention information collection process execution time) is the present, the initial centroid candidate 401 used in the attention information collection process may be set based on the result of past feature grouping. In other words, the initial centroid candidate 401 used in the attention information collection process may be set based on the result of feature grouping executed before the attention information collection process execution time.

[0161] For example, each of the representative points 301 of the multiple feature groups obtained as a result of feature grouping in an information collection process executed for the target farm at the same time in the past as the time when the target information collection process is executed may be adopted as the initial centroid candidate 401 in the target information collection process. In this case, if the year in which the target information collection process is executed is the current year, each of the representative points 301 of the multiple feature groups obtained as a result of feature grouping in an information collection process executed for the target farm at the same time last year (e.g., the same week or month last year) as the time when the target information collection process is executed may be adopted as the initial centroid candidate 401 in the target information collection process. Alternatively, if a similar information collection process is executed for another farm that grows crops of the same variety as the crops grown in the target farm, each of the representative points 301 of the multiple feature groups obtained as a result of feature grouping in an information collection process executed for the other farm at the same time in the past as the time when the target information collection process is executed may be adopted as the initial centroid candidate 401 in the target information collection process. The information required to determine the initial centroid candidate 401 in the attention information collection process is stored in the storage unit 31 .

[0162] Hereinafter, in the description of the attention information collection process, "past" means a time before the attention information collection process was executed. Also, in the description of the attention information collection process, the same time period in the past as the time the attention information collection process was executed may be referred to as the "same time period." Also, in the description of the attention information collection process, the target image acquired in the attention information collection process may be referred to as the "current target image." Also, simply referring to "other farms" means other farms that grow the same variety of crops as those grown in the target farm and that perform the same information collection process as the target farm.

[0163] When the display unit 37 is displaying the feature map 400, for example, when the input unit 38 receives a selection input to select an initial centroid candidate 401, the display unit 37 displays a setting screen 450 for setting the selected initial centroid candidate 401 (also called the selected initial centroid candidate 401) as the initial centroid, as shown in FIG. 27.

[0164] The setting screen 450 includes, for example, a target image 150 having a feature that is located at the same position in the feature space as the selected initial centroid candidate 401. However, if there is no feature that is located at the same position in the feature space as the selected initial centroid candidate 401, the setting screen 450 includes a target image 150 having a feature that is located closest to the selected initial centroid candidate 401. The setting screen 450 also includes an instruction area 451 that instructs setting the selected initial centroid candidate 401 as the initial centroid. When the input unit 38 receives a selection input that selects the instruction area 451, the second grouping unit 130 sets the selected initial centroid candidate 401 as one of the initial centroids. When one initial centroid is set, the number of groups increases.

[0165] Furthermore, when the input unit 38 receives a selection input for selecting the position of a feature on the feature map 400, a setting screen 460 for setting the selected position as the initial center of gravity is displayed on the display unit 37, as shown in FIG. 28.

[0166] The setting screen 460 includes, for example, the target image 150 having the feature amount of the selected position. The setting screen 460 also includes a designation area 461 for instructing to set the selected position as an initial center of gravity. When the input unit 38 receives a selection input for selecting the designation area 461, the second grouping unit 130 sets the selected position as one of the initial centers of gravity. When one initial center of gravity is set, the number of groups increases.

[0167] In this way, the user sets as many initial centers of gravity as necessary. By setting as many initial centers of gravity as necessary, the user can specify the number of groups.

[0168] Fig. 29 is a schematic diagram showing a display example of a feature amount map 400 in a case where the user has set three initial centers of gravity. In the feature amount map 400 shown in Fig. 29, the value 440 of the number of groups specified by the user indicates "3".

[0169] After the user sets the required number of initial centroids, that is, after the input unit 38 receives the required parameters from the user, step s11 is executed when the input unit 38 receives a predetermined input from the user. In step s11, feature grouping is executed based on the required parameters (e.g., the number of groups and the initial centroid positions) received by the input unit 38. Thereafter, the server 3 operates in the same manner as described above.

[0170] 30 , a predetermined range 402 from an initial centroid candidate 410 may be shown in a feature map 400. For example, a representative point 301 of a feature group obtained as a result of feature grouping in an information collection process executed at the same time in the past as the execution time of the interesting information collection process for the target farm may be adopted as the initial centroid candidate 401 in the interesting information collection process, and a predetermined range 302 from the representative point 301 may be adopted as the predetermined range 402 from the initial centroid candidate 401. Alternatively, a representative point 301 of a feature group obtained as a result of feature grouping in an information collection process executed at the same time in the past as the execution time of the interesting information collection process for another farm may be adopted as the initial centroid candidate 401 in the interesting information collection process, and a predetermined range 302 from the representative point 301 may be adopted as the predetermined range 402 from the initial centroid candidate 401.

[0171] In the above example, the number of groups is set by the user setting as many initial centroids as necessary, but the user may specify the number of groups and then set initial centroids for each of the specified number of feature amount groups. In this case, for example, the input unit 38 may be able to accept a selection input for selecting an area in the feature amount map 400 where the group amount value 440 is indicated, and then accept a designation input from the user for specifying the group amount value 440. After the user specifies the number of users, the user uses the setting screen 450 or the setting screen 460 as described above to set initial centroids in the same number as the specified number of users.

[0172] Furthermore, as described above, when the feature amount map 300 indicating the group number value 340 is displayed, and the user changes the group number value 340 and re-executes feature amount grouping, the user may be allowed to set an initial center of gravity using the setting screen 460 before the re-execution of feature amount grouping. For example, when a feature amount of the feature amount map 300 is selected, the setting screen 460 may be displayed so that the user can set an initial center of gravity. Furthermore, the display unit 37 may display each representative point 301 before the group number value 340 is changed as an initial center of gravity candidate 401, and the user may be allowed to set an initial center of gravity using the setting screen 450 that is displayed when the initial center of gravity candidate 401 is selected.

[0173] In this way, the parameters required for feature grouping are specified by a user (for example, an expert), so that the second grouping section 130 can appropriately perform feature grouping.

[0174] The second grouping unit 130 may set parameters necessary for feature grouping of the feature amounts of the plurality of target images based on the results of feature grouping of the plurality of images acquired before the plurality of target images. In other words, the second grouping unit 130 may set parameters necessary for feature grouping in the attention information collection process based on the results of past feature grouping.

[0175] For example, the second grouping unit 130 may use, as the initial centroids for the feature grouping in the information collection process for the target farm, each of the centroids of the feature groups obtained as a result of the feature grouping in the information collection process performed at the same time as the execution time of the interesting information collection process. In this case, the number of groups obtained by the feature grouping in the information collection process for the target farm will be the same as the number of feature groups obtained as a result of the feature grouping in the information collection process performed at the same time in the past as the execution time of the interesting information collection process. Alternatively, the second grouping unit 130 may use, as the initial centroids for the feature grouping in the interesting information collection process, each of the centroids of the feature groups obtained as a result of the feature grouping in the information collection process performed at the same time in the past as the execution time of the interesting information collection process for another farm. In this case, the number of groups obtained by the feature grouping in the information collection process of interest will be the same as the number of feature groups obtained as a result of the feature grouping in the information collection process performed for the other farm at the same time in the past as the time when the information collection process of interest was performed (i.e., the information collection process performed for the other farm at the same time in the past).

[0176] In this way, the parameters required for feature grouping of the features of multiple target images are set based on the results of feature grouping of multiple images acquired before the multiple target images, allowing the second grouping unit 130 to appropriately perform feature grouping.

[0177] The second grouping unit 130 may have a reference feature that serves as a reference for each of the multiple feature groups, and may set parameters necessary for feature grouping based on the reference feature. For example, the second grouping unit 130 may use the position of the reference feature that serves as the reference for a certain feature group in the feature space as the initial center of gravity of the certain feature group. In this case, the number of groups will be the same as the number of reference features.

[0178] The reference feature may be a feature of an image showing a crop in a standard growth state. In this example, for example, a plurality of standard growth states are defined for each growth stage of a crop. The standard growth states can also be considered to be representative growth states. For example, the reference feature of a plurality of feature groups used in the information collection process executed at a certain growth stage may be a feature of a plurality of images showing a crop in a plurality of standard growth states at the certain growth stage.

[0179] The representative image of a certain processing target group acquired by the representative image acquisition unit 131 may be an image having the same feature amount as the reference feature amount of the feature amount group corresponding to the certain processing target group. Furthermore, as the representative image of a certain processing target group in the information collection process of interest, a target image having a feature amount at the same or a nearby position in the feature amount space as the representative point 301 of the certain feature amount group may be adopted from among multiple target images obtained in an information collection process executed at the same time in the past for the target farm (i.e., an information collection process executed at the same time in the past for the target farm as the execution time of the information collection process of interest). Furthermore, as the representative image of a certain processing target group in the information collection process of interest, a target image having a feature amount at the same or a nearby position in the feature amount space as the representative point 301 of the certain feature amount group may be adopted from among multiple target images obtained in an information collection process executed at the same time in the past for another farm.

[0180] 31, the control unit 30 of the server 3 may include an attention need determination unit 135 that determines whether a treatment target shown in a target image requires attention based on the target image. The attention need determination unit 135 is a functional block formed by the CPU of the control unit 30 executing a program 31a in the storage unit 31. The attention need determination unit 135 can determine whether a treatment target requires attention using, for example, two types of determination methods.

[0181] <First judgment method> In this example, when the feature of a target image is located outside the predetermined range 302 of the feature group to which it belongs in the feature space, the caution required determination unit 135 determines that the treatment target shown in the target image requires caution. In this case, the predetermined range 302 can also be considered as an allowable range.

[0182] Here, if the feature amount of a certain target image is located outside the predetermined range 302 of the feature amount group to which it belongs, it can be said that the degree of similarity between the certain target image and the representative image of the treatment target group to which the treatment target appearing in the certain target image belongs is low. It can also be said that the attention need determination unit 135 determines whether the treatment target appearing in the certain target image requires attention based on the degree of similarity between the target image and the representative image of the treatment target group to which the treatment target appearing in the certain target image belongs. If the degree of similarity between the target image and the representative image of the treatment target group to which the treatment target appearing in the certain target image belongs is low, the attention need determination unit 135 determines that the treatment target appearing in the certain target image requires attention.

[0183] <Second judgment method> In this example, the caution need determination unit 135 determines that a treatment target appearing in a target image having a feature that is equidistant from the representative point 301 of two feature groups in the feature space needs attention. Specifically, for example, the caution need determination unit 135 determines that a treatment target appearing in a target image having a feature that is equidistant from the representative point 301 of two feature groups in the feature space and that belongs to one of the two feature groups needs attention. In other words, if the feature of the target image is equidistant from the representative point 301 of one feature group to which the feature belongs and from the representative point 301 of a feature group other than the one feature group, the caution need determination unit 135 determines that the treatment target appearing in the target image needs attention.

[0184] Here, when the feature of a certain target image is equidistant from the representative points 301 of two feature groups in the feature space, it can be said that the degree of similarity of the certain target image to the representative images of the two treatment target groups is the same. Therefore, it can be said that the caution need determination unit 135 determines whether the treatment target appearing in the target image needs attention based on the degree of similarity of the target image to the representative images of the two treatment target groups. When the degree of similarity of the target image to the representative images of the two treatment target groups is the same, the caution need determination unit 135 determines that the treatment target appearing in the target image needs attention.

[0185] The display unit 37 may display a target image showing a treatment target (also referred to as a treatment target requiring caution) that has been determined to require caution by the caution need determination unit 135. For example, the display unit 37 may display the target image showing the treatment target requiring caution together with the feature amount map 300.

[0186] 13 above, a target image 150 having a feature located outside the predetermined range 302 of the feature group, that is, a target image 150 having a low degree of similarity to the representative image of the treatment target group to which the treatment target depicted in the target image 150 belongs, is displayed, and therefore it can be said that Fig. 13 shows an example of a display of a target image depicting a treatment target requiring caution. Also, Fig. 19 above shows a target image 150 having a feature located at the same distance from the representative points 301 of the two feature groups, that is, a target image 150 having the same degree of similarity to the representative images of the two treatment target groups, and therefore it can be said that Fig. 19 shows an example of a display of a target image depicting a treatment target requiring caution.

[0187] In the feature amount map 300, when the input unit 38 receives a selection input for selecting the position of a feature amount of a target image in which a suspicious action-requiring object is shown, the display unit 37 may display the target image together with information indicating that the target image in which the object is shown requires attention. Furthermore, when the input unit 38 receives a selection input for selecting the position of a feature amount of a target image in which a suspicious action-requiring object is shown, the display unit 37 may display the target image together with information indicating that the target image in which the object is shown requires attention on the setting screen 380 (see FIG. 24 ). Furthermore, in the feature amount map 300, the position of the feature amount of a target image in which a suspicious action-requiring object is shown may be displayed in a different manner from the positions of other feature amounts. For example, the circle 305 indicating the position of the feature amount of a target image in which a suspicious action-requiring object is shown may be displayed larger than the circle 305 indicating the positions of other feature amounts.

[0188] When a treatment target group includes multiple targets requiring careful treatment, the display unit 37 may, for example, display a list screen showing a list of multiple target images in which the multiple targets requiring careful treatment appear, in response to a predetermined input to the input unit 38. Furthermore, when multiple treatment target groups each include multiple targets requiring careful treatment, the display unit 37 may, for example, display a list screen showing a list of multiple target images in which the multiple targets requiring careful treatment appear, for each treatment target group, in response to a predetermined input to the input unit 38.

[0189] The caution-required determination unit 135 may determine whether the treatment target requires caution using only one of the first and second determination methods.

[0190] In the above example, the server 3 transmits the generated treatment information to the terminal device 4, but the treatment information may also be transmitted to an automatic work machine that automatically performs treatment (for example, fertilizer supply) on crops in the farm. In this case, the automatic work machine performs treatment on the crops in the farm based on the received treatment information.

[0191] <Other examples of information gathering processes> Next, another example of the information collection process will be described. Hereinafter, the example of the information collection process described above will be referred to as the first information collection process, and the other example of the information collection process described below will be referred to as the second information collection process. Furthermore, in the description of the second information collection process, the term "interesting information collection process" refers to the interesting second information collection process.

[0192] Fig. 32 is a schematic diagram showing an example of a plurality of functional blocks included in a processing system 1 (also referred to as processing system 1a) that performs the second information collection process. As shown in Fig. 32, in processing system 1a, compared to the processing system 1 shown in Fig. 4, terminal device 4 further includes an attention need determination unit 145. Similar to the above-mentioned attention need determination unit 135 (see Fig. 31), the attention need determination unit 145 determines whether a treatment target shown in a target image requires attention, based on the target image. The attention need determination unit 145 is a functional block formed when the CPU of control unit 40 executes program 41a in storage unit 41.

[0193] Figure 33 is a schematic diagram showing the second information collection process. As shown in Figure 33, in the second information collection process, step s21 is first executed. In step s21, the terminal device 4 acquires information required for the second information collection process (also referred to as first required information) from the server 3. The terminal device 4 receives the first required information from the server 3, for example, when it is located indoors where the communication environment with the server 3 is good (for example, inside a building near the target farm). The server 3 generates the first required information and transmits it to the terminal device 4. The terminal device 4 stores the first required information from the server 3 in the memory unit 41.

[0194] The first required information in the attention information collection process includes, for example, the results of past treatment target grouping by the server 3. For example, the first required information in the attention information collection process includes the results of treatment target grouping in an information collection process executed for the target farm at the same time in the past as the time when the attention information collection process is executed. Hereinafter, treatment target grouping in an information collection process executed for the target farm at the same time in the past as the time when the attention information collection process is executed will also be simply referred to as treatment target grouping for the same time in the past.

[0195] The first required information in the attention information collection process includes, for example, for each process target group obtained as a result of process target grouping performed at the same time in the past, information for identifying a feature group corresponding to the process target group, information for identifying the representative point 301 and predetermined range 302 of the feature group, and process target IDs of the multiple process targets that make up the process target group. In the first required information, for each feature group, information for identifying the feature group, information for identifying the representative point 301 and predetermined range 302 of the feature group, and the process target IDs of each process target that make up the process target group that corresponds to the feature group are associated with one another.

[0196] In the first required information, the information for identifying a feature group and the treatment target IDs of each treatment target that constitutes the treatment target group corresponding to that feature group are associated with each other. This can also be seen as the feature group and the treatment target IDs of each treatment target that constitutes the treatment target group corresponding to that feature group being associated with each other in the first required information.

[0197] Furthermore, in the first required information, the information for identifying the representative point 301 and the specified range 302 of the feature group and the processing target IDs of each processing target that constitutes the processing target group corresponding to the feature group are associated with each other. This can also be seen as the first required information being associated with the representative point 301 and the specified range 302 of the feature group and the processing target IDs of each processing target that constitutes the processing target group corresponding to the feature group.

[0198] It can be said that the treatment target appearing in the target image acquired in step s21 of the attention information collection process previously belonged to the treatment target group corresponding to the feature group to which the treatment target ID of the treatment target is associated in the first required information.

[0199] In addition, the first required information in the information collection process of interest includes, for example, the dimension reduction transformation formula used in the information collection process executed for the target farm at the same time in the past as the time when the information collection process of interest is executed.

[0200] After step s21, the vehicle 10 equipped with the terminal device 4 etc. travels within the target farm, and the camera 5 photographs each of the crops in the target farm. The terminal device 4 then executes steps s1 and s2 described above. As a result, feature amounts of multiple target images each showing multiple crops in the target farm are acquired.

[0201] After step s2, step s22 is executed in the terminal device 4. In step s22, the caution requirement determination unit 145 of the terminal device 4 determines whether the treatment target shown in each target image acquired in step s1 requires caution.

[0202] In step s22, the caution need determination unit 145 first reduces the number of dimensions of the feature amounts of each target image acquired in step s2. The caution need determination unit 145 reduces the number of dimensions of the feature amounts of the target images using a dimension reduction transformation formula included in the first necessary information acquired in step s21. Then, the caution need determination unit 145 arranges the feature amounts of the multiple target images in a feature amount space. For example, if the number of dimensions of the feature amounts has been reduced to two dimensions, the feature amount space becomes two-dimensional.

[0203] Next, the attention need determination unit 145 arranges each representative point 301 and each predetermined range 302 identified from the first necessary information acquired in step s21 in a feature amount space. Next, the attention need determination unit 145 determines, for each target image, whether a treatment target appearing in the target image requires attention. The attention need determination unit 145 can determine whether a treatment target appearing in a target image requires attention using, for example, two types of determination methods.

[0204] <First determination method in second information collection process> In this example, the attention need determination unit 145 determines that the treatment target appearing in the target image requires attention if, in the feature amount space, the feature amount of the target image is located outside the predetermined range 302 of the feature amount group to which the treatment target ID of the treatment target appearing in the target image is associated in the first necessary information. That is, the attention need determination unit 145 determines that the treatment target appearing in the target image requires attention if, in the feature amount space, the feature amount of the target image is located outside the predetermined range 302 of the feature amount group corresponding to the treatment target group to which the treatment target appearing in the target image previously belonged. In the attention information collection process, if the feature amount of the target image is located outside the predetermined range 302 of the feature amount group corresponding to the treatment target group to which the treatment target appearing in the target image previously belonged, it can be said that the similarity between the target image and the representative image of the treatment target group to which the treatment target appearing in the target image previously belonged is low. It can also be said that the attention need determination unit 145 determines whether the treatment target shown in the target image needs attention based on the similarity between the target image and a representative image of a treatment target group to which the treatment target shown in the target image previously belonged in the attention information collection process. In the attention information collection process, if the similarity between the target image and a representative image of a treatment target group to which the treatment target shown in the target image previously belonged is low, the attention need determination unit 145 determines that the treatment target shown in the target image needs attention.

[0205] <Second determination method in second information collection process> In this example, the caution need determination unit 145 determines that a treatment target appearing in a target image having a feature that is equidistant from two representative points 301 in the feature amount space needs attention. Specifically, for example, when a target image has a feature that is equidistant from two representative points 301 in the feature amount space and the treatment target ID of the treatment target appearing in the target image is associated with one of the two representative points 301 in the first necessary information, the caution need determination unit 145 determines that a treatment target appearing in the target image needs attention.

[0206] Here, a feature group having a representative point 301 with which the processing target ID of the processing target appearing in the target image is associated in the first necessary information (i.e., a feature group with which the processing target ID of the processing target appearing in the target image is associated in the first necessary information) is referred to as one feature group corresponding to the target image. Also, the processing target of interest is referred to as the processing target of interest, and the target image in which the processing target of interest appears is referred to as the processing target image of interest. Also, a processing target group to which the processing target of interest previously belonged is referred to as the processing target group to which it previously belonged.

[0207] It can also be said that the attention treatment target needs attention determination unit 145 determines that the attention treatment target needs attention when the feature amount of the attention target image is equidistant from the representative point 301 of one feature amount group corresponding to the attention target image and the representative point 301 of a feature amount group other than the one feature amount group. In other words, it can also be said that the attention treatment target needs attention when the feature amount of the attention target image is equidistant from the representative point 301 of the feature amount group corresponding to the previous target group and the representative point 301 of the feature amount group other than the one feature amount group.

[0208] When the feature of the image of interest is equidistant from the representative point 301 of one feature group corresponding to a previously-belonged processing target group and the representative point 301 of another feature group, it can be said that the similarity of the image of interest between the representative image of the previously-belonged processing target group and the representative image of the other processing target group is about the same. It can also be said that the attention need determination unit 145 determines that the attention target needs attention based on the similarity of the image of interest between the representative image of the previously-belonged processing target group and the representative image of the other processing target group. The attention need determination unit 145 determines that the attention target needs attention when the similarity of the image of interest between the representative image of the previously-belonged processing target group and the representative image of the other processing target group is about the same.

[0209] The caution-required determination unit 145 may determine whether the treatment target requires caution by using only one of the first determination method and the second determination method.

[0210] In this manner, when it is determined for each target image whether the treatment target appearing in the target image requires attention, in step s23, the treatment target information generation unit 142 generates first treatment target information for each treatment target.

[0211] In this example, the first treatment target information for a treatment target (also called a treatment target requiring caution) that has been determined to require caution by the caution determination unit 145 includes, for example, information indicating that the first treatment target information is information regarding a treatment target requiring caution, a target image in which the treatment target requiring caution is shown, features of the target image, treatment target position information for the treatment target requiring caution, and a treatment target ID for the treatment target requiring caution.

[0212] On the other hand, the first treatment target information for a treatment target (also called a non-attention treatment target) that is not determined to require attention by the attention determination unit 145 includes the feature amount of the target image in which the non-attention treatment target appears, the treatment target position information of the non-attention treatment target, and the treatment target ID of the non-attention treatment target, but does not include the target image in which the non-attention treatment target appears.

[0213] After step s23, in step s24, the terminal device 4 transmits the plurality of pieces of first process target information generated by the process target information generating unit 142 to the server 3.

[0214] Thus, in the second information collection process, the terminal device 4 transmits to the server 3 target images showing targets requiring attention to be treated, but does not transmit to the server 3 target images showing targets not requiring attention to be treated. This makes it possible to reduce the amount of communication between the terminal device 4 and the server 3. Therefore, for example, even if the communication status between the terminal device 4 and the server 3 is not good, the terminal device 4 can transmit to the server 3 multiple pieces of first treatment target information generated by the treatment target information generation unit 142.

[0215] When step s24 is executed, the server 3 executes steps s11 to s15 described above, similar to the first information collection process. Although the terminal device 4 does not transmit any target images that include targets not requiring treatment among the multiple target images acquired in step s1, the server 3 transmits all feature amounts of the multiple target images acquired in step s1. Therefore, the server 3 can perform feature grouping in the same manner as described above. Furthermore, in the second information collection process, the server 3 receives target images that include targets requiring treatment, and can display the target images that include targets requiring treatment in the same manner as described above. Therefore, an expert user of the server 3 can visually confirm the growth status of the target requiring treatment by checking the target images that include the target requiring treatment on the display screen of the display unit 37. When step s15 is executed and treatment information is generated, the second information collection process ends.

[0216] As described above, in the second information collection process, the terminal device 4 does not transmit target images that depict targets not requiring attention to be treated to the server 3. Therefore, in the attention information collection process, the server 3 may not acquire some of the multiple target images acquired by the terminal device 4 in the attention information collection process. In the attention information collection process, if the server 3 does not acquire a current target image having a feature at a certain position in the feature space, the server 3 may use, instead of the current target image, a previous target image having a feature at the same or a similar position in the feature space from multiple past target images obtained in information collection processes performed at the same time in the past for the target farm or another farm. For example, in the attention information collection process, if the server 3 does not acquire a current target image having a feature at the same or a similar position in the feature space as the representative point 301 in the attention information collection process, the server 3 may use, as the representative image corresponding to the representative point 301, a previous target image having a feature at the same or a similar position in the feature space from multiple past target images obtained in information collection processes performed at the same time in the past for the target farm or another farm.

[0217] Furthermore, in the attention information collection process, if the server 3 does not receive a current target image containing a certain treatment target, instead of the current target image, it may include in the second treatment target information regarding the certain treatment target a past target image that has features in the feature space that are at the same or close to the features of the current target image, from among multiple past target images obtained in information collection processes performed at the same time in the past for the target farm or other farms.

[0218] <Example of monitoring process> Fig. 34 is a schematic diagram showing an example of a plurality of functional blocks included in the processing system 1 that performs the monitoring process. As shown in Fig. 34, the processing system 1 that performs the monitoring process includes, for example, the above-mentioned first grouping unit 100, feature amount map generation unit 132, treatment content identification unit 133, treatment information generation unit 134, image processing unit 141, treatment target information generation unit 142, and attention required determination unit 145 as functional blocks.

[0219] FIG. 35 is a schematic diagram showing an example of a monitoring process. At each growth stage, the monitoring process shown in FIG. 35 is executed multiple times. Hereinafter, when focusing on a particular monitoring process, the particular monitoring process may be referred to as a target monitoring process. Furthermore, the growth stage at which the target monitoring process is executed may be referred to as a target growth stage. Furthermore, in the description of the monitoring process, a target information collection process refers to an information collection process executed at a target growth stage.

[0220] As shown in FIG. 35, in the monitoring process, step s51 is first executed. In step s51, the terminal device 4 acquires information required for the monitoring process (also referred to as second required information) from the server 3. The terminal device 4 receives the second required information from the server 3, for example, when the terminal device 4 is located indoors where the communication environment with the server 3 is good (for example, inside a building near the target farm). The server 3 generates the second required information and transmits it to the terminal device 4. The terminal device 4 stores the second required information from the server 3 in the memory unit 41.

[0221] The second required information for the attention monitoring process includes, for example, the latest results of feature grouping at the attention growth stage. For example, the second required information for the attention monitoring process includes information on the latest N feature groups at the attention growth stage. For example, the second required information includes, for each of the latest N feature groups, information for identifying the feature group and the multiple features constituting the feature group. Furthermore, the second required information includes, for each of the latest N feature groups, information for identifying the feature group and information for identifying the representative point 301 and the predetermined range 302 of the feature group. Furthermore, the second required information includes, for each of the latest N feature groups, correspondence between the feature and the treatment target ID corresponding to the feature. Here, the treatment target ID corresponding to a certain feature means the treatment target ID of the treatment target appearing in the treatment target image having the certain feature. Furthermore, the second required information for the attention monitoring process includes, for example, the dimensionality reduction transformation formula used in the information collection process at the attention growth stage.

[0222] For example, consider a case where the attention monitoring process is the first monitoring process executed after the attention information collection process during the attention growth stage. In this case, the N feature groups finally obtained in the attention information collection process are the most recent N feature groups.

[0223] Also, consider a case where the attention monitoring process is a monitoring process executed for the second or subsequent time after the attention information collection process during the attention growth stage. In this case, the N feature groups finally obtained in the monitoring process executed immediately before the attention monitoring process become the most recent N feature groups.

[0224] After step s51, a vehicle 10 equipped with a terminal device 4 and the like travels within the target farm, and a camera 5 photographs each of the crops in the target farm. The terminal device 4 then executes steps s1 and s2 described above. This results in the acquisition of multiple target images each capturing multiple crops in the target farm, along with feature values ​​for the multiple target images and crop location information (in other words, treatment location information) for the multiple crops. Hereinafter, in the description of the monitoring process, in order to distinguish between the feature values ​​acquired in step s2 and the feature values ​​included in the second required information, the feature values ​​acquired in step s2 may be referred to as new feature values, and the feature values ​​included in the second required information may be referred to as the previous feature values. The target image acquired in step s1 may also be referred to as the new target image.

[0225] After step s2, step s52 is executed in the terminal device 4. In step s52, the caution requirement determination unit 145 of the terminal device 4 determines whether each treatment target requires caution, based on the multiple new target images obtained in step s2 and the second necessary information.

[0226] The attention need determination unit 145 may determine whether the treatment target appearing in the new target image requires attention, for example, based on the degree of similarity between the new target image and a representative image of a treatment target group to which the treatment target appearing in the new target image belongs. Also, the attention need determination unit 145 may determine whether the treatment target appearing in the new target image requires attention, for example, based on the degree of similarity between the new target image and representative images of the two treatment target groups.

[0227] Furthermore, the attention need determination unit 145 may determine whether an object to be treated needs attention based on, for example, the degree of change over time in the target image in which the object to be treated appears. Furthermore, the attention need determination unit may determine whether an object to be treated needs attention based on the direction of change over time in the target image in which the object to be treated appears. The operation of the attention need determination unit 145 in step s52 will be described in detail later.

[0228] After step s52, in step s53, the treatment target information generating unit 142 generates first treatment target information for each treatment target.

[0229] In this example, the first action target information about the action target determined to be in need of attention by the attention need determination unit 145, that is, the first action target information about the action target requiring attention (also referred to as action target information requiring caution), includes, for example, information indicating that the first action target information is information about the action target requiring caution, a new target image in which the action target requiring caution is shown, feature amounts of the new target image, action target position information of the action target requiring caution, and a action target ID of the action target requiring caution. The multiple pieces of first action target information generated in step s53 include at least one piece of action target information requiring caution.

[0230] On the other hand, the first treatment target information for a treatment target that is not determined to require attention by the attention determination unit 145, i.e., the first treatment target information for a non-attention treatment target, includes the feature quantities of a new target image in which the non-attention treatment target appears, the treatment target position information of the non-attention treatment target, and the treatment target ID of the non-attention treatment target, but does not include a new target image in which the non-attention treatment target appears.

[0231] After step s53, in step s54, the terminal device 4 transmits the plurality of pieces of first processing target information generated by the processing target information generating unit 142 to the server 3.

[0232] In this way, in the monitoring process, similarly to the second information collection process described above, the terminal device 4 transmits target images showing targets requiring attention to the server 3, but does not transmit target images showing targets not requiring attention to the server 3. This makes it possible to reduce the amount of communication between the terminal device 4 and the server 3.

[0233] After step s54 is executed, the server 3 executes step s61. In step s61, the second grouping unit 130 updates each of the N feature groups based on a plurality of new feature amounts included in each of the plurality of first processing target information from the terminal device 4.

[0234] Here, in the monitoring process of this example, no treatment target grouping is performed. Therefore, in the monitoring process of this example, feature grouping (i.e., clustering of multiple features) is also not performed. The number of N treatment target groups in the attention monitoring process is the same as the number of N treatment target groups finally determined in the attention information collection process. Furthermore, for each of the N treatment target groups, the multiple treatment targets that make up one treatment target group in the attention monitoring process are the same as the multiple treatment targets that make up that one treatment target group finally determined in the attention information collection process. In this example, the N treatment target groups are fixed at each development stage.

[0235] In step s61, the second grouping unit 130 updates one feature group corresponding to one processing target group by replacing the multiple feature amounts constituting the one feature group corresponding to the one processing target group (i.e., the multiple previous feature amounts included in the second necessary information) with multiple new feature amounts corresponding to the multiple processing targets constituting the one processing target group, which have been sent from the terminal device 4. The new feature amounts corresponding to the processing target are the feature amounts of the new target image containing the processing target, acquired in step s2. The second grouping unit 130 performs this update process for each of the N feature groups corresponding to the N processing target groups.

[0236] After step s61, in step s62, the feature amount map generation unit 132 updates the feature amount map 300 based on the N feature amount groups updated in step s61. When the input unit 38 receives a predetermined input, the display unit 37 may display the updated feature amount map 300.

[0237] Next, in step s63, the feature amount map generating unit 132 generates a caution-requiring feature amount map 500 that indicates the distribution in feature amount space of the feature amounts included in at least one piece of caution-requiring action target information from the terminal device 4. The caution-requiring feature amount map 500 generated in step s63 is displayed on the display unit 37 in step s64. Hereinafter, the feature amounts included in the caution-requiring action target information, that is, the feature amounts of the target image in which the caution-requiring action target is shown, may be referred to as caution-requiring feature amounts.

[0238] FIG. 36 is a schematic diagram showing a display example of a caution feature map 500. As shown in FIG. 36, in the caution feature map 500, the position of each caution feature in the feature space is indicated by, for example, a circle 505. In the caution feature map 500, similar to the feature map 300, the positions of multiple caution feature values ​​are indicated in different ways for each feature group. In the example of FIG. 36, a circle 505a representing the position of a caution feature included in the first feature group is indicated with horizontal hatching. A circle 505b representing the position of a caution feature included in the second feature group is indicated with sand hatching. A circle 505c representing the position of a caution feature included in the third feature group is indicated with hatching sloping upward to the left.

[0239] 36 , the attention-requiring feature map 500 may show, for example, a representative point 301 and a predetermined range 302 in the feature space of each feature group. When the input unit 38 receives a selection input for selecting the representative point 301 of a feature group, the display unit 37 may display a representative image 310 of the treatment target group corresponding to the feature group.

[0240] Furthermore, the attention-requiring feature map 500 may show, for example, loci 308 of points equidistant from the representative point 301 of the two feature amount groups. In the example of Fig. 36, the attention-requiring feature map 500 shows a first locus 308a of points equidistant from the representative point 301a of the first feature amount group and the representative point 301b of the second feature amount group, a second locus 308b of points equidistant from the representative point 301b of the second feature amount group and the representative point 301c of the third feature amount group, and a second locus 308c of points equidistant from the representative point 301a of the first feature amount group and the representative point 301c of the third feature amount group.

[0241] When the display unit 37 is displaying the attention-requiring feature map 500, if the input unit 38 receives a selection input for selecting a position 515 of a attention-requiring feature in the attention-requiring feature map 500, the display unit 37 may display a setting screen 580 similar to the above-mentioned setting screen 380, as shown in Fig. 37. Hereinafter, a attention-requiring feature whose position has been selected by the user in the attention-requiring feature map 500 may be referred to as a selected attention-requiring feature.

[0242] The setting screen 580 is a screen for changing the feature group to which the selected caution-requiring feature belongs. The setting screen 580 includes, for example, a new target image 150 having the selected caution-requiring feature. It can also be said that when the attention-requiring determination unit 145 determines that the treatment target requires attention, the display unit 37 displays a new target image 150 in which the treatment target appears.

[0243] The setting screen 580 also includes an instruction area 581 for instructing that the selected cautionary feature should be set to belong to the first group. The setting screen 580 also includes an instruction area 582 for instructing that the selected cautionary feature should be set to belong to the second group. The setting screen 580 also includes an instruction area 583 for instructing that the selected cautionary feature should be set to belong to the third group.

[0244] In the example of FIG. 37 , the position 515 of the caution feature belonging to the second group (i.e., the second feature group) is selected, and the selected caution feature is included in the second group. When the input unit 38 receives a selection input selecting the designation region 581, the second grouping unit 130 changes the feature group to which the selected feature belongs from the second group to the first group. Furthermore, when the input unit 38 receives a selection input selecting the designation region 583, the second grouping unit 130 changes the feature group to which the selected feature belongs from the second group to the third group. In the example of FIG. 37 , the selection of the designation region 581 and the designation region 583 can also be said to be an instruction to change the feature group to which the selected caution feature belongs. When the input unit 38 receives an instruction to change the feature group to which the selected caution feature belongs, the second grouping unit 130 changes the feature group to which the selected caution feature belongs in accordance with the change instruction. When the input unit 38 receives a selection input for selecting the designation area 582, the second grouping unit 130 does not change the affiliation of the selected feature amount, but keeps it in the second group.

[0245] When the feature amount group to which the selected attention-requiring feature belongs is changed, the feature amount map generation unit 132 updates the attention-requiring feature map 500 and the feature amount map 300. The display unit 37 displays the updated attention-requiring feature map 500. In this example, since the action content is set for each action target group, in other words, for each feature amount group, when the feature amount group to which the selected attention-requiring feature belongs is changed, the action content for the action target appearing in the new target image 150 having the selected attention-requiring feature changes. Therefore, when the feature amount group to which the selected attention-requiring feature belongs is changed, the action information generation unit 134 updates the action information in the storage unit 31 (step s65). The updated action information is saved in the storage unit 31.

[0246] The setting screen 580 may not include a designation area corresponding to the feature group to which the selected attention-requiring feature currently belongs, among the designation areas 581, 582, and 583. In the example of Fig. 37 , the setting screen 580 may not include the designation area 582.

[0247] Furthermore, when the position of a cautionary feature on the locus 308 of points equidistant from the representative point 301 of a certain feature group and the representative point 301 of another feature group is selected, the setting screen 580 may include only the designation area corresponding to the certain feature group and the designation area corresponding to the other feature group out of the designation areas 581, 582, and 583.

[0248] Furthermore, when a feature that is included in a certain feature group and whose position is on locus 308 of points equidistant from representative point 301 of the certain feature group and representative point 301 of another feature is selected, setting screen 580 may include only the designation area corresponding to the other feature group out of designation areas 581, 582, and 583. In the example of FIG. 37 , setting screen 580 may include only designation area 581 out of designation areas 581, 582, and 583.

[0249] Selection of the position of a caution-requiring feature in the caution-requiring feature map 500 can also be seen as selection of a caution-requiring action target appearing in a target image having the caution-requiring feature. In this example, when the feature group to which the selected caution-requiring feature belongs is changed, the action target group to which the caution-requiring action target (also referred to as the selected caution-requiring action target) appearing in the target image having the selected caution-requiring feature belongs is changed. Therefore, the setting screen 580 can also be said to be a screen for changing the action target group to which the selected caution-requiring action target belongs. Furthermore, the instruction area 581 included in the setting screen 580 can be said to be an instruction area for instructing to set the affiliation of the selected caution-requiring action target to the first action target group corresponding to the first group. Similarly, the instruction area 582 included in the setting screen 580 can be said to be an instruction area for instructing to set the affiliation of the selected caution-requiring action target to the second action target group corresponding to the second group. Furthermore, the instruction area 583 included in the setting screen 580 can be said to be an instruction area for instructing to set the affiliation of the selected caution-requiring action target to the third action target group corresponding to the third group.

[0250] When the selected attention-requiring treatment target belongs to the first treatment target group, the selection in the instruction area 582 and the instruction area 583 can be said to be an instruction to change the treatment target group to which the selected attention-requiring treatment target belongs. When the input unit 38 receives an instruction to change the treatment target group to which the selected attention-requiring treatment target belongs, the second grouping unit 130 changes the treatment target group to which the selected attention-requiring treatment target belongs in accordance with the change instruction.

[0251] When the display unit 37 displays the caution-requiring feature amount map 500, the input unit 38 can receive from the user a treatment content for the caution-requiring treatment target (also referred to as a treatment content for the caution-requiring treatment target). That is, the user can set a treatment content for the caution-requiring treatment target.

[0252] As described above, a common action content is set for a plurality of action targets that make up one action target group. However, for a caution-requiring action target, the user can exceptionally set a special action content for the caution-requiring action target, rather than the action content common to the action target group to which the caution-requiring action target belongs.

[0253] For example, when the input unit 38 receives a selection input from the user selecting the position of a certain caution-requiring feature included in the displayed caution-requiring feature map 500, it can receive from the user a caution-requiring target treatment content for the certain caution-requiring target. This selection input is different from the selection input when the setting screen 580 is displayed.

[0254] As in step s14 described above, the voice input unit of the input unit 38 accepts natural language spoken by the user, which expresses the treatment content for the attention-required target. Then, as in step s14, the treatment content identification unit 133 identifies the treatment content for the attention-required target expressed by the natural language accepted by the voice input unit.

[0255] In this example, a representative image showing the target requiring special attention is displayed, and the user can check the representative image showing the target requiring special attention and then input the target requiring special attention action content for the target requiring special attention into the input unit 38. For example, consider a case where the action content common to the first target group is to increase the amount of fertilizer by 10%. In this case, the user can check the target image 150 showing the target requiring special attention that belongs to the first target group on the display surface of the display unit 37, and then exceptionally input, for example, the action content for the target requiring special attention, to the input unit 38, to increase the amount of fertilizer by 20%.

[0256] When the treatment content specifying unit 133 specifies the treatment content for the target requiring attention that is received by the input unit 38, the treatment information generating unit 134 updates the treatment information so that the treatment information indicates the treatment content for the target requiring attention as the treatment content for the target requiring attention (step s65). The updated treatment information is stored in the storage unit 31.

[0257] In this way, the input unit 38 can receive the attention-requiring treatment content for the attention-requiring treatment target from the user, and therefore the convenience of the processing system 1 is improved.

[0258] When the user has completed the necessary input to the input unit 38, the user performs a notification input to the input unit 38 to notify that the necessary input has been completed. When the input unit 38 accepts the notification input, the server 3 transmits the updated treatment information in the storage unit 31 to the terminal device 4. The terminal device 4 stores the updated treatment information from the server 3 in the storage unit 41. Then, the terminal device 4 displays the updated treatment information on the display unit 47 as described above. The worker, who is the user of the terminal device 4, performs treatment on each crop in the target farm in accordance with the treatment content indicated in the updated treatment information displayed on the display unit 47.

[0259] A treatment target that has been determined to require attention in the attention monitoring process and for which a treatment content for a treatment target requiring attention has been set may no longer be determined to require attention in step s52 of another monitoring process in the attention development stage that is executed after the attention monitoring process. In this case, in the other monitoring process, the treatment information generation unit 134 updates the treatment information so that the treatment information indicates, as the treatment content for the treatment target, not the treatment content for the treatment target requiring attention, but the treatment content corresponding to the treatment target group to which the treatment target belongs (in other words, the treatment content common to the treatment target group to which the treatment target belongs). As a result, when a treatment target that was determined to require attention is no longer determined to require attention, the treatment content for the treatment target will return from the exceptional treatment content for a treatment target requiring attention to the original treatment content (in other words, the original treatment content).

[0260] Furthermore, even if a treatment target is determined to require attention in the attention monitoring process, if treatment content for the treatment target requiring attention is not set, the treatment content for the treatment target will remain the same even if it is no longer determined to require attention in another monitoring process.

[0261] As a result of the above-described monitoring process for the target farm being performed multiple times at each growth stage each year, the memory unit 41 of the terminal device 4 accumulates the second necessary information used in the monitoring process performed multiple times at each growth stage each year for the target farm.

[0262] <Example of operation of attention-required determination unit in monitoring process> In step s52, the caution need determination unit 145 first reduces the number of dimensions of each of the multiple new feature amounts acquired in step s2. The caution need determination unit 145 reduces the number of dimensions of the new feature amounts by using a number-of-dimensions reduction transformation formula included in the second necessary information acquired in step s51.

[0263] Next, the attention need determination unit 145 arranges multiple new feature amounts in the feature amount space. For example, if the number of dimensions of the new feature amounts is reduced to two, the feature amount space becomes two-dimensional. Furthermore, the attention need determination unit 145 arranges N representative points 301 and N predetermined ranges 302 identified from the second necessary information in the feature amount space. Furthermore, the attention need determination unit 145 arranges multiple previous feature amounts included in the second necessary information in the feature amount space.

[0264] Next, the caution need determination unit 145 determines whether or not each treatment target needs attention for the treatment target. The caution need determination unit 145 can determine whether or not a treatment target needs attention using, for example, four types of determination methods.

[0265] <First judgment method in monitoring process> In this example, if the feature of a new target image (in other words, the new feature) is located outside the predetermined range 302 of the feature group corresponding to the treatment target group to which the treatment target appearing in the new target image belongs in the feature space, the attention need determination unit 145 determines that the treatment target appearing in the new target image needs attention. The attention need determination unit 145 can identify the predetermined range 302 of the feature group corresponding to the treatment target group to which the treatment target appearing in the new target image belongs, based on the second necessary information. In the second necessary information, the predetermined range 302 of the feature group to which the feature associated with the treatment target ID of the treatment target appearing in the new target image belongs becomes the predetermined range 302 of the feature group corresponding to the treatment target group to which the treatment target appearing in the new target image belongs.

[0266] Here, if the feature amount of the new target image is located outside the predetermined range 302 of the feature amount group corresponding to the treatment target group to which the treatment target appearing in the new target image belongs, it can be said that the similarity between the new target image and the representative image of the treatment target group to which the treatment target appearing in the new target image belongs is low. Therefore, it can be said that the attention need determination unit 145 determines whether the treatment target appearing in the new target image needs attention based on the degree of similarity between the new target image and the representative image of the treatment target group to which the treatment target appearing in the new target image belongs. If the degree of similarity between the new target image and the representative image of the treatment target group to which the treatment target appearing in the new target image belongs is low, the attention need determination unit 145 determines that the treatment target appearing in the new target image needs attention.

[0267] <Second judgment method in monitoring process> In this example, the attention need determination unit 145 determines that the treatment target appearing in the new target image requires attention if the distance in the feature amount space between the position of the feature amount of the new target image (i.e., the new feature amount) and the position of the previous feature amount corresponding to the treatment target appearing in the new target image is not appropriate. Specifically, the attention need determination unit 145 determines that the treatment target appearing in the new target image requires attention if the distance in the feature amount space between the position of the feature amount of the new target image and the position of the previous feature amount corresponding to the treatment target appearing in the new target image does not exist within a threshold range. The previous feature amount corresponding to the treatment target appearing in the new target image is the feature amount associated with the treatment target ID of the treatment target in the second necessary information.

[0268] The previous feature value corresponding to the processing target appearing in the new target image is the feature value of the target image in which the processing target appears that was acquired before the new target image. The attention-needed determination unit 145 can be said to determine that the processing target appears as requiring attention when the new feature value for the target image in which the processing target appears has changed significantly from the previous feature value or when it has not changed at all from the previous feature value. In other words, the attention-needed determination unit 145 can be said to determine that the processing target appears as requiring attention when the change over time in the target image in which the processing target appears is too large or when the change over time is too small. The attention-needed determination unit 145 can also be said to determine whether the processing target appears as requiring attention based on the degree of change over time in the target image in which the processing target appears.

[0269] FIG. 38 is a schematic diagram showing an example of a position 602a of a new feature and a position 601a of the previous feature for an image of interest that includes the target of interest in feature space. In the example of FIG. 38, the target of interest is included in the first target of interest group. In FIG. 38, "t" is shown, representing the time when the new feature was acquired, and "t-1" is shown, representing the time when the previous feature was acquired. In FIG. 38, the arrow pointing from the previous feature 601a to the position 602a of the new feature indicates the direction of change in the position of the feature of the image of interest in feature space.

[0270] In the example of Fig. 38, the distance between the position 602a of the new feature and the position 601a of the previous feature is greater than the upper limit of the threshold range, and the new feature has changed significantly from the previous feature. In other words, the time change of the image of interest is very large. In such a case, the attention need determination unit 145 determines that the attention target requires attention.

[0271] <Third judgment method in monitoring process> In this example, the attention-required determination unit 145 determines that the treatment target appearing in the new target image requires attention if the vector pointing from the position of the previous feature corresponding to the treatment target appearing in the new target image to the position of the feature (i.e., new feature) of the new target image in the feature space is inappropriate. Specifically, the attention-required determination unit 145 determines that the treatment target appearing in the new target image requires attention if the degree of similarity between the reference vector and the vector (also referred to as a feature change vector) pointing from the position of the previous feature corresponding to the treatment target appearing in the new target image to the position of the feature of the new target image in the feature space is low. It can also be said that the attention-required determination unit 145 determines that the treatment target appears in the new target image requires attention if the degree of similarity between the feature change vector pointing from the position of the previous feature to the position of the new feature in the feature space is low for the target image in which the treatment target appears. The reference vector can also be said to be, for example, an appropriate value of the feature change vector. The reference vector can also be said to be, for example, an appropriate vector.

[0272] Here, the feature amount change vector for the processing target can be said to be a vector representing the direction of time change of the target image in which the processing target appears. Therefore, it can also be said that the attention need determination unit 145 determines whether the processing target requires attention based on the direction of time change of the target image in which the processing target appears. It can also be said that the attention need determination unit 145 determines whether the processing target requires attention based on the degree of similarity between the vector representing the direction of time change of the target image in which the processing target appears and the reference vector.

[0273] The caution need determination unit 145, for example, calculates the cosine similarity between the feature amount change vector and the reference vector. Then, when the calculated cosine similarity is less than a threshold, the caution need determination unit 145 determines that the treatment target needs attention. The threshold is set to, for example, 0 or more and 1 or less. The threshold may be set to, for example, 0.6 or 0.7.

[0274] The reference vector is set individually for each treatment target group. When determining whether a treatment target belonging to a treatment target group requires attention, the attention need determination unit 145 uses the reference vector corresponding to the treatment target group.

[0275] The reference vector in the attention monitoring process may be set by the user of the server 3, for example, in the information collection process at the attention development stage. For example, the user of the server 3 can set the reference vector via the input unit 38 when the display unit 37 is displaying the feature amount map 300 showing the representative points 301 and the predetermined ranges 302. An example of how the user can set the reference vector will be described below.

[0276] For example, consider a case where N=3 and there are a first treatment target group, a second treatment target group, and a third treatment target group. The third treatment target group is a group of crops that grow relatively slowly (see representative image 310c in FIG. 12), the second treatment target group is a group of crops that are growing well (see representative image 310b in FIG. 11), and the third treatment target group is a group of crops that grow relatively quickly (see representative image 310a in FIG. 10).

[0277] In this case, in the feature space, the feature of the third processing target group tends to move toward the position of the feature of the second processing target group over time. For example, the skilled person can set the reference vector ref3 according to the third processing target group by, for example, performing a selection input on the input unit 38 to select the representative point 301c of the third feature group according to the third processing target group, and then performing a selection input to select the representative point 301c of the second feature group according to the second processing target group. In this case, as shown in FIG. 39 , the direction from the representative point 301c of the third feature group toward the representative point 301b of the second feature group becomes the reference vector ref3 according to the third processing target group.

[0278] Furthermore, in the feature space, the feature of the second processing target group tends to move toward the position of the feature of the first processing target group over time. For example, the skilled person can set the reference vector ref2 according to the second processing target group by, for example, performing a selection input on the input unit 38 to select the representative point 301b of the second feature group according to the second processing target group, and then performing a selection input to select the representative point 301a of the first feature group according to the first processing target group. In this case, as shown in FIG. 39 , the direction from the representative point 301b of the second feature group toward the representative point 301a of the first feature group becomes the reference vector ref2 according to the second processing target group.

[0279] Furthermore, by checking the target image 150 showing the target belonging to the first group and the representative image 310a of the first group on the display screen of the display unit 37 as described above, the skilled person can estimate the direction in which the feature values ​​of the target belonging to the first group move over time. The skilled person can then set the reference vector ref1 according to the first group by, for example, inputting a selection to the input unit 38 to select the representative point 301a of the first feature group according to the first group, and then inputting a selection to select a position 309 within the predetermined range 302a of the first feature group. In this case, as shown in FIG. 39 , the direction from the representative point 301a of the first feature group toward the position 309 becomes the reference vector ref1 according to the first group.

[0280] 40 is a schematic diagram showing an example of a position 612b of a new feature and a position 611b of the previous feature in a first target image containing a first target to be processed in feature space, and an example of a position 622b of a new feature and a position 621b of the previous feature in a second target image containing a second target to be processed in feature space. In the example of FIG. 40, the first target to be processed and the second target to be processed are included in a second target to be processed group.

[0281] In the example of Figure 40, for the first target object to be treated, the degree of similarity between a feature amount change vector 611v directing from a position 611b of the previous feature amount to a position 612b of the new feature amount and the reference vector ref2 according to the second target object to be treated is low. Therefore, the attention need determination unit 145 determines that the first target object to be treated requires attention. On the other hand, for the second target object to be treated, the degree of similarity between a feature amount change vector 621v directing from a position 621b of the previous feature amount to a position 622b of the new feature amount and the reference vector ref2 according to the second target object to be treated is high. Therefore, the attention need determination unit 145 does not determine that the second target object to be treated requires attention.

[0282] <Fourth Judgment Method in Monitoring Processing> In this example, the caution need determination unit 145 determines that a treatment target appearing in a new target image having a feature that is equidistant from the representative points 301 of two feature groups in the feature space is caution needing. Specifically, for example, if the new target image has a feature that is equidistant from the representative points 301 of the two feature groups in the feature space and the treatment target appearing in the new target image belongs to one of two treatment target groups corresponding to the two feature groups, respectively, the caution need determination unit 145 determines that a treatment target appearing in the new target image is caution needing. In other words, if the feature of the new target image is equidistant from the representative point 301 of one feature group corresponding to the treatment target group to which the treatment target appearing in the new target image belongs and the representative point 301 of a feature group other than the one feature group, the caution need determination unit 145 determines that a treatment target appearing in the new target image is caution needing. The attention required determination unit 145 can specify, from the second necessary information, the representative point 301 of one feature amount group corresponding to the treatment target group to which the treatment target appearing in the new target image belongs. In the second necessary information, the representative point 301 of the feature amount group to which the feature amount associated with the treatment target ID of the treatment target appearing in the new target image belongs becomes the representative point 301 of one feature amount group corresponding to the treatment target group to which the treatment target appearing in the new target image belongs.

[0283] Here, when the feature of a new target image is equidistant from the representative points 301 of the two feature groups in the feature space, it can be said that the degree of similarity of the new target image to the representative images of the two treatment target groups is the same. Therefore, it can be said that the caution need determination unit 145 determines whether the treatment target appearing in the new target image requires caution based on the degree of similarity of the new target image to the representative images of the two treatment target groups. When the degree of similarity of the new target image to the representative images of the two treatment target groups is the same, the caution need determination unit 145 determines that the treatment target appearing in the new target image requires caution.

[0284] In this manner, the caution-needed determination unit 145 determines whether or not the treatment target shown in each of the multiple new target images acquired in step s1 needs caution.

[0285] The attention need determination unit 145 determines a treatment target determined to require attention by the first determination method as a first type of attention needing treatment target. Moreover, the attention need determination unit 145 determines a treatment target determined to require attention by the second determination method as a second type of attention needing treatment target. Moreover, the attention need determination unit 145 determines a treatment target determined to require attention by the third determination method as a third type of attention needing treatment target. And the attention need determination unit 145 determines a treatment target determined to require attention by the fourth determination method as a fourth type of attention needing treatment target. However, the attention need determination unit 145 determines a treatment target determined to require attention by the first determination method and also determined to require attention by another determination method as a first type of attention needing treatment target. Moreover, the attention need determination unit 145 determines a treatment target determined to require attention by the second determination method and also determined to require attention by at least one of the third determination method and the fourth determination method as a second type of attention needing treatment target. Then, the caution-needing determination unit 145 determines that a treatment target that is determined to be caution-needing by the third determination method and that is determined to be caution-needing by the fourth determination method as a third-type caution treatment target.

[0286] In the server 3, when the input unit 38 receives from the user a treatment content for a target requiring attention, the treatment information generation unit 134 updates the treatment information so that the treatment information indicates the treatment content for the target requiring attention as the treatment content for the target requiring attention. Furthermore, the treatment information generation unit 134 includes type information indicating the type of the target requiring attention in the second treatment object information for the target requiring attention, which is included in the treatment information. For example, when the input unit 38 receives from the user a treatment content for a target requiring attention of a first type, the treatment information generation unit 134 includes type information indicating the first type in the second treatment object information for the first type target requiring attention, which is included in the treatment information. This makes it possible to identify what type of target requiring attention is the target requiring attention from the second treatment object information for the target requiring attention.

[0287] In the above example, the reference vector is set by the user, but it may also be set by the server 3. In other words, the reference vector may be automatically set by the server 3. An example of the operation of the server 3 in this case will be described below.

[0288] 41, the control unit 30 of the server 3 includes a reference vector setting unit 136. The reference vector setting unit 136 is a functional block formed when the CPU of the control unit 30 of the server 3 executes the program 31a in the storage unit 31.

[0289] The reference vector setting unit 136 sets a reference vector used in the attention monitoring process in the information collection process at the attention growth stage (i.e., the attention information collection process). In the attention information collection process, the reference vector setting unit 136 arranges, in the feature space, each representative point 301 and each predetermined range 302 finally obtained in the attention information collection process. Here, it is assumed that a representative point 301a and a predetermined range 302a of the first feature group, a representative point 301b and a predetermined range 302b of the second feature group, and a representative point 301c and a predetermined range 302c of the third feature group are arranged in the feature space.

[0290] Next, the reference vector setting unit 136 identifies, for each treatment target, a trajectory of time change over one year of the feature position in the feature space (also referred to as a feature trajectory) based on the feature values ​​for the past year for each treatment target stored in the storage unit 31. For example, the reference vector setting unit 136 may use all feature values ​​acquired for a treatment target in the target farm by the feature acquisition unit 140 in the year prior to the year in which the target monitoring process is executed as feature values ​​for the past year for the treatment target. The feature values ​​for the past year for each treatment target are stored in the storage unit 31.

[0291] Next, the reference vector setting unit 136 sets a reference vector based on the feature trajectory for each processing target. For example, the reference vector setting unit 136 identifies the order in which the most feature trajectories pass through the predetermined range 302a, the predetermined range 302b, and the predetermined range 302c in the feature space. Then, the reference vector setting unit 136 sets a reference vector based on the order in which the most feature trajectories pass through.

[0292] For example, if the number of feature trajectories passing through predetermined range 302c, predetermined range 302b, and predetermined range 302a in this order is greatest, the reference vector setting unit 136 sets a vector extending from representative point 301c corresponding to predetermined range 302c to representative point 301b corresponding to predetermined range 302b as the reference vector for the third processing target group. The reference vector setting unit 136 also sets a vector extending from representative point 301b corresponding to predetermined range 302b to representative point 301a corresponding to predetermined range 302a as the reference vector for the second processing target group. The reference vector for the first processing target group may be set by the user as described above, or may be set by the reference vector setting unit 136 based on the feature trajectories for each processing target.

[0293] In the monitoring process, the first grouping unit 100 may determine whether to perform treatment target grouping based on the number of treatment targets determined to require caution by the caution needing determination unit 145. In this case, in step s54, the terminal device 4 transmits to the server 3 the number of treatment targets determined to require caution in step s52 (also referred to as the number of treatment targets requiring caution) together with the first treatment target information for each treatment target.

[0294] In the server 3, before step s61, the second grouping unit 130 determines whether to perform feature grouping based on the number of treatment-requiring targets. Since treatment target groups are determined by feature grouping, determining whether to perform feature grouping based on the number of treatment-requiring targets can also be said to be determining whether to perform treatment target grouping based on the number of treatment-requiring targets.

[0295] When the number of objects requiring careful treatment is equal to or greater than the threshold value, that is, when the number of objects requiring careful treatment is large, the second grouping unit 130 determines to perform feature grouping. On the other hand, when the number of objects requiring careful treatment is less than the threshold value, that is, when the number of objects requiring careful treatment is small, the second grouping unit 130 does not determine to perform feature grouping. When it is not determined to perform feature grouping, steps s61 to s65 are executed as described above.

[0296] When the second grouping unit 130 determines to perform feature grouping, it performs grouping of the multiple new feature amounts included in each of the multiple pieces of first processing target information from the terminal device 4. Thereafter, the above-mentioned steps s12 to s15 are executed in the same manner as in the information collection process. The newly generated processing information in step s15 is transmitted to the terminal device 4. The terminal device 4 stores the received processing information in the storage unit 41 and displays it.

[0297] When treatment target grouping is performed in the attention monitoring process, the result of the treatment target grouping performed in the attention monitoring process is used in the monitoring process next to the attention monitoring process instead of the result of the treatment target grouping performed in the attention information collection process. For example, the second necessary information used in the monitoring process next to the attention monitoring process includes the result of the feature amount grouping performed in the attention monitoring process. Furthermore, the multiple treatment target groups in the monitoring process next to the attention monitoring process are the same as the multiple treatment target groups obtained as a result of the treatment target grouping performed in the attention monitoring process.

[0298] In the monitoring process, the server 3 may obtain an evaluation value of the effect of the treatment content on the treatment target, that is, an effect evaluation value. An example of the operation of the server 3 in this case will be described below.

[0299] 42, the control unit 30 of the server 3 includes an evaluation value acquisition unit 137 that calculates an effect evaluation value for the treatment content for each treatment target. The evaluation value acquisition unit 137 is a functional block formed when the CPU of the control unit 30 executes the program 31a in the storage unit 31.

[0300] The evaluation value acquisition unit 137 arranges, in the feature space, multiple new features from the terminal device 4 and multiple previous feature values ​​to be included in the second necessary information. Next, the evaluation value acquisition unit 137 calculates the distance between the position of the new feature value and the position of the previous feature value for the target object of interest. This distance can also be considered as the amount of movement of the position of the feature value for the target object of interest. Furthermore, the evaluation value acquisition unit 137 calculates the similarity between the vector pointing from the position of the previous feature value to the position of the new feature value for the target object of interest, i.e., the feature change vector, and the above-mentioned reference vector corresponding to the target object group to which the target object of interest belongs. This similarity may be, for example, the cosine similarity between the feature change vector and the reference vector.

[0301] Next, the evaluation value acquisition unit 137 calculates an effect evaluation value of the treatment content for the treatment target of interest based on the calculated distance and similarity. The evaluation value acquisition unit 137 may calculate the effect evaluation value using, for example, a predetermined arithmetic expression with the distance and similarity as variables. The effect evaluation value may be set, for example, so that it becomes smaller the farther the distance is from the threshold range and becomes larger the higher the similarity. In this case, the larger the effect evaluation value, the greater the effect of the treatment content. In other words, the larger the effect evaluation value, the more effective the treatment content.

[0302] In this way, the evaluation value acquisition unit 137 obtains an effect evaluation value of the treatment content for each treatment target during the monitoring treatment. However, if a new treatment content different from the previous treatment content is set for a treatment target requiring attention during the attention monitoring process, the evaluation value acquisition unit 137 does not perform processing to obtain an effect evaluation value of the new treatment content for the treatment target requiring attention. On the other hand, if the treatment content for a treatment target requiring attention during the attention monitoring process is not changed, the evaluation value acquisition unit 137 obtains an effect evaluation value of the treatment content for the treatment target requiring attention.

[0303] When the evaluation value acquisition unit 137 obtains the effect evaluation value of the treatment content for the treatment target of interest, the treatment information generation unit 134 generates second treatment target information for the treatment target of interest, including the effect evaluation value. The terminal device 4, which has received the second treatment target information for the treatment target of interest from the server 3, can display a screen 360 including an effect evaluation value 364 of the treatment content for the treatment target of interest, as shown in Figs. 15 and 16 above.

[0304] In addition, in the monitoring process, the caution determination unit 145 may determine whether the treatment target requires caution using at least one of the above-mentioned first determination method, second determination method, third determination method, and part of the fourth determination method.

[0305] <Other examples of monitoring processes> Next, another example of the monitoring process will be described. Hereinafter, the above-mentioned example of the monitoring process will be referred to as the first monitoring process, and the monitoring process described below will be referred to as the second monitoring process. In addition, in the description of the second monitoring process, the term "monitoring process of interest" refers to the second monitoring process of interest, the term "growth stage of interest" refers to the growth stage in which the second monitoring process of interest is executed, and the term "information collection process of interest" refers to the information collection process executed in the growth stage in which the second monitoring process of interest is executed.

[0306] Fig. 43 is a schematic diagram showing an example of a plurality of functional blocks included in a processing system 1 (also referred to as processing system 1b) that performs the second monitoring process. As shown in Fig. 43, the processing system 1b includes, as functional blocks, the above-mentioned first grouping unit 100, feature amount map generation unit 132, evaluation value acquisition unit 137, image processing unit 141, treatment target information generation unit 142, caution required determination unit 145, treatment content determination unit 146, and treatment information update unit 147. The control unit 40 of the terminal device 4 includes, as functional blocks, the treatment content determination unit 146 and treatment information update unit 147.

[0307] Fig. 44 is a schematic diagram showing an example of the second monitoring process. As shown in Fig. 44, in the second monitoring process, similarly to the first monitoring process, the above-mentioned steps s51, s1, s2, and s52 are executed in this order.

[0308] In this example, the second necessary information in the attention monitoring process includes not only information on the latest N feature groups in the attention growth stage, but also the latest treatment information in the attention growth stage. For example, consider a case where the attention monitoring process is the first monitoring process executed after the attention information collection process in the attention growth stage. In this case, the treatment information finally obtained in the attention information collection process is included in the second necessary information in the attention monitoring process. Also consider a case where the attention monitoring process is the second or subsequent monitoring process executed in the attention growth stage after the attention information collection process. In this case, the treatment information finally obtained in the monitoring process executed immediately before the attention monitoring process is included in the second necessary information in the attention monitoring process. The latest treatment information included in the second necessary information includes an effect evaluation value of the treatment content for each treatment target.

[0309] After step s52, in step s56, the treatment content determination unit 146 determines treatment content for the treatment target determined to be in need of attention by the attention need determination unit 145, i.e., the treatment content for the treatment target requiring attention. For example, the treatment content determination unit 146 may adopt, as the treatment content for the treatment target requiring attention, the treatment content performed on the treatment target appearing in a target image having a feature amount similar to the feature amount of a new target image in which the treatment target requiring attention appears.

[0310] When determining a treatment content for a first type of caution target requiring treatment, the treatment content determination unit 146 extracts, from a plurality of features stored in the storage unit 41, a feature of the first type of caution target requiring treatment (also referred to as a first type similar feature) that is similar to a feature of a new target image in which the first type of caution target requiring treatment appears. For example, if the cosine similarity between a feature vector as a feature of the new target image in which the first type of caution target requiring treatment appears and a feature vector as a feature stored in the storage unit 41 is equal to or less than a threshold, the treatment content determination unit 146 determines that the feature stored in the storage unit 41 is similar to the feature of the new target image in which the first type of caution target requiring treatment appears. Then, the treatment content determination unit 146 adopts the treatment content performed on the treatment target appearing in the target image having the extracted first type similar feature as a treatment content for the first type of caution target requiring treatment.

[0311] As can be understood from the above description, the treatment content identified from the second treatment target information including the first type of similar feature and stored in the storage unit 41 can be considered to be the treatment content applied to the first type of caution treatment target that appears in the target image having the first type of similar feature. The treatment content determination unit 146, for example, adopts the treatment content identified from the second treatment target information including the first type of similar feature and stored in the storage unit 41 as the caution treatment content for the first type of caution treatment target. The treatment content identified from the second treatment target information including the first type of similar feature and stored in the storage unit 41 can be considered to be the treatment content according to the growth state of the crop that appears in the target image having the first type of similar feature.

[0312] When the plurality of feature quantities stored in the storage unit 41 includes a plurality of similar feature quantities of the first type, the treatment content determination unit 146 may determine a treatment content for the first type of caution treatment target based on the effect evaluation value. For example, the treatment content determination unit 146 acquires, from the storage unit 41, effect evaluation values ​​of a plurality of treatment contents (also referred to as a plurality of specific treatment contents) performed on a plurality of treatment targets appearing in a plurality of target images each having a plurality of similar feature quantities of the first type. The treatment content determination unit 146 sets the effect evaluation value included in the second treatment target information including the similar feature quantities of the first type stored in the storage unit 41 as the effect evaluation value of the treatment contents performed on the treatment targets appearing in the target images having the similar feature quantities of the first type. Then, the treatment content determination unit 146 adopts the specific treatment content with the largest effect evaluation value among the acquired plurality of effect evaluation values ​​as the treatment content for the first type of caution treatment target. The specific treatment content with the largest effect evaluation value among the plurality of effect evaluation values ​​can be said to be the treatment content with the greatest effect among the plurality of specific treatment contents.

[0313] Similarly, the treatment content determination unit 146 determines treatment content for the second type of target requiring attention treatment, treatment content for the third type of target requiring attention treatment, and treatment content for the fourth type of target requiring attention treatment.

[0314] After step s56, in step s57, the treatment information update unit 147 updates the treatment information included in the second necessary information. For each caution treatment target, the treatment information update unit 147 updates the treatment information so that the treatment information indicates the caution treatment target treatment content determined in step s56 as the treatment content for the caution treatment target.

[0315] After step s57, in step s58, the treatment target information generation unit 142 generates first treatment target information for each treatment target. In this example, regardless of whether the treatment target is a treatment target requiring attention or not, the first treatment target information for the treatment target does not include a new target image in which the treatment target appears. The first treatment target information for each treatment target includes feature amounts of a new target image in which the treatment target appears, treatment target position information for the treatment target, and a treatment target ID for the treatment target.

[0316] After step s58, in step s59, the terminal device 4 transmits to the server 3 the treatment information updated in step s57 and the plurality of pieces of first treatment target information generated in step s58.

[0317] After step s59, the server 3 executes steps s61 and s62 described above. The server 3 also stores the updated processing information from the terminal device 4 in the storage unit 31 as the latest treatment information. In step s68, the evaluation value acquisition unit 137 calculates an effect evaluation value for the treatment content for each treatment target, as described above. The effect evaluation value calculated by the evaluation value acquisition unit 137 is included in the treatment information transmitted to the terminal device 4, as described above.

[0318] In this way, in the second monitoring process, regardless of whether the attention need determination unit 145 determines that the treatment target shown in the new target image requires attention, the terminal device 4 does not transmit the new target image to the server 3. This makes it possible to reduce the amount of communication between the terminal device 4 and the server 3.

[0319] Note that some of the processes executed by the terminal device 4 may be executed by the server 3, and some of the processing functions executed by the server 3 may be executed by the terminal device 4. For example, the server 3 may not include the second grouping unit 130, and the terminal device 4 may include the second grouping unit 130. In this case, the terminal device 4 may include the first grouping unit 100 that performs grouping of treatment targets. Also, the terminal device 4 may not include the feature amount acquisition unit 140, and the server 3 may include the feature amount acquisition unit 140. In this case, the server 3 may include the first grouping unit 100 that performs grouping of treatment targets.

[0320] Furthermore, all of the processes executed by the server 3 may be executed by the terminal device 4. That is, the processing system 1 may be configured, for example, only by the terminal device 4. In this case, multiple functional blocks such as the second grouping unit 130 and the feature amount map generation unit 132 included in the control unit 30 of the server 3 are provided in the control unit 40 of the terminal device 4.

[0321] The functions of the elements disclosed herein may be implemented using circuitry or processing circuitry, including general-purpose processors, special-purpose processors, integrated circuits, ASICs ("application-specific integrated circuits"), conventional circuitry, and / or combinations thereof, configured to perform the disclosed elements or programmed to perform the disclosed functions. A processor is considered to be processing circuitry or circuitry when it includes transistors and other circuitry therein. In this disclosure, a circuitry, unit, or means is hardware that performs the recited function or hardware programmed to perform the function. The hardware may be any hardware disclosed herein or other known hardware that is programmed to perform or configured to perform the recited function. When the hardware is a processor, which may be considered as a type of circuitry, the circuitry, means, or unit is a combination of hardware and software, software used to configure the hardware, and / or processor.

[0322] Although the processing system has been described in detail above, the above description is illustrative in all respects and does not limit the present disclosure. Furthermore, the various examples described above can be combined and applied as long as they are not mutually inconsistent. It is understood that countless variations not illustrated can be envisioned without departing from the scope of the present disclosure.

[0323] The present disclosure includes the following aspects.

[0324] The processing system according to the first aspect includes a target image acquisition unit that acquires a plurality of first target images each depicting a plurality of treatment targets, a first grouping unit that performs treatment target grouping by dividing the plurality of treatment targets into a plurality of treatment target groups based on the plurality of first target images, and a representative image acquisition unit that acquires a plurality of representative images that respectively represent the plurality of treatment target groups.

[0325] The processing system according to the second aspect is the processing system according to the first aspect, and includes a first display unit that displays the plurality of representative images, an input unit that receives from a user, for each of the plurality of treatment target groups, treatment content common to the plurality of treatment targets that make up one treatment target group, and a treatment information generation unit that generates first treatment information indicating the plurality of types of treatment content corresponding to each of the plurality of treatment target groups received by the input unit.

[0326] A processing system according to a third aspect is a processing system according to the second aspect, wherein the input unit has a voice input unit that accepts natural language spoken by a user and expresses a treatment content, and a treatment content identification unit that identifies the treatment content expressed by the natural language accepted by the voice input unit.

[0327] A processing system according to a fourth aspect is a processing system according to the second or third aspect, wherein the treatment information generation unit generates second treatment information that indicates treatment content in more detail than the first treatment information.

[0328] A processing system according to a fifth aspect is a processing system according to any one of the second to fourth aspects, wherein the first processing information includes a processing target map showing the distribution of the plurality of processing targets, and in the processing target map, the position of each processing target is shown in a different manner depending on the processing content, and the processing system is provided with a second display unit that displays the processing target map.

[0329] A processing system according to a sixth aspect is a processing system according to the fifth aspect, in which when the input unit receives a selection input from a user selecting the position of a first processing object on the processing object map, the second display unit displays the processing content for the first processing object.

[0330] A processing system according to a seventh aspect is a processing system according to the sixth aspect, in which, when the input unit receives a selection input from a user selecting the position of the first treatment target on the treatment target map, the second display unit displays the treatment content for the first treatment target and an evaluation value of the effectiveness of the treatment content for the first treatment target.

[0331] A processing system according to an eighth aspect is a processing system according to any one of the first to seventh aspects, wherein the first grouping unit has a feature acquisition unit that acquires features of the plurality of first target images, and a second grouping unit that performs the processing target grouping based on the features of the plurality of first target images.

[0332] A processing system according to a ninth aspect is a processing system according to the eighth aspect, wherein the second grouping unit performs feature grouping to divide the features of the plurality of first target images into a plurality of feature groups, and includes a plurality of processing targets that appear in a plurality of first target images, each of which has a plurality of features that constitute one feature group, in one processing target group.

[0333] A processing system according to a tenth aspect is a processing system according to the ninth aspect, wherein the second grouping unit reduces the number of dimensions of the features of the first target images, and then divides the features of the first target images into the feature groups.

[0334] A processing system according to an eleventh aspect is a processing system according to the ninth or tenth aspect, wherein the second grouping unit sets the parameters required for the feature grouping based on the results of grouping the features of a plurality of images acquired before the plurality of first target images.

[0335] A processing system according to a twelfth aspect is a processing system according to the ninth or tenth aspect, wherein the second grouping unit has, for each of the plurality of feature groups, a reference feature that serves as a basis for that feature group, and sets parameters necessary for the feature grouping based on the reference feature.

[0336] A processing system according to a thirteenth aspect is a processing system according to any one of the ninth to twelfth aspects, wherein the representative image acquisition unit sets a first target image corresponding to a representative point of a first feature group included in the plurality of feature groups in the feature space of the features as a representative image of a first processing target group corresponding to the first feature group.

[0337] A processing system according to a fourteenth aspect is a processing system according to any one of the ninth to thirteenth aspects, and includes a feature map generation unit that generates a feature map representing the distribution of the features of the plurality of first target images in a feature space of the features, and a display unit that displays the feature map.

[0338] A processing system according to a fifteenth aspect is a processing system according to the fourteenth aspect, wherein the feature map represents a distribution of the features in the feature space with a reduced number of dimensions.

[0339] A processing system according to a 16th aspect is a processing system according to the 14th or 15th aspect, wherein the feature map indicates a representative point in the feature space of a first feature group included in the plurality of feature groups.

[0340] A processing system according to a seventeenth aspect is the processing system according to the sixteenth aspect, wherein the feature map indicates a predetermined range from the representative point.

[0341] A processing system according to an 18th aspect is a processing system according to any one of the 9th to 17th aspects, and includes a display unit that displays a first target image having a first feature that is equidistant from representative points of two feature groups in the feature space of the feature.

[0342] A processing system according to a 19th aspect is a processing system according to the 18th aspect, further comprising a feature map generation unit that generates a feature map representing the distribution of the features of the plurality of first target images in a feature space of the features, and the first display unit displays the feature map, and in the feature map, the positions of the features of the plurality of first target images are shown in a different manner for each feature group, and a locus of points equidistant from the representative points of the two feature groups is shown.

[0343] A processing system according to a twentieth aspect is a processing system according to any one of the ninth to nineteenth aspects, and includes a display unit that displays a first target image having a first feature that is located outside a predetermined range from a representative point of a first feature group among a plurality of first features that constitute a first feature group included in the plurality of feature groups in a feature space of the features.

[0344] A processing system according to a 21st aspect is a processing system according to the 20th aspect, and includes a feature map generation unit that generates a feature map representing the distribution of the features of the plurality of target images in a feature space of the features, and the feature map indicates the representative point and the specified range.

[0345] The processing system of the 22nd aspect is a processing system of any one of the 1st to 21st aspects, and is equipped with an attention determination unit that determines whether a second treatment target depicted in a second target image includes a second target image included in the plurality of first target images and requires attention, based on the second target image.

[0346] The processing system of the 23rd aspect is a processing system of the 22nd aspect, in which the attention-requiring determination unit determines whether the second treatment object requires attention based on the degree of similarity between the second target image and a representative image of the treatment object group to which the second treatment object belongs.

[0347] A processing system according to a 24th aspect is a processing system according to the 22nd or 23rd aspect, in which the attention-requiring determination unit determines that the second treatment target requires attention based on the degree of similarity of the second target image with representative images of two treatment target groups.

[0348] The processing system of the 25th aspect is a processing system of any one of the second to seventh aspects, and includes an attention-requiring determination unit that determines whether a first treatment target included in the plurality of treatment targets requires attention based on a new first target image acquired by the target image acquisition unit, in which the first treatment target is shown, and when the attention-requiring determination unit determines that the first treatment target requires attention, the first display unit displays the new first target image.

[0349] The processing system of the 26th aspect is a processing system of the 25th aspect, in which, when the input unit receives from a user an instruction to change the processing target group to which the first processing target determined to require attention belongs, the grouping unit changes the processing target group to which the first processing target belongs in accordance with the change instruction.

[0350] The processing system of the 27th aspect is a processing system of the 25th aspect, and is provided with a treatment information update unit that updates the first treatment information so that, when the input unit receives from the user treatment content for the first treatment object that has been determined to require attention, the first treatment information indicates the treatment content for the first treatment object as the treatment content for the first treatment object.

[0351] The processing system of the 28th aspect is a processing system of any one of the second to seventh aspects, and includes an attention determination unit that determines whether a first treatment target included in the plurality of treatment targets requires attention based on a new first target image acquired by the target image acquisition unit, in which the first treatment target is shown; a treatment content determination unit that determines treatment content for the first treatment target that has been determined to require attention; and a treatment information update unit that updates the first treatment information so that the first treatment information indicates the treatment content for the first treatment target as the treatment content for the first treatment target.

[0352] A processing system according to a 29th aspect is a processing system according to the 28th aspect, in which the treatment content determination unit adopts a second treatment content performed on a second treatment target appearing in a second target image having features similar to those of the new first target image as the treatment content for the target requiring attention.

[0353] A processing system according to a 30th aspect is a processing system according to the 29th aspect, wherein the treatment content determination unit has an evaluation value of the effects of multiple second treatment contents performed on multiple second treatment targets that appear in multiple second target images each having features similar to the features of the new first target image, and based on the evaluation value, adopts one of the multiple second treatment contents as the treatment content for the target requiring attention.

[0354] The processing system of the 31st aspect is a processing system of any one of the 27th to 30th aspects, wherein the target image acquisition unit acquires another new first target image that depicts the first treatment target that has been determined to require attention, and when the attention determination unit determines whether the first treatment target requires attention based on the another new first target image and the first treatment target is not determined to require attention, the treatment information update unit updates the first treatment information so that the first treatment information indicates, as the treatment content for the first treatment target, treatment content corresponding to the treatment target group to which the first treatment target belongs, rather than treatment content for the target requiring attention.

[0355] A processing system according to a 32nd aspect is a processing system according to any one of the 25th to 31st aspects, wherein the target image acquisition unit acquires a new plurality of first target images each depicting the plurality of treatment targets, the attention determination unit determines whether each of the plurality of treatment targets requires attention based on the new plurality of first target images, and the first grouping unit decides whether to re-group the treatment targets based on the number of treatment targets determined to require attention among the plurality of treatment targets.

[0356] A processing system according to the 33rd aspect is a processing system according to any one of the 25th to 32nd aspects, in which the attention-requiring determination unit determines whether the first treatment object requires attention based on the degree of change over time in a first object image in which the first treatment object is depicted.

[0357] A processing system according to the 34th aspect is a processing system according to any one of the 25th to 33rd aspects, in which the attention-requiring determination unit determines whether the first treatment object requires attention based on the direction of time change of a first object image in which the first treatment object is depicted.

[0358] A processing system according to the 35th aspect is a processing system according to the 34th aspect, in which the attention-requiring determination unit determines whether the first treatment object requires attention based on the degree of similarity between a vector representing the direction of time change in a first target image in which the first treatment object appears and a reference vector.

[0359] A processing system according to the 36th aspect is a processing system according to any one of the 25th to 35th aspects, in which the attention-requiring determination unit determines whether the first treatment target requires attention based on the degree of similarity between the new first target image and a representative image of the treatment target group to which the first treatment target belongs.

[0360] The processing system according to the 37th aspect is a processing system according to any one of the first to 36th aspects, and comprises a terminal device having the image acquisition unit and a server having the representative image acquisition unit.

[0361] The processing system of the 38th aspect is a processing system of any one of the 8th to 21st aspects, and includes a terminal device having the image acquisition unit and the feature acquisition unit, and a server having the second grouping unit and the representative image acquisition unit, and the terminal device transmits the features of the plurality of first target images and at least one of the plurality of first target images to the server.

[0362] A processing system according to a 39th aspect is a processing system according to the 38th aspect, wherein the terminal device has an attention-needed judgment unit that judges whether a second treatment target appearing in a second target image included in the plurality of first target images requires attention based on the second target image, and transmits the features of the plurality of first target images to the server, and when the attention-needed judgment unit determines that the second treatment target requires attention, transmits the second target image to the server, and when the attention-needed judgment unit does not determine that the second treatment target requires attention, does not transmit the second target image to the server.

[0363] The processing system of the 40th aspect is a processing system of any one of the 25th to 27th aspects, and comprises a terminal device having the image acquisition unit and the attention-requiring judgment unit, and a server having the representative image acquisition unit, the first display unit, the input unit, and the treatment information generation unit, and when the attention-requiring judgment unit judges that the first treatment target requires attention, the terminal device transmits the new first target image to the server, and when the attention-requiring judgment unit does not judge that the first treatment target requires attention, the terminal device does not transmit the new first target image to the server.

[0364] The processing system of the 41st aspect is a processing system of any one of the 28th to 30th aspects, and comprises a terminal device having the image acquisition unit, the attention-requiring determination unit, and the treatment information update unit, and a server having the representative image acquisition unit, the first display unit, the input unit, and the treatment information generation unit, and the terminal device does not send the new first target image to the server regardless of whether the attention-requiring determination unit determines that the first treatment target requires attention.

[0365] A server according to the 42nd aspect is a server included in the processing system according to any one of the 37th to 41st aspects.

[0366] A terminal device according to a 42nd aspect is a terminal device included in the processing system according to any one of the 37th to 41st aspects. [Explanation of symbols]

[0367] 1 Processing System 3 Server 4 Terminal Devices 37, 47 Display section 100 First Grouping Section 130 Second Grouping Section 131 Representative image acquisition unit 133 Treatment Content Identification Unit 134 Treatment information generation unit (treatment information update unit) 135, 145 Caution judgment part 140 Feature acquisition unit 141 Image acquisition unit 146 Treatment Content Decision Unit 147 Treatment Information Update Unit 150 target images 300, 400 feature maps 301, 301a, 301b, 301c Representative points 305, 305a, 305b, 305c specified range 308 Trajectory 308a 1st trajectory 308b 2nd locus 308c 3rd locus 310, 310a, 310b, 310c Representative images 350 Treatment Map 611v, 621v Feature change vector ref1, ref2, ref3 reference vectors

Claims

1. a target image acquisition unit that acquires a plurality of first target images each showing a plurality of treatment targets; a first grouping unit that performs treatment object grouping by dividing the plurality of treatment objects into a plurality of treatment object groups based on the plurality of first object images; a representative image acquisition unit that acquires a plurality of representative images that respectively represent the plurality of treatment target groups; Provided is a processing system.

2. 10. The processing system of claim 1, a first display unit that displays the plurality of representative images; an input unit that receives, from a user, a treatment content common to a plurality of treatment targets constituting one treatment target group for each of the plurality of treatment target groups; a treatment information generating unit that generates first treatment information that indicates a plurality of types of treatment contents corresponding to the plurality of treatment target groups, which are received by the input unit; A processing system comprising:

3. 3. The processing system of claim 2, the input unit has a voice input unit that accepts natural language uttered by a user and expressing the content of the treatment, A processing system comprising: a procedure content specifying unit that specifies procedure content expressed by the natural language received by the voice input unit.

4. 3. The processing system of claim 2, The treatment information generation unit generates second treatment information that indicates treatment details in more detail than the first treatment information.

5. 3. The processing system of claim 2, the first treatment information includes a treatment target map representing a distribution of the plurality of treatment targets; In the treatment object map, the position of each treatment object is displayed in a different manner depending on the treatment content, A processing system comprising a second display unit that displays the treatment target map.

6. 6. The processing system of claim 5, When the input unit receives a selection input from a user selecting a position of a first treatment object on the treatment object map, the second display unit displays treatment details for the first treatment object.

7. 7. The processing system of claim 6, When the input unit receives a selection input from a user selecting the position of the first treatment target on the treatment target map, the second display unit displays the treatment content for the first treatment target and an evaluation value of the effect of the treatment content for the first treatment target.

8. 10. The processing system of claim 1, The first grouping unit a feature amount acquisition unit that acquires feature amounts of the plurality of first target images; a second grouping unit that performs the grouping of the processing target images based on the feature amounts of the plurality of first target images; A processing system comprising:

9. 9. The processing system of claim 8, The second grouping unit performing feature grouping to divide the feature amounts of the plurality of first target images into a plurality of feature amount groups; A processing system that includes, in one processing target group, a plurality of processing targets that appear in a plurality of first target images each having a plurality of feature amounts that form one feature amount group.

10. 10. The processing system of claim 9, The second grouping unit divides the feature amounts of the first target images into the plurality of feature amount groups after reducing the number of dimensions of the feature amounts of the first target images.

11. 10. The processing system of claim 9, The second grouping unit sets parameters necessary for the feature grouping based on results of grouping features of a plurality of images acquired before the plurality of first target images.

12. 10. The processing system of claim 9, The second grouping unit a reference feature serving as a reference for each of the plurality of feature groups; A processing system that sets parameters necessary for the feature grouping based on the reference feature.

13. 10. The processing system of claim 9, The representative image acquisition unit sets a first target image corresponding to a representative point of a first feature group included in the plurality of feature groups in a feature space of the features as a representative image of a first processing target group corresponding to the first feature group.

14. 10. The processing system of claim 9, a feature amount map generating unit that generates a feature amount map representing a distribution of the feature amounts of the plurality of first target images in a feature amount space of the feature amounts; a display unit that displays the feature amount map; A processing system comprising:

15. 15. The processing system of claim 14, The feature map represents a distribution of the features in the feature space of the features with a reduced dimensionality.

16. 15. The processing system of claim 14, The feature map indicates a representative point in the feature space of a first feature group included in the plurality of feature groups.

17. 17. The processing system of claim 16, A processing system, wherein the feature map indicates a predetermined range from the representative point.

18. 10. The processing system of claim 9, a display unit that displays a first target image having a first feature that is equidistant from representative points of two feature groups in a feature space of the feature;

19. 20. The processing system of claim 18, a feature amount map generating unit that generates a feature amount map representing a distribution of the feature amounts of the plurality of first target images in a feature amount space of the feature amounts; the first display unit displays the feature amount map; In the feature map, positions of the feature amounts of the plurality of first target images are displayed in different modes for each feature amount group; A processing system in which a locus of points equidistant from the representative points of the two feature groups is indicated.

20. 10. The processing system of claim 9, a display unit that displays, in a feature space of the features, a first target image having a first feature that is located outside a predetermined range from a representative point of the first feature group, among a plurality of first feature that constitute a first feature group included in the plurality of feature groups.

21. 21. The processing system of claim 20, a feature amount map generation unit that generates a feature amount map representing a distribution of the feature amounts of the plurality of target images in a feature amount space of the feature amounts; The feature map indicates the representative points and the predetermined range.

22. 10. The processing system of claim 1, A processing system comprising an attention-requiring determination unit that determines whether a second treatment target appearing in a second target image requires attention, based on a second target image included in the plurality of first target images.

23. 23. The processing system of claim 22, The attention-requiring determination unit determines whether the second treatment object requires attention based on the degree of similarity between the second treatment object image and a representative image of a treatment object group to which the second treatment object belongs.

24. 23. The processing system of claim 22, The attention-requiring determination unit determines that the second processing target needs attention based on the degree of similarity of the second processing target image with representative images of two processing target groups.

25. 3. The processing system of claim 2, a caution-needed determination unit that determines whether a first treatment target included in the plurality of treatment targets requires attention based on a new first target image acquired by the target image acquisition unit, the first target image including the first treatment target; When the attention-needing determination unit determines that the first treatment target requires attention, the first display unit displays the new first target image.

26. 26. The processing system of claim 25, When the input unit receives an instruction from a user to change the treatment target group to which the first treatment target determined to require attention belongs, the grouping unit changes the treatment target group to which the first treatment target belongs in accordance with the change instruction.

27. 26. The processing system of claim 25, A processing system comprising a treatment information update unit that updates the first treatment information so that, when the input unit receives treatment content for a first treatment object determined to require attention from a user, the first treatment information indicates the treatment content for the first treatment object as the treatment content for the first treatment object.

28. 3. The processing system of claim 2, an attention-needing determination unit that determines whether a first treatment target included in the plurality of treatment targets requires attention based on a new first target image acquired by the target image acquisition unit, the first target image including the first treatment target; a treatment content determination unit that determines treatment content for the first treatment object determined to be in need of attention; a treatment information updating unit that updates the first treatment information so that the first treatment information indicates the treatment content for the target requiring attention as the treatment content for the first treatment target; A processing system comprising:

29. 29. The processing system of claim 28, A processing system in which the treatment content determination unit adopts a second treatment content performed on a second treatment target appearing in a second target image having features similar to those of the new first target image as the treatment content for the target requiring attention.

30. 30. The processing system of claim 29, The treatment content determination unit an evaluation value of the effect of a plurality of second treatment contents performed on a plurality of second treatment targets appearing in a plurality of second target images each having a feature amount similar to the feature amount of the new first target image; A processing system that adopts one of the plurality of second action contents as the action content for the target requiring attention based on the evaluation value.

31. 28. The processing system of claim 27, the target image acquisition unit acquires another new first target image in which the first treatment target determined to require attention is captured; A processing system in which, when the attention determination unit determines whether the first treatment target requires attention based on the other new first target image and the result is that the first treatment target is not determined to require attention, the treatment information update unit updates the first treatment information so that the first treatment information indicates, as the treatment content for the first treatment target, treatment content corresponding to the treatment target group to which the first treatment target belongs, rather than treatment content for the target requiring attention.

32. 26. The processing system of claim 25, the target image acquisition unit acquires a plurality of new first target images in which the plurality of treatment targets are respectively captured; the attention need determination unit determines whether each of the plurality of treatment targets requires attention based on the new plurality of first target images; The first grouping unit determines whether to regroup the treatment targets based on the number of treatment targets determined to require attention among the plurality of treatment targets.

33. 26. The processing system of claim 25, The attention-needing determination unit determines whether the first treatment object needs attention based on the degree of time change in a first object image in which the first treatment object appears.

34. 26. The processing system of claim 25, The attention-needing determination unit determines whether the first treatment object needs attention based on a direction of time change of a first object image in which the first treatment object appears.

35. 35. The processing system of claim 34, The attention-requiring determination unit determines whether the first treatment object requires attention based on the degree of similarity between a vector representing the direction of time change of a first target image in which the first treatment object appears and a reference vector.

36. 26. The processing system of claim 25, A processing system in which the attention-requiring determination unit determines whether the first treatment object requires attention based on the degree of similarity between the new first treatment object image and a representative image of the treatment object group to which the first treatment object belongs.

37. 37. A processing system according to any one of claims 1 to 36, comprising: a terminal device having the image acquisition unit; a server having the representative image acquisition unit; A processing system comprising:

38. 22. A processing system according to any one of claims 8 to 21, a terminal device having the image acquisition unit and the feature amount acquisition unit; a server having the second grouping unit and the representative image acquisition unit; Equipped with The terminal device transmits the feature amounts of the plurality of first target images and at least one of the plurality of first target images to the server.

39. 39. The processing system of claim 38, The terminal device a caution-needing determination unit that determines whether a second treatment target shown in a second target image needs attention, based on a second target image included in the plurality of first target images; transmitting the feature amounts of the plurality of first target images to the server; When the attention need determination unit determines that the second treatment target needs attention, the second target image is transmitted to the server; When the attention-needing determination unit does not determine that the second processing target requires attention, the processing system does not transmit the second target image to the server.

40. 28. A processing system according to any one of claims 25 to 27, comprising: a terminal device having the image acquisition unit and the caution determination unit; a server including the representative image acquisition unit, the first display unit, the input unit, and the treatment information generation unit; Equipped with The terminal device When the attention need determination unit determines that the first treatment target needs attention, the new first target image is transmitted to the server; When the attention-needing determination unit does not determine that the first processing target requires attention, the new first target image is not transmitted to the server.

41. 31. A processing system according to any one of claims 28 to 30, comprising: a terminal device having the image acquisition unit, the attention requirement determination unit, and the treatment information update unit; a server including the representative image acquisition unit, the first display unit, the input unit, and the treatment information generation unit; Equipped with The terminal device does not transmit the new first target image to the server regardless of whether the attention-needing determination unit determines that the first processing target requires attention.

42. 38. A server comprising the processing system of claim 37.

43. A terminal device provided in the processing system according to claim 37.

Images