Information processing device, information processing method, and program

Abstract

To generate a work schedule on the basis of a difference of a plant state in a farm field and level a work load in each working day.SOLUTION: An information processing device comprises: an information acquisition unit which acquires an image captured by a camera mounted on a weeding robot for each of one or more work areas included in a work section of a farm field according to a work time of the weeding robot; a weed quantity determination unit which determines a weed quantity for each of the one or more work areas on the basis of the image and determines a weed quantity for each work section by aggregating the determined weed quantity for each of the one or more work areas; a weeding work intensity prediction unit which predicts a future value of the weeding work intensity being an index value indicating a level of a load of the weeding work for each work section on the basis of a work history including the past work time and weed quantity; and a weeding work planning unit which plans the weeding work by the weeding robot in the farm field on the basis of the weeding work intensity.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, there are known techniques for scheduling weeding work to be performed by an autonomous weeding robot in a field. For example, Patent Document 1 discloses a technique for analyzing the sunlight conditions of a work area in a field and generating a work schedule for each sub-area included in the work area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 168565 Summary of the Invention

[0004] The technology described in Patent Document 1 determines the amount of sunlight and the duration of sunlight for each work area based on images taken by a camera mounted on a lawnmower, and uses this information to create a work schedule. However, with this conventional technology, it was not possible to create a work schedule that took into account differences in the condition of plants in the field, making it difficult to level out the work load on each work day. [Problem to be solved by the invention]

[0005] The present invention has been made in consideration of these circumstances, and one of its objectives is to provide an information processing device, an information processing method, and a program that can generate a work schedule taking into account differences in plant conditions in a field and level out the work load on each work day. [Means for solving the problem]

[0006] An information processing device that is one aspect of the present invention comprises an information acquisition unit that acquires images taken by a camera mounted on a weeding robot for each of one or more work areas included in a work section of a field, along with the work time of the weeding robot; a weed amount determination unit that determines the amount of weeds for each of the one or more work areas based on the images and determines the amount of weeds for each of the work sections by aggregating the determined weed amounts for each of the one or more work areas; a weeding intensity prediction unit that predicts a future value of weeding intensity, which is an index value that indicates the level of the burden of weeding work, for each of the work sections based on a work history including the past work time and the weed amount; and a weeding work planning unit that plans weeding work by the weeding robot in the field based on the weeding intensity.

[0007] The weeding work planning unit may plan the weeding work by aggregating multiple work sections into one or more packets, which are unit areas in which the weeding work can be performed within a unit time, and arranging the aggregated one or more packets so that the weeding work is performed on a predetermined appropriate work date.

[0008] The future value of the weeding intensity may include the future value of the weed amount, and the weeding work planning unit may aggregate work sections with large future values ​​of the weed amount into a packet in which the weeding work is carried out earlier.

[0009] The information acquisition unit may further accept input of external information by the user, the external information including at least one of information regarding personnel on the appropriate work day and information regarding weather on the appropriate work day, and the weeding work planning unit may arrange the one or more packets based on the accepted setting information.

[0010] The information acquisition unit may further accept setting information input by a user, the setting information including at least one of the unit time of the packet and the battery capacity of the weeding robot, and the weeding work planning unit may determine the size of the one or more packets based on the accepted setting information.

[0011] The weed quantity determination unit may determine the amount of weeds for each of the one or more work areas using a trained model that has been trained to output a discrete value indicating the amount of weeds in the image when an image is input. [Effects of the Invention]

[0012] According to the present invention, a work schedule can be generated taking into account differences in plant conditions in a field, and the work load on each work day can be leveled. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of a usage environment and configuration of an information processing device 100. FIG. [Figure 2] 10 is a diagram showing an example of a combination of image information, position information, and work time acquired by the information acquisition unit 110. FIG. [Figure 3] FIG. 10 is a diagram showing an example of a method for determining the amount of weeds by a weed amount determining section 120. [Figure 4] 10 is a diagram showing the amount of weeds for each work section calculated by the weed amount determination unit 120. FIG. [Figure 5] FIG. 10 is a diagram for explaining a method in which the weeding work planning unit 140 aggregates work sections into packets. [Figure 6] FIG. 10 is a diagram showing an example in which the weeding work planning unit 140 arranges packets on the appropriate work date. [Figure 7] FIG. 10 is a diagram showing another example in which the weeding work planning unit 140 arranges packets on the appropriate work date. [Figure 8] FIG. 10 is a diagram showing an example of a screen for allowing the user of the terminal device 30 to set packets. [Figure 9] 10 is a flowchart showing an example of the flow of processing executed by the information processing device 100. DETAILED DESCRIPTION OF THE INVENTION

[0014] [First embodiment] An information processing device 100 according to an embodiment of the present invention will be described below with reference to the drawings. Fig. 1 is a diagram showing an example of the usage environment and configuration of the information processing device 100 according to the embodiment. The information processing device 100 operates in cooperation with, for example, a weeding robot 20 equipped with a camera 10 and a terminal device 30.

[0015] The camera 10 is a digital camera that uses a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) and has a GPS (Global Positioning System) function. That is, when capturing an image, the camera 10 uses the GPS function to also acquire location information (longitude and latitude information) of the location where the image was captured. The camera 10 is installed in a position that allows it to capture images of the plane of the field in the vertical direction, where the weeding robot 20 is traveling. FIG. 1 shows, as an example, a configuration in which a support rod is installed on the weeding robot 20 and the camera 10 is supported by the support rod, allowing it to capture images of the plane of the field in the vertical direction.

[0016] The weeding robot 20 autonomously travels through a field and performs weeding work in accordance with a weeding work plan described below. The weeding robot 20 performs weeding work, for example, by rotating a blade attached to the bottom of the vehicle body to scrape off soil on the surface of the field. The mechanism for performing weeding work may be the same as that of a conventional weed cutter. The weeding robot 20 further includes a wireless communication device (not shown) that wirelessly transmits images captured by the camera 10 to the information processing device 100. In the following explanation, as an example, the weeding robot 20 will be described as traveling through a field where organic vegetables such as spinach are grown by seeding. When organic vegetables are grown by seeding, the period in which weeding can be performed in the field is limited to approximately two to four weeks after sowing, which increases the need to create an effective weeding work plan.

[0017] The terminal device 30 is a computer device such as a personal computer, a smartphone, a tablet terminal, etc. The terminal device 30 communicates with the information processing device 100 via the network NW and inputs external information and setting information for generating a weeding work plan, which will be described later.

[0018] The information processing device 100 is a server device such as a web server. The information processing device 100 includes, for example, an information acquisition unit 110, a weed amount determination unit 120, a weeding intensity prediction unit 130, a weeding work planning unit 140, and a memory unit 150. Each of the information acquisition unit 110, the weed amount determination unit 120, the weeding intensity prediction unit 130, and the weeding work planning unit 140 is realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as a hard disk drive (HDD) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM and installed by inserting the storage medium into a drive device. The storage unit 150 is realized by a storage device such as a HDD, flash memory, or RAM (Random Access Memory). The storage unit 150 stores, for example, a trained model 152, a weeding work history 154, and a weeding work plan 156.

[0019] The information acquisition unit 110 acquires images captured by the camera 10 mounted on the weeding robot 20, along with location information for the images and the work time required to weed the work area captured in the images. FIG. 2 is a diagram showing an example of a combination of image information, location information, and work time acquired by the information acquisition unit 110. As shown in FIG. 2, while the weeding robot 20 is traveling through the field and performing weeding work, the information acquisition unit 110 acquires, in chronological order, images of the work area captured by the camera 10, longitude and latitude information for the work area, and the work time required to weed the work area from the weeding robot 20. To measure the work time, for example, a CPU mounted on the weeding robot 20 may calculate the period between the times at which each image was captured and consider the calculated period to be the work time.

[0020] The weed amount determination unit 120 determines the amount of weeds for each work area based on the image acquired by the information acquisition unit 110. More specifically, when an image is input, the weed amount determination unit 120 determines the amount of weeds for each work area using a trained model 152 that has been trained to output a discrete value indicating the amount of weeds in the image. Alternatively, the weed amount determination unit 120 may input the image to a filter that extracts weed-colored pixels (for example, a range specified by RGB values) and determine the amount of weeds by calculating the proportion of weed-colored pixels in the entire image.

[0021] 3 is a diagram showing an example of a method for determining the amount of weeds by the weed amount determination unit 120. The administrator of the information processing device 100 prepares training data that links images of weeds in a field taken in advance with values ​​that represent the amount of weeds in the images, and generates a trained model 152 by having any machine learning model learn the training data. The generated trained model 152 is stored in the storage unit 150. When the information acquisition unit 110 acquires an image from the weeding robot 20, the weed amount determination unit 120 inputs the acquired image into the trained model 152 to obtain a value that represents the amount of weeds in the image.

[0022] 3 shows a case where the trained model 152, in response to an input image, represents and outputs the amount of weeds in the input image using three values ​​(1, 2, 3) where a higher value indicates a greater amount of weeds. However, the trained model 152 of the present invention is not limited to such a configuration, and more generally, it may be any model that outputs the amount of weeds (in other words, the proportion of the area of ​​weeds in the image) using a discrete value.

[0023] After determining the amount of weeds for each work area, the weed amount determination unit 120 aggregates the determined weed amounts to calculate the amount of weeds for each work section that includes one or more work areas. Fig. 4 is a diagram showing the amount of weeds for each work section calculated by the weed amount determination unit 120. In Fig. 4, work sections A to H each include one or more work areas. A work area is a field area that is a unit of photography by the camera 10, while a work section represents the smallest work unit in which the weeding robot 20 weeds in one weeding operation in the field. The work sections are defined in advance by the administrator of the information processing device 100.

[0024] The weed amount determination unit 120 calculates the weed amount and work time for each work section based on the weed amount and work time of one or more work areas included in the work section. In the example of FIG. 4, the weed amount determination unit 120 sets the integrated value obtained by integrating the weed amount and work time of the work areas included in each work section as the weed amount and work time of the work section. However, the present invention is not limited to this configuration, and the weed amount determination unit 120 may, for example, set the average value of the weed amount of the work areas included in each work section as the weed amount of the work section. The weed amount determination unit 120 stores the calculated combination of weed amount and work time for each work area in the memory unit 150 as weeding work history 154.

[0025] The weeding intensity prediction unit 130 predicts, for each work section, a future value of the weeding intensity, which is an index value indicating the level of the weeding load, based on the weeding history 154. For example, the weeding intensity is expressed by a combination of the work section, the amount of weeds, and the work time (a one-row, three-column vector). In other words, the weeding intensity is a vector that includes not only the amount of weeds but also the work time, and therefore can be said to be a concept that takes into account the number of turns made by the weeding robot 20 and the work content (for example, one-way travel or round-trip travel). The weeding intensity prediction unit 130 can predict the future value of the weeding intensity, for example, by performing a regression analysis on past weed amount and work time data recorded for each work section in the field.

[0026] The weeding work planning unit 140 aggregates multiple work sections into packets, which are unit areas in which weeding can be performed within a unit time, based on the future values ​​of the weeding intensity predicted for each work section by the weeding intensity prediction unit 130. More specifically, the weeding work planning unit 140 aggregates work sections with larger values ​​of the weed quantity included in the weeding intensity into packets in which weeding will be performed earlier. At the same time, the weeding work planning unit 140 aggregates multiple work sections into packets so that the total value of the work time included in the weeding intensity does not exceed the unit time of the packet.

[0027] FIG. 5 is a diagram illustrating a method by which the weeding work planning unit 140 aggregates work sections into packets. In FIG. 5, packets 1 to 4 each represent a unit time (e.g., 180 minutes) during which the weeding robot 20 performs weeding in one weeding operation, and each packet includes at least one work section. FIG. 5 illustrates a situation in which work sections with larger weed volume values ​​are aggregated in order into packets with smaller numbers. If adding a new work section to packet N causes the total work time of the work sections included in packet N to exceed the unit time of packet N, the work section is aggregated into the next packet N+1. In this way, by arranging work sections into packets taking into account not only weed volume but also work time, the number of turns and work content of the weeding robot 20 can be reflected in the scheduling of weeding work. Note that, for ease of understanding, the values ​​of weed volume and work time shown in FIG. 5 (i.e., future values) are expressed as the same values ​​as the values ​​of weed volume and work time shown in FIG. 4 (i.e., current values), but in reality, these values ​​are different from each other.

[0028] After aggregating the work sections into packets, the weeding work planning unit 140 plans weeding work by arranging the obtained packets so that weeding work will be carried out on a predetermined appropriate work date. Figure 6 is a diagram showing an example in which the weeding work planning unit 140 arranges packets on appropriate work dates. Figure 6 shows a scene in which the weeding work planning unit 140 transmits a weeding work schedule screen on which the packets have been arranged to the terminal device 30 via the network NW and displays it on the terminal device 30.

[0029] In FIG. 6, symbol A1 denotes an area for setting the sowing date. As shown in the calendar in FIG. 6, when the sowing date is set in area A1, the appropriate work date corresponding to the set sowing date (for example, in the case of spinach, a period of 2 to 4 weeks from the sowing date) is displayed. In FIG. 6, as an example, the appropriate work date is displayed as an area surrounded by a thick line. Symbol A2 denotes an area for specifying a date and time period when there will be a shortage of workers. When a date and time period are specified in area A2, the date and time period are excluded from the targets for packet placement. Symbol A3 denotes an area for specifying a date and time period when the weather will be bad. When a date and time period are specified in area A3, the date and time period are excluded from the targets for packet placement.

[0030] The weeding work planning unit 140 generates a weeding work plan by arranging packets 1 to 4 in order for the appropriate work dates, and displays the weeding work plan as a calendar on the terminal device 30. The user of the terminal device 30 checks the weeding work plan on the calendar and finally confirms the weeding work plan by pressing button B2. When the information acquiring unit 110 receives information indicating that the user has confirmed the weeding work plan, it stores the confirmed weeding work plan in the storage unit 150 as a weeding work plan 156. Thereafter, when the weeding work date stored in the weeding work plan 156 arrives, the information processing device 100 transmits the weeding work plan 156 to the weeding robot 20, and the weeding robot 20 performs weeding work in the field in accordance with the order of the packets (and the work sections included in the packets) defined in the weeding work plan 156.

[0031] FIG. 7 is a diagram showing another example in which the weeding work planning unit 140 arranges packets on appropriate work days. Unlike FIG. 6, FIG. 7 illustrates a situation in which the user of the terminal device 30 specifies a worker shortage in area A2 and bad weather days in area A3. When the user of the terminal device 30 specifies this information, the information acquisition unit 110 accepts this information via the network NW. The weeding work planning unit 140 generates a weeding work plan by arranging packets in order, excluding the worker shortage days and bad weather days from appropriate work days. The weeding work planning unit 140 displays the generated weeding work plan as a calendar on the terminal device 30. The user of the terminal device 30 checks the weeding work plan on the calendar and, as in FIG. 6, finally confirms the weeding work plan by pressing button B2.

[0032] Fig. 8 is a diagram showing an example of a screen for allowing the user of the terminal device 30 to set a packet. In Fig. 8, symbol A4 represents an area for inputting the battery capacity of the weeding robot 20, and symbol A5 represents an area for inputting the unit time of the packet. When the user inputs the battery capacity in area A4, for example, the weeding work planning unit 140 calculates the maximum unit time that can be set for the packet based on the input battery capacity and displays it on the terminal device 30. The user of the terminal device 30 can specify the unit time of the packet in area A5 while referring to the displayed maximum unit time.

[0033] When the user specifies the unit time of a packet, the weeding work planning unit 140 aggregates work sections into a packet having the specified unit time. FIG. 8 shows a case where the unit time of a packet is changed from 180 minutes to 240 minutes, resulting in each packet containing more work sections and a decrease in the number of packets from four to three. In this way, the user can set the unit time of a packet taking into consideration the battery capacity of the weeding robot 20 as well as their own working hours and break times. When the user presses button B3, the unit time of the packet is confirmed, and the weeding work planning unit 140 generates a weeding work plan by arranging work sections so that they fit within the set unit time of the packet.

[0034] Next, the flow of processing executed by the information processing device 100 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the flow of processing executed by the information processing device 100. First, the information acquisition unit 110 acquires an image of a work area in a field from the weeding robot 20 performing weeding work in the field, together with position information of the work area and the work time (step S101). Next, the weed amount determination unit 120 inputs the image into the trained model 152, thereby determining the weed amount in the work area corresponding to the image (step S102).

[0035] Next, the weed amount determination unit 120 calculates the weed amount and work time for each work section by integrating the weed amount and work time for each determined work area (step S103). Next, the weeding intensity prediction unit 130 predicts the future value of the weeding intensity for that work section based on the calculated weed amount and work time for each work section (step S104).

[0036] Next, the weeding work planning unit 140 aggregates the work sections into packets based on the predicted future values ​​of the weeding work intensity (step S105). More specifically, the weeding work planning unit 140 aggregates the work sections into packets so that weeding work is carried out in order starting with the work section with the highest future value of the weeding work intensity, so that the weeding work fits within the unit time of the packet specified by the user. Next, the weeding work planning unit 140 generates a weeding work plan by arranging the aggregated packets in order on appropriate work days (step S106). At this time, if the user specifies days on which there is a shortage of workers or days on which the weather is bad, the weeding work planning unit 140 arranges the packets on appropriate work days excluding the specified days on which there is a shortage of workers or days on which the weather is bad. This completes the processing of this flowchart.

[0037] According to the present embodiment described above, the weeding work by the weeding robot can be leveled out by aggregating the work sections of the field where weeding work is to be carried out into packets with a set unit time.

[0038] Furthermore, according to this embodiment, the work sections are aggregated into packets so that the total work time of the multiple work sections aggregated into the packet falls within the unit time range of the packet, taking into account not only the amount of weeds in the work section but also the work time. This allows the number of rotations of the weeding robot and the work content to be reflected in the formulation of the weeding work plan.

[0039] Furthermore, according to this embodiment, in response to the user's designation of days when there are shortages of workers or days when the weather is bad, the weeding schedule is generated by excluding these designated days from the dates when weeding work is appropriate and arranging packets in order, which can contribute to the allocation of other support work and personnel. [Explanation of symbols]

[0040] 10 Camera 20 Weeding robot 30 Terminal Equipment 100 Information processing device 110 Information Acquisition Department 120 Weed quantity determination unit 130 Weeding work intensity prediction section 140 Weeding Work Planning Department 150 Storage section 152 trained models 154 Weeding work history 156 Weeding Work Plan

Claims

1. an information acquisition unit that acquires, for each of one or more work areas included in a work section of a field, images taken by a camera mounted on the weeding robot together with the work time of the weeding robot; a weed amount determination unit that determines the amount of weeds for each of the one or more work areas based on the image and aggregates the determined amount of weeds for each of the one or more work areas to determine the amount of weeds for each of the work sections; a weeding intensity prediction unit that predicts a future value of a weeding intensity, which is an index value indicating the level of the load of weeding work, for each work section based on a work history including the past work time and the past weed amount; a weeding work planning unit that plans weeding work to be performed by the weeding robot in the field based on the weeding work intensity, Information processing device.

2. the weeding work planning unit aggregates the plurality of work sections into one or more packets, which are unit areas in which the weeding work can be performed within a unit time, and plans the weeding work by arranging the aggregated one or more packets so that the weeding work is performed on a predetermined appropriate work date. The information processing device according to claim 1 .

3. the future value of the weeding intensity includes a future value of the weed amount, the weeding work planning unit aggregates work sections having a large future value of the weed amount into a packet in which the weeding work is to be carried out earlier; The information processing device according to claim 2 .

4. the information acquisition unit further accepts input of external information by a user; the external information includes at least one of information regarding personnel on the optimal work day and information regarding weather on the optimal work day; the weeding work planning unit arranges the one or more packets further based on the received external information; The information processing device according to claim 2 .

5. the information acquisition unit further accepts input of setting information by a user; the setting information includes at least one of the unit time of the packet and a battery capacity of the weeding robot; the weeding work planning unit determines a size of the one or more packets further based on the received setting information. The information processing device according to claim 2 .

6. the weed amount determination unit, when an image is input, determines the amount of weeds for each of the one or more work areas using a trained model that has been trained to output a discrete value indicating the amount of weeds in the image; The information processing device according to claim 1 .

7. The computer acquiring images taken by a camera mounted on a weeding robot together with the working time of the weeding robot for each of one or more work areas included in a work section of a farm field; determining the amount of weeds for each of the one or more work areas based on the image, and aggregating the determined amount of weeds for each of the one or more work areas to determine the amount of weeds for each of the work sections; predicting a future value of a weeding intensity, which is an index value indicating the level of the load of weeding work, for each work section based on a work history including the past work time and the weed amount; planning weeding work to be performed by the weeding robot in the field based on the weeding work intensity; Information processing methods.

8. On the computer, acquiring images taken by a camera mounted on a weeding robot together with the working time of the weeding robot for each of one or more work areas included in a work section of a field; determining the amount of weeds for each of the one or more work areas based on the image, and aggregating the determined amount of weeds for each of the one or more work areas to determine the amount of weeds for each of the work sections; predicting a future value of a weeding intensity, which is an index value indicating the level of the load of weeding work, for each work section based on a work history including the past work time and the weed amount; planning weeding work to be performed by the weeding robot in the field based on the weeding work intensity; program.

Images