Display controller, display control system, display control method, and program
The display control device addresses the challenge of inappropriate situational assessment in smart agriculture by prioritizing display of growth information and user actions, enabling effective decision-making.
Patent Information
- Application Number
- JP2023079443
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing display systems in smart agriculture fail to appropriately determine the current situation and necessary actions for agricultural fields.
A display control device that acquires growth information from multiple plants using environmental and plant information, determines information to be displayed based on user actions, and prioritizes display order to facilitate appropriate judgment of the current situation.
Enables users to accurately assess the current situation and necessary actions in agricultural fields, enhancing decision-making efficiency.
Smart Images

Figure 2024163651000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a display control device, a display control system, a display control method, and a program. [Background technology]
[0002] In recent years, smart agriculture using remote sensing technology has been proposed for the purpose of improving the efficiency and labor saving of agricultural work. Patent Document 1 discloses a configuration in which, when displaying the results of measurements on agricultural crops as a graph, the measurement values of each of multiple items are converted into display coordinate values on the graph so that the reference values set for each of the multiple items match on the graph. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2021-005339 A Summary of the Invention [Problem to be solved by the invention]
[0004] The configuration of Patent Document 1 makes it impossible to appropriately determine the current situation, such as which action should be taken in which field.
[0005] An object of the present invention is to provide a display control device that enables a user to appropriately judge a current situation. [Means for solving the problem]
[0006] A display control device as one aspect of the present invention is characterized by having an acquisition unit that acquires growth information of a plurality of plants based on at least one of environmental information and plant information, and a decision unit that determines information to be displayed on the display unit based on the growth information and information related to actions performed by the user. Effect of the Invention
[0007] According to the present invention, it is possible to provide a display control device that enables a user to appropriately judge a current situation. [Brief description of the drawings]
[0008] [Figure 1] FIG. 1 is a block diagram of an information processing system according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram of a growth information acquisition system according to a first embodiment. [Diagram 3] FIG. 2 is a diagram showing a state in which an image of a subject is captured using the growth information acquisition system of the first embodiment. [Figure 4] 6 is a flowchart showing a process until information is displayed on the display unit in the first embodiment. [Diagram 5] FIG. 4 is a diagram showing an example of information displayed on a display unit in the first embodiment. [Figure 6] FIG. 11 is a block diagram of an information processing system according to a second embodiment. [Figure 7] 10 is a flowchart showing a process until information is displayed on a display unit in the second embodiment. [Figure 8] FIG. 11 is a diagram showing an example of information displayed on a display unit according to the second embodiment. [Figure 9] FIG. 11 is a diagram illustrating an example of action priority order information according to the second embodiment. [Figure 10] FIG. 13 is a diagram showing another example displayed on the display unit of the second embodiment. [Figure 11] FIG. 13 is a diagram showing another example displayed on the display unit of the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same components are designated by the same reference numerals, and duplicated descriptions will be omitted. EXAMPLES
[0010] 1 is a block diagram of an information processing system (display control system) 100 of this embodiment. The information processing system 100 has growth information acquisition systems 101, 102, 103, and 104. The information processing system 100 also has an analysis unit 120, a control unit 121, a storage unit 130, and a display unit 140. The analysis unit 120 and the control unit 121 configure a display control device. Each unit of the information processing system 100 only needs to be connected to at least the display control device so as to be able to communicate with the display control device.
[0011] The growth information acquisition systems 101-104 each have a sensor unit (detection unit) 111, 112, 113, 114. Each sensor unit includes, for example, a camera, a color sensor, a distance sensor, a temperature sensor, and a water level sensor, and detects environmental information and plant information. In this embodiment, each sensor unit detects environmental information and plant information of different regions. The environmental information is, for example, temperature, humidity, illuminance, color of ambient light, and water level. The plant information is, for example, the color, size, and number of each part of the plant (crop). In this embodiment, rice is used as an example of a plant. In this embodiment, the number of growth information acquisition systems is four, but other numbers may be used.
[0012] The analysis unit 120 analyzes the environmental information and plant information detected by the sensor unit to obtain analysis result information 131 including plant growth information based on at least one of the environmental information and the plant information. The growth information is, for example, leaf color, plant height, stalk number, grain color, accumulated temperature, estimated rice transplantation date information, estimated rice heading date information, estimated yield information, and the like. The analysis result information 131 may be a graph with the horizontal axis representing time series and the vertical axis representing growth information. The analysis result information 131 may also include information regarding abnormalities such as obstacle information and system abnormality information.
[0013] The control unit (determination unit) 121 determines information to be displayed on the display unit 140 according to the plant growth information and information related to an action taken by the user. Determining the information to be displayed on the display unit 140 may mean determining the display order of the growth information, or may mean displaying a display that prompts the user to take an action.
[0014] The storage unit 130 stores analysis result information 131. The storage unit 130 stores an action database 132 including information on user actions. The information on user actions includes action information 133 that lists actions performed by the user, and action schedule information 134 that is a condition for plant growth information corresponding to the action. The action information 133 is, for example, information that lists actions such as mid-season drainage, irrigation, pesticide spraying, fertilization, harvesting, obstacle removal, and replacement due to equipment failure. The action schedule information 134 is, for example, "the number of stalks per rice plant is 20 or more" if the corresponding action is "perform mid-season drainage", or "the accumulated temperature reaches 1000 degrees based on the rice heading date" if the corresponding action is "perform harvesting".
[0015] The information regarding the user's actions may be configured to be changeable in response to a user's instruction, or may be configured to allow items to be added.
[0016] The display unit 140 displays the growth information for each of the multiple growth information acquisition systems 101-104 in parallel. In this embodiment, the control unit 121 determines the display order of the growth information on the display unit 140.
[0017] 2 is a block diagram of the growth information acquisition system 101. Note that the other growth information acquisition systems have the same configuration as the growth information acquisition system 101, so detailed explanations will be omitted.
[0018] The growth information acquisition system 101 includes the above-mentioned sensor unit 111, an image processing unit 154, a control unit 155, a storage unit 156, and a communication unit 157.
[0019] The sensor unit 111 includes an ambient light information acquisition unit 150 consisting of a color sensor, a temperature information acquisition unit 151 consisting of a thermometer, a water level information acquisition unit 152 consisting of a water level gauge, and an imaging unit 153 consisting of a camera.
[0020] The imaging unit 153 includes an imaging optical system 153a and an imaging element 153b, and is capable of acquiring at least a visible light image. The imaging element 153b photoelectrically converts an optical image (subject image) formed via the imaging optical system 153a, and outputs an image (image data) to the image processing unit 154. The image processing unit 154 includes an acquiring unit 154a and a separating unit 154b.
[0021] In this embodiment, the sensor unit 111 includes four sensor members (sensing devices), but it is sufficient that the sensor unit 111 includes at least one sensor member that detects information that enables the analysis unit 120 to obtain the analysis result information 131. If no sensor member related to images is provided, the image processing unit 154 is not necessary.
[0022] The growth information acquisition system 101 may be provided inside an electronic device such as a camera or various sensor components. The growth information acquisition system 101 may also be configured so that some functions such as the image processing unit 154 are realized on a computer (user PC) away from the electronic device or on cloud computing. In this case, the electronic device is configured to have some of the functions of the growth information acquisition system 101.
[0023] FIG. 3 is a diagram showing a state where a subject 160, which is a plant, is imaged under outdoor ambient light using the growth information acquisition system 101. The ambient light information acquisition unit 150 includes an ambient light sensor 150a that acquires the illuminance of light incident on the subject 160 for each predetermined wavelength. The wavelength range that the ambient light sensor 150a can acquire is the wavelength range of visible light, specifically, the wavelength range of blue, green, and red in the region of about 400 nm to 700 nm. The ambient light sensor 150a may be, for example, a photoelectric conversion element such as a CCD or CMOS sensor on which red, green, and blue color filters are provided. The range indicated by the dotted line in FIG. 3 is the angle of view range 171 of the imaging unit 153.
[0024] 4 is a flowchart showing the process up to when information is displayed on the display unit 140. The process of each step in FIG. 4 is executed by a display control device.
[0025] In step S11, the control unit 121 acquires a user request to display the analysis result information 131. The user makes the display request for the analysis result information 131 via, for example, a browser or an application executable on a smartphone, a PC, or the like.
[0026] In step S12, the analysis unit 120 acquires the analysis result information 131.
[0027] In step S13, the control unit 121 acquires the action schedule information 134. Furthermore, the control unit 121 checks whether there is an action item to be displayed on the display unit 140 based on the current date and time information and the action schedule information 134.
[0028] In this embodiment, the process of step S13 is performed after the process of step S12, but the process of step S12 may be performed after the process of step S13.
[0029] In step S14, the control unit 121 determines information to be displayed on the display unit 140 based on the execution contents up to step S13. In this embodiment, the control unit 121 determines the display order of the growth information. Note that in this embodiment, the control unit 121 determines the display order so that the priority is high, but the growth information with high priority may be highlighted or the color of the display frame may be changed. Also, a message may be displayed to inform the user that the priority is high.
[0030] It is preferable that the control unit 121 acquires from the storage unit 130 the growth information last viewed when the user made the previous display request, and determines the information to be displayed on the display unit 140 based on the acquired growth information. For example, if the display based on the initial settings is performed every time the user makes a display request, it is necessary to select the growth information every time in order to display the information the user wants to view. Therefore, it is preferable that the control unit 121 displays the growth information while maintaining the information last viewed by the user (for example, the display contents arranged in order of the earliest harvest time). This allows the display to be quickly restored to the same state as the previous time, even if the display is turned off once and then displayed again.
[0031] Furthermore, when there is no previous display history and the information is to be displayed for the first time, the control unit 121 may determine the information to be displayed on the display unit 140 according to the degree of agreement between the growth information and the action schedule information 134 .
[0032] In step S15, the control unit 121 causes the information determined in step S14 to be displayed on the display unit 140. In this embodiment, the control unit 121 causes the display unit 140 to display the growth information in the display order determined in step S14.
[0033] FIG. 5 is a diagram showing an example of information displayed on the display unit 140, and shows an example in which growth information based on information from each growth information acquisition system that senses the state of rice in a paddy field is displayed side by side. In FIG. 5, the growth information on the far left is the information that the user is most interested in viewing (has the highest priority), and the priority decreases as the information moves to the right. The horizontal axis of each growth information in FIG. 5 is the date and time, and the vertical axis is the number of stalks, which is an example of growth information. As described above, there are many action items, but for simplicity, we will focus on mid-drying. In addition, when the action schedule information 134 of "rice variety A with 20 or more stalks" is satisfied, a display encouraging mid-drying is displayed. Furthermore, when the action schedule information 134 of "rice variety B with 25 or more stalks" is satisfied, the action schedule information 134 encouraging mid-drying is displayed.
[0034] In FIG. 5, the number of stalks based on information from the growth information acquisition system 104 that senses rice plants that have reached the time for mid-drying is displayed first (leftmost) counting from the left. From the second one counting from the left, the number of stalks based on information from the growth information acquisition system that senses rice plants for which the difference between the current number of stalks and the number of stalks that will trigger a display encouraging mid-drying is small (the time for mid-drying is approaching) is displayed in order. For example, the number of stalks acquired from the growth information acquisition system 101 is greater than the number of stalks acquired from the growth information acquisition system 102, but the number of stalks acquired by the growth information acquisition system 102 has a smaller difference between the current number of stalks and the number of stalks that will trigger a display encouraging mid-drying. Therefore, the number of stalks acquired from the growth information acquisition system 102 is displayed as information with a higher priority than the number of stalks acquired from the growth information acquisition system 102.
[0035] In FIG. 5, the number of stalks with a small difference from the number of stalks that will be displayed to encourage mid-drying is displayed with a higher priority, but this is not limited as long as it is possible to display the number of stalks of rice information that is likely to be the timing for mid-drying with a higher priority. For example, a ratio may be used instead of a difference.
[0036] In addition, when the user needs to take action, such as "mid-drying," it is preferable to display a message encouraging the user to take action, such as "It's time for mid-drying" in Figure 5, which explains what action is required.
[0037] In addition, when the user performs the corresponding action after the display prompting the action is displayed, a display indicating that the action is completed may be displayed on the display unit 140. Note that the user performing the action may not be the user viewing the display unit 140, but may be a user (worker) near the corresponding growth acquisition system.
[0038] In addition, the information on the actions is preset at the time of shipment from the factory, but may be configured to be changeable by the user, and items may be added. For example, if the number of stalks and the height of the plant can be output as the growth information in the initial state, it is possible to change so that an index obtained by multiplying the number of stalks and the height of the plant is displayed as the growth information.
[0039] As described above, the configuration of this embodiment enables the user to appropriately determine the current situation. EXAMPLES
[0040] 6 is a block diagram of an information processing system 100 of this embodiment. The basic configuration of the information processing system 100 of this embodiment is similar to that of the information processing system 100 of embodiment 1. In this embodiment, only configurations different from embodiment 1 will be described, and descriptions of common configurations will be omitted.
[0041] This embodiment differs from the first embodiment in that the storage unit 130 is configured on the cloud. Also, the information on the action differs from the first embodiment in that it includes action priority information 135 and action auxiliary information 136. Also, the number of growth information acquisition systems differs from the first embodiment in that it is five or more, although some of them are not shown.
[0042] The action priority information 135 is information that specifies the priority of actions. For example, in the case of multiple rice plants sensed by different growth information acquisition systems, when the date for spreading fertilizer and the date for spraying pesticide are the same day, it is possible to determine which action to display first (with higher priority) using the action priority information 135. This allows the control unit 121 to determine the display order based on the action priority information 135 even if different actions are performed on the same day.
[0043] The action auxiliary information 136 is information for performing an action more efficiently, and is used in combination with the action information 133. The action auxiliary information 136 is, for example, current location information of the user, installation location information of the growth information acquisition system, past analysis result information, cultivation calendar information, variety information of plants in the field where the growth information acquisition system is installed, the rice transplanting date, and the rice heading date.
[0044] The action priority information 135 and the action auxiliary information 136 may be configured so as to be changeable by the user, and items may be added.
[0045] Fig. 7 is a flowchart showing the process up to the display of information on display unit 140. The process of each step in Fig. 7 is mainly executed by the display control device.
[0046] The processes in steps S21 to S23 are similar to those in steps S11 to S13 in FIG. 4, respectively, and therefore will not be described in detail.
[0047] In step S24, the control unit 121 acquires the action priority information and the action auxiliary information. In this embodiment, the process of step S24 is performed after the process of step S23, but the process of step S24 may be performed between the processes of steps S21 and S25.
[0048] The processes in steps S25 to S26 are similar to those in steps S14 to S15 in FIG. 4, respectively, and therefore will not be described in detail.
[0049] The following describes an example of utilizing the action priority information 135. Fig. 8 is a diagram showing an example of information displayed on the display unit, and shows information displayed when the growth information acquisition system 104 is in a state where normal sensing cannot be performed due to a malfunction or the like.
[0050] In this embodiment, when the action schedule information 134 "grass height exceeds 60 cm" is met, a display is set to prompt the user to take the action of "spread fertilizer." In Fig. 8, the rice plant height sensed by the growth information acquisition system 101 exceeds 60 cm, and a display is displayed to prompt the user to take the action "It's time to spread fertilizer."
[0051] For the grass heights third from the left onwards, the order in which the next action is needed (for example, the timing to spray pesticides to control pests) is displayed in order of proximity, along with an indicator for determining this (for example, the number of stalks).
[0052] Fig. 9 is a diagram showing an example of the action priority information 135. When the days on which an action is required are the same, the priorities described in the action priority information 135 are displayed on the display unit 140 in descending order of priority. In Fig. 9, the priority of "spreading fertilizer (fertilization)" is 5, and the priority of "replacement due to equipment failure" is 1, so the number of stalks of the growth information acquisition system that needs to be replaced is displayed in the left position with the highest priority.
[0053] After replacement due to equipment failure is performed, the display order is changed or updated. FIG. 10 is a diagram showing another example of information displayed on the display unit 140 when a change occurs in the display order. When a change occurs in the display order determined by the control unit 121 and the display order is changed or updated based on the time when the user last made a display request, it is preferable that the control unit 121 functions as a notification unit that notifies the user that the display order has changed. For example, the control unit 121 may display a message such as "the display order has changed" on the display unit 140, or may change the color of an icon or a display frame. Note that the notification method is not limited to displaying on the display unit 140, and may be any method that can notify the user of the change.
[0054] An example of using the action auxiliary information 136 will be described below. Assume that the user's current location (or the location registered in the action auxiliary information) is stored as the action auxiliary information 136, and that it is time for mid-season drying of rice plants sensed by two of the operating growth information acquisition systems. FIG. 11 is a diagram showing another example of information displayed on the display unit 140 based on the display order based on the action auxiliary information and the action information. In FIG. 11, analysis result information of the growth information acquisition system closer to the user's current location is displayed with a higher priority. This is preferable because it makes it easier for the user to consider what route to take to get to the site even for action information within the same day.
[0055] In this embodiment, the display order is the order of proximity to the user's current location, but it may be the order of distance from the user's current location. It may also be the order of deviation from past analysis result information, or the order of high / low reliability of analysis result information. Combining the action information 133 and the action support information 136 in this way is preferable because it allows the display to be made in a display order that includes suggestions that allow the user to make more efficient considerations.
[0056] As described above, the configuration of this embodiment enables the user to appropriately determine the current situation. [Other Examples] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-mentioned embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0057] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) an acquisition unit that acquires growth information of the plurality of plants based on at least one of environmental information and plant information; A display control device comprising: a decision unit that decides information to be displayed on a display unit according to the growth information and information related to an action taken by a user. (Configuration 2) The control device according to configuration 1, wherein the determination unit determines a display order of the growth information on the display unit according to the growth information and information related to the action. (Configuration 3) 3. The control device according to configuration 1 or 2, wherein the determination unit causes the display unit to display a message encouraging the user to take the action in accordance with the growth information and information regarding the action. (Configuration 4) The control device according to configuration 3, wherein the decision unit causes the display unit to display a message indicating that the action has been completed. (Configuration 5) 5. The control device according to any one of configurations 1 to 4, wherein the information regarding the action includes a condition for the growth information corresponding to the action. (Configuration 6) 6. The information processing system according to configuration 5, wherein the information about the action includes information defining a priority of the action. (Configuration 7) 7. The control device according to configuration 5 or 6, further comprising a storage unit that stores information relating to the action. (Configuration 8) A control device described in any one of configurations 1 to 7, further comprising a notification unit that notifies of a change when the information to be displayed on the display unit changes from the information previously displayed on the display unit. (Configuration 9) 9. The control device according to any one of configurations 1 to 8, wherein the information regarding the action is changeable in response to an instruction from a user. (Configuration 10) A display control device according to any one of configurations 1 to 9; A display control system comprising a detection unit that detects the environmental information and the plant information. (Configuration 11) the detection unit includes a first detection unit and a second detection unit, The display control system according to configuration 10, wherein the first detection unit and the second detection unit acquire the environmental information and the plant information in different regions. (Method 1) acquiring growth information of a plurality of plants based on at least one of environmental information and plant information; A display control method comprising a step of determining information to be displayed on a display unit according to the growth information and information related to an action taken by a user. (Configuration 12) A program for causing a computer to execute the display control method according to method 1.
[0058] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]
[0059] 120 Analysis section (acquisition section) 121 Control unit (decision unit) 140 Display section
Claims
1. an acquisition unit that acquires growth information of the plurality of plants based on at least one of environmental information and plant information; A display control device comprising: a decision unit that decides information to be displayed on a display unit in accordance with the growth information and information related to an action taken by a user.
2. The control device according to claim 1 , wherein the determination unit determines a display order of the growth information on the display unit in accordance with the growth information and information related to the action.
3. The control device according to claim 1 or 2, wherein the determination unit causes the display unit to display a message encouraging the user to take the action in accordance with the growth information and the information regarding the action.
4. The control device according to claim 3 , wherein the determination unit causes the display unit to display a display indicating that the action has been completed.
5. 3. The control device according to claim 1, wherein the information about the action includes a condition for the growth information corresponding to the action.
6. 6. The information processing system according to claim 5, wherein the information about the action includes information defining a priority of the action.
7. The control device according to claim 5 , further comprising a storage unit that stores information about the action.
8. 3. The control device according to claim 1, further comprising a notification unit that notifies the user of a change in the information displayed on the display unit when the information displayed on the display unit changes from the information previously displayed on the display unit.
9. 3. The control device according to claim 1, wherein the information about the action is changeable in response to an instruction from a user.
10. A display control device according to claim 1 or 2; A display control system comprising a detection unit that detects the environmental information and the plant information.
11. the detection unit includes a first detection unit and a second detection unit, The display control system according to claim 10 , wherein the first detection unit and the second detection unit acquire the environmental information and the plant information in different regions.
12. acquiring growth information of a plurality of plants based on at least one of environmental information and plant information; A display control method comprising a step of determining information to be displayed on a display unit according to the growth information and information related to an action taken by a user.
13. A program causing a computer to execute the display control method according to claim 12.
Citation Information
Patent Citations
Information processing apparatus, control method for the same, and program
JP2021005339A
Cited By
Agricultural automation support system, growth management device, agricultural automation support method and program
JP2026138780A
Agricultural automation support system, growth management device, agricultural automation support method and program
JP7873808B1
