Information processing device, system, information processing method, and program

Abstract

To achieve more intuitive water level settings for users.SOLUTION: An information processing device includes: an acquisition unit that acquires an image of a field and a water-use facility, captured with a camera in the vicinity of the field or the water-use facility; a generation unit that generates a virtual water surface representing the water surface of the field or the water-use facility shown in the image, and generates an AR image where the generated virtual water surface has been superimposed upon the image; a display control unit that causes the AR image to be displayed on a user terminal device in a manner that a position of the virtual water surface can be changed; and a setting unit that sets target water levels of the field and the water-use facility on the basis of operation to change the position of the virtual water surface on the user terminal device.SELECTED DRAWING: Figure 1

Description

Technical Field

[0006]

[0001] The present invention relates to an information processing apparatus, a system, an information processing method, and a program.

Background Art

[0002] Conventionally, a technique of installing a water level gauge in a field or a water conservancy facility to measure the water level and performing water supply or drainage to the field or the water conservancy facility based on the measured water level is known. For example, Patent Document 1 discloses a technique of calculating the required water supply amount for each group of fields based on the collected water supply state information of the fields and remotely controlling the opening and closing of the water supply taps of each field.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] The technique described in Patent Document 1 receives a set water level from a user's terminal device and calculates the required water supply amount based on the received set water level. At this time, generally, in the prior art, the current water level measured by the water level gauge is displayed on the terminal device, and the user sets the target water level by numerically specifying it while referring to the displayed current water level.

[0005] However, in the prior art, an error occurs due to the limitation of the measurement accuracy of the water level gauge, so the user cannot always accurately set the target water level. In addition, the installation location of the water level gauge is restricted. For example, when installing the water level gauge in a paddy field, it is necessary to appropriately select a location that does not interfere with agricultural operations other than water management. Furthermore, numerically specifying the water level is not intuitive in the first place, and it may be difficult to set an appropriate water level, resulting in low convenience for the user.

Problems to be Solved by the Invention

[0006] This invention was made in consideration of these circumstances, and one of its objectives is to provide an information processing device, system, information processing method, and program that enable more intuitive water level setting for the user. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present invention comprises: an acquisition unit that acquires images of a field or water facility captured by a camera located inside or near the field or water facility; a generation unit that generates a virtual water surface representing the water surface of the field or water facility as captured in the image, and generates an AR image by superimposing the generated virtual water surface onto the image; a display control unit that displays the AR image on a user terminal device so that the position of the virtual water surface can be changed; and a setting unit that sets a target water level for the field or water facility based on an operation to change the position of the virtual water surface performed on the user terminal device.

[0008] The acquisition unit may acquire images of the field or water facilities captured by the camera located near the field or water facilities and in a position that facilitates human recognition of the water surface.

[0009] The generation unit may generate the virtual water surface on a two-dimensional plane defined by the boundary line between the water surface and non-water surface of the field or irrigation facility as captured in the image, and a reference line perpendicular to the boundary line.

[0010] The generation unit may determine the water surface and generate the virtual water surface by analyzing wavelengths other than visible light detected by sensors located near the field or water facilities, or the wavelengths of light emitted from the water surface.

[0011] The display control unit may cause the user terminal device to display the AR image so that the position of the virtual water surface can be changed in a direction orthogonal to the two-dimensional plane.

[0012] The setting unit may set the target water level by increasing or decreasing the water level of the field or irrigation facility according to the amount of operation performed.

[0013] The setting unit may set the target water level such that when the position of the virtual water surface is changed in the upward direction of the orthogonal directions, the water level of the field or irrigation facility is increased, and when the position of the virtual water surface is changed in the downward direction of the orthogonal directions, the water level of the field or irrigation facility is decreased. [Effects of the Invention]

[0014] According to the present invention, it is possible to achieve intuitive water level setting for the user without using a water level gauge and without explicitly specifying a numerical value. [Brief explanation of the drawing]

[0015] [Figure 1] This figure shows an example of the operating environment and configuration of the information processing device 100. [Figure 2] This figure shows an example of an image IM acquired by the image acquisition unit 110. [Figure 3] This figure shows an example of an AR image generated by the AR image generation unit 120. [Figure 4] This figure shows an example of an AR image whose display is controlled by the display control unit 130. [Figure 5] This figure shows another example of an AR image whose display is controlled by the display control unit 130. [Figure 6] This flowchart shows an example of the processing flow performed by the information processing device 100. [Figure 7] This figure shows an example of the operating environment and configuration of the information processing device 100 related to a modified example. [Modes for carrying out the invention]

[0016] [overview] Hereinafter, referring to the drawings, the information processing apparatus 100 according to an embodiment of the present invention will be described. The information processing apparatus 100 generates a virtual water surface representing the water surface of the farm field from an image of the farm field captured by a camera, and generates an AR (augmented reality) image by superimposing the generated virtual water surface on the image. Next, the information processing apparatus 100 causes the generated AR image to be displayed on the user's terminal device so that the position of the virtual water surface can be changed, and accepts an operation by the user. The information processing apparatus 100 sets a target water level of the farm field based on the operation received on the virtual water surface. Thereby, the information processing apparatus 100 realizes a more intuitive water level setting for the user. Hereinafter, the details of the information processing apparatus 100 will be described.

[0017] FIG. 1 is a diagram showing an example of the usage environment and configuration of the information processing apparatus 100. The information processing apparatus 100 operates in cooperation with, for example, a camera 10, a water supply device 20, a base station 30, and a terminal device 40.

[0018] The camera 10 is, for example, a digital camera that uses a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is installed near the farm field F such as a paddy field, a pond, or a water conservancy facility, and images the farm field F. The camera 10 may be installed at a position where a part or the whole of the farm field F can be imaged, and has fewer restrictions on the installation location compared to a water level gauge. Further, the camera 10 is installed at a position that facilitates the user's recognition of the water surface of the farm field in order to enhance the convenience of the operation of the terminal device 40 by the user described later. Here, the position that facilitates the user's recognition of the water surface of the farm field is, for example, a position near the farm field F and corresponding to the height of a person's line of sight. By installing the camera 10 at a position corresponding to the height of a person's line of sight, the user can recognize the water surface at an angle that is familiar from everyday life on the image, and the convenience of operating the terminal device 40 can be enhanced. The camera 10 transmits the captured image IM1 of the farm field F to the information processing apparatus 100 via the base station 30 and the network NW.

[0019] The water supply device 20 is a device that can supply irrigation water supplied from, for example, irrigation water facilities (not shown) to the farmland F via a water faucet. The water supply device 20 is provided with, for example, an electric motor that drives the water faucet, a control circuit that controls the electric motor, and a communication circuit that wirelessly communicates with the information processing device 100 via the base station 30. The water supply device 20 operates, for example, a motor and a communication circuit that drive the water faucet by the generated electric power from a solar panel. In the drawing, the water supply device 20 is shown as having a solar panel, but the present invention is not limited to such a configuration, and the water supply device 20 may be driven by a dry battery or wireless power supply.

[0020] The terminal device 40 is, for example, a computer device such as a personal computer, a smartphone, an HDR (Head Mounted Display), or a tablet terminal. The terminal device 40 communicates with the information processing device 100 via the network NW, receives the AR image generated by the AR image generation unit 120 described later, accepts an operation by the user, and transmits the accepted operation content to the information processing device 100. Hereinafter, in the present embodiment, the terminal device 40 will be described as a tablet terminal.

[0021] The information processing device 100 is a server device such as a web server. The information processing device 100 includes, for example, an image acquisition unit 110, an AR image generation unit 120, a display control unit 130, and a target water level setting unit 140. Each of the image acquisition unit 110, the AR image generation unit 120, the display control unit 130, and the target water level setting unit 140 is realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit), or by the cooperation of software and hardware. The program may be stored in advance on a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device equipped with a non-transient storage medium), or it may be stored on a removable storage medium such as a DVD or CD-ROM (a non-transient storage medium) and installed when the storage medium is inserted into a drive device.

[0022] [Information Processing Device] The image acquisition unit 110 acquires an image IM1 of the field F captured by the camera 10 via the base station 30 and the network NW. Figure 2 shows an example of an image IM1 acquired by the image acquisition unit 110. In Figure 2, the image IM1 shows a paddy field as the field F, but the present invention is not limited to such a configuration, and the image IM1 may, for example, show a water facility such as a reservoir.

[0023] The AR image generation unit 120 first generates a virtual water surface representing the water surface of field F as depicted in the image IM1, based on the image IM1 acquired by the image acquisition unit 110. More specifically, the AR image generation unit 120 first identifies the boundary line BL between the water surface and non-water surface in the acquired image IM1. At this time, the AR image generation unit 120 may identify the boundary line BL using any image processing. For example, the AR image generation unit 120 may identify the boundary line BL between the water surface and non-water surface on the lower side (spatially the foreground side) of the image IM1, or it may identify the boundary line BL by inputting the image IM1 into a trained model that has been trained to output the boundary line between the water surface and non-water surface in response to the image input.

[0024] The AR image generation unit 120 sets the identified boundary line BL as the x-axis for defining a two-dimensional plane as a virtual water surface, and sets the straight line VL perpendicular to the boundary line BL as the y-axis. If there are landmarks in the paddy field, such as rice plants, that can be used to identify the y-axis direction, the AR image generation unit 120 may use such landmarks (for example, along the direction in which the rice plants grow) to set the y-axis. However, since distortion is likely to occur in areas far from the center of image IM1, it is preferable to use boundary lines or landmarks located as close to the center of image IM1 as possible when setting the x and y axes using a rule-based method. The AR image generation unit 120 generates a two-dimensional plane defined by the x and y axes as a virtual water surface.

[0025] If no water surface is visible in image IM1, the AR image generation unit 120 may identify a horizontal line from features present around field F and set the x-axis accordingly. For example, if field F is a paddy field, the AR image generation unit 120 may assume the upper end of the levee is the horizontal line and set it as the x-axis, so that a virtual water surface can be projected below the horizontal line. Alternatively, if field F is a paddy field, the AR image generation unit 120 may assume the field surface (the ground area inside the levee) is the horizontal plane and set the virtual water surface in a direction perpendicular to the horizontal line.

[0026] Furthermore, in this embodiment, the AR image generation unit 120 generates a virtual water surface based on the image IM1. However, the present invention is not limited to such a configuration. The water surface may be determined and a virtual water surface generated by sensing wavelengths other than visible light, such as infrared and ultraviolet light, with a sensor, and analyzing the wavelengths emitted from the water body and objects present around the water body.

[0027] The AR image generation unit 120 generates a virtual water surface and then generates AR image IM2 by superimposing the generated virtual water surface onto image IM1. Figure 3 shows an example of AR image IM2 generated by the AR image generation unit 120. In Figure 3, the symbol VS1 represents the virtual water surface defined by the x and y axes set in Figure 2, and is the water surface that serves as the reference for user operations described later.

[0028] When the AR image generation unit 120 generates an AR image IM2, the display control unit 130 transmits the generated AR image IM2 to the terminal device 40 via the network NW, and displays the virtual water surface VS1 on the terminal device 40 so that its position can be changed. Figure 4 shows an example of the AR image IM2 whose display is controlled by the display control unit 130. As shown in Figure 4, for example, a user of the terminal device 40 can visually grasp the current water level of field F by observing the virtual water surface VS1 displayed on the display of the terminal device 40.

[0029] Furthermore, users of the terminal device 40 can specify the target water level of field F as a difference from the current water level, for example, by touching the virtual water level VS1 and then operating the virtual water level VS1 upwards or downwards (vertically) while keeping it touched. For example, users of the terminal device 40 can increase the target water level from the current water level by operating the virtual water level VS1 upwards, or conversely, decrease the target water level from the current water level by operating the virtual water level VS1 downwards. Figure 4 shows the situation where, as a result of the user of the terminal device 40 operating the virtual water level VS1 upwards, a virtual water level VS2 representing a target water level higher than the current water level is displayed. In other words, compared to conventional technology, where the target water level is set by numerical specification by referring to the current water level measured by the water level gauge, users of the terminal device 40 can specify the target water level of field F more intuitively. Furthermore, since the camera is positioned to facilitate the user's recognition of the water surface in the field, the user of the terminal device 40 can more intuitively specify the target water level in the familiar field F environment.

[0030] When a user of terminal device 40 completes the operation of virtual water surface VS1 and specifies virtual water surface VS2, they confirm the specification of virtual water surface VS2, for example, by pressing confirmation button B. Once the specification of virtual water surface VS2 is confirmed, terminal device 40 transmits information about AR image IM2, including the specification of virtual water surface VS2, to information processing device 100.

[0031] Figure 5 shows another example of an AR image controlled by the display control unit 130. Figure 5 shows AR image IM3, which is a cropped portion of AR image IM2 shown in Figure 4. For example, a user of the terminal device 40 can select and trace only the area of ​​interest from AR image IM2 shown in Figure 4 with their finger, and the terminal device 40 will crop the selected area from AR image IM2 and enlarge it to display as AR image IM3. The user of the terminal device 40 can then similarly increase or decrease the target water level by manipulating the virtual water level VS1 of the enlarged AR image IM3 upwards or downwards. For example, in the case of Figure 5, the user of the terminal device 40 can specify the desired water level in more detail while comparing the height of the rice seedlings in field F with the water level.

[0032] When the target water level setting unit 140 obtains information regarding the AR image IM2, including the designation of the virtual water level VS2, from the terminal device 40, it sets the target water level of field F based on the operation to change from the virtual water level VS1 to the virtual water level VS2. More specifically, the target water level setting unit 140 uses arbitrary image processing to identify the water level difference corresponding to the difference between the virtual water level VS1 and the virtual water level VS2, and sets the target water level by adding the identified water level difference to the current water level. The current water level used in this calculation may be, for example, the value of the current water level measured by any type of water level gauge. If a water level gauge is not used, the target water level setting unit 140 may calculate water volume information corresponding to the identified water level difference.

[0033] The target water level setting unit 140 transmits information indicating the set target water level or the amount of water corresponding to the water level difference to the water supply device 20 via the network NW and base station 30 using a transmitter provided in the information processing device 100. Based on the received information, the water supply device 20 supplies water to field F so that the water level in field F reaches the target water level. If the water supply device 20 receives information indicating the amount of water corresponding to the water level difference, the water supply device 20 supplies water to field F by the amount corresponding to that amount. Alternatively, the target water level setting unit 140 may transmit the information indicating the water level difference directly to the water supply device 20, and the water supply device 20 may determine the corresponding amount of water based on the water level difference. The configuration including the information processing device 100, the transmitter, and the water supply device 20 is an example of a "system".

[0034] In the above description, we have explained the case in which the information processing device 100 has all the functions of the image acquisition unit 110, the AR image generation unit 120, the display control unit 130, and the target water level setting unit 140. However, the present invention is not limited to such a configuration, and at least some of the functions of the image acquisition unit 110, the AR image generation unit 120, the display control unit 130, and the target water level setting unit 140 may be implemented as an application program installed on the terminal device 40.

[0035] For example, if terminal device 40 is equipped with an application program that implements the functions of the AR image generation unit 120 and the display control unit 130, terminal device 40 acquires an image IM1 of field F from the information processing device 100, generates an AR image IM2 from the acquired image IM1, and displays the generated AR image IM2 so that the position of the virtual water surface VS1 can be changed. Alternatively, if terminal device 40 is equipped with an application program that implements only the function of the display control unit 130, terminal device 40 acquires an AR image IM2 from the information processing device 100, and displays the acquired AR image IM2 so that the position of the virtual water surface VS1 can be changed as a function of the native application.

[0036] Furthermore, the above description explains a case in which the target water level setting unit 140 transmits information to the water supply device 20 indicating the amount of water corresponding to the set target water level or water level difference, and the water supply device 20 supplies water based on the received information. As another example, if a water level gauge is not used, the water surface of field F may be continuously imaged by the camera 10 while the water supply device 20 is supplying water, and the water supply device 20 may supply or drain water until the imaged water surface matches the virtual water level VS2.

[0037] Next, with reference to Figure 6, the flow of processing performed by the information processing device 100 will be described. Figure 6 is a flowchart showing an example of the flow of processing performed by the information processing device 100.

[0038] First, the image acquisition unit 110 acquires an image IM1 of field F captured by the camera 10 (step S100). Next, the AR image generation unit 120 generates a virtual water surface VS1 representing the water surface of field F as captured in the acquired image IM1, and generates an AR image IM2 by superimposing the generated virtual water surface VS1 onto the image IM1 (step S102). Next, the display control unit 130 displays the generated AR image IM2 on the terminal device 40 so that the position of the virtual water surface VS1 can be changed (step S104).

[0039] Next, the target water level setting unit 140 receives operation information from the user regarding the change from virtual water level VS1 to virtual water level VS2 (step S106). Then, based on the received operation information, the target water level setting unit 140 sets the target water level for field F (step S108). This completes the processing in this flowchart.

[0040] As described above, according to the embodiment, the information processing device 100 generates a virtual water surface representing the water surface of a field or irrigation facility from an image of the field or irrigation facility, generates an AR image by superimposing the generated virtual water surface onto the image, displays the generated AR image on the user's terminal device so that the position of the virtual water surface can be changed, and sets the target water level of the field or irrigation facility based on the operation of the virtual water surface by the user. This makes it possible to set the water level more intuitively for the user.

[0041] [Differentiation] In the above embodiment, an example was described in which the camera 10 is installed near the field F. However, alternatively, the camera 10 may be mounted on an aerial vehicle such as a drone or a satellite. Figure 7 is a diagram showing an example of the usage environment and configuration of the modified information processing device 100. In Figure 7, the drone 10B is equipped with the functions of the camera 10 and flies to image part or all of the field F. In this case, it is preferable that the drone 10B flies at eye level, similar to the camera 10 installed at eye level, to image the field F.

[0042] If the drone 10B images the field F from directly above, it may be difficult to generate a virtual water surface. However, even in such cases, the water surface changes depending on the water level because the levees and reservoir embankments are slopes, so the information processing device 100 can generate a virtual water surface by utilizing the changes in the water surface. However, it is difficult for users to set the water level using the virtual water surface obtained from aerial imaging. Therefore, when the drone 10B images the field F from above, it is preferable to also collect images of the field F taken from a height close to a person's eye level, and to set the virtual water surface using the latter images.

[0043] Furthermore, the embodiments and modifications described above illustrate an example in which the base station 30 relays communication between the information processing device 100 and the camera 10 and water supply device 20. However, the present invention is not limited to such a configuration, and the base station 30 may be excluded from the configuration. In that case, the information processing device 100 and the camera 10 and water supply device 20 will directly transmit and receive data via wireless communication. This allows for flexible setting of the amount of data and the transmission interval.

[0044] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0045] 10 Cameras 20 Water supply equipment 30 base station 40 Terminal devices 100 Information Processing Devices 110 Image acquisition unit 120 AR Image Generation Unit 130 Display Control Unit 140 Target water level setting section NW Network

Claims

1. An acquisition unit that acquires images of the fields and water facilities captured by cameras located near the fields and water facilities, A generation unit generates a virtual water surface representing the water surface of the field or water facility shown in the image, and generates an AR image by superimposing the generated virtual water surface onto the image. A display control unit that displays the AR image on a user terminal device so that the position of the virtual water surface can be changed, The system includes a setting unit that sets a target water level for the field or water facility based on an operation performed on the user terminal device to change the position of the virtual water surface, Information processing device.

2. The acquisition unit acquires images of the field and water facilities captured by the camera located near the field and water facilities and in a position that facilitates the user's recognition of the water surface. The information processing apparatus according to claim 1.

3. The generation unit generates the virtual water surface on a two-dimensional plane defined by the boundary line between the water surface and non-water surface of the field or irrigation facility as depicted in the image, and a reference line perpendicular to the boundary line. The information processing apparatus according to claim 1.

4. The generation unit determines the water surface and generates the virtual water surface by analyzing wavelengths other than visible light detected by sensors located near the field or water facilities, or the wavelengths of light emitted from the water surface. The information processing apparatus according to claim 1.

5. The display control unit causes the AR image to be displayed on the user terminal device so that the position of the virtual water surface can be changed in a direction orthogonal to the two-dimensional plane. The information processing apparatus according to claim 3.

6. The setting unit sets the target water level by increasing or decreasing the water level of the field or irrigation facility according to the amount of operation performed. The information processing apparatus according to claim 1.

7. The setting unit sets the target water level such that when the position of the virtual water surface is changed in the upward direction of the orthogonal directions, it increases the water level of the field or irrigation facility, and when the position of the virtual water surface is changed in the downward direction of the orthogonal directions, it decreases the water level of the field or irrigation facility. The information processing apparatus according to claim 5.

8. An information processing device according to any one of claims 1 to 7, A transmitter that transmits the set target water level to a water supply machine that supplies water to the field or water facilities, The system includes a water supply device that supplies water to the fields and water facilities so that the water level in the fields and water facilities reaches the target water level. system.

9. Computers Images of the fields and water facilities are captured by cameras located near the fields and water facilities. A virtual water surface representing the water surface of the field or irrigation facility shown in the image is generated, and an AR image is generated by superimposing the generated virtual water surface onto the image. The AR image is displayed on the user terminal device so that the position of the virtual water surface can be changed. Based on the operation performed on the user terminal device to change the position of the virtual water surface, the target water level of the field or water facility is set. Information processing methods.

10. On the computer, Images of the fields and water facilities are captured by cameras located near the fields and water facilities. A virtual water surface representing the water surface of the field or water facility shown in the image is generated, and an AR image is generated by superimposing the generated virtual water surface onto the image. The AR image is displayed on the user terminal device so that the position of the virtual water surface can be changed. Based on the operation performed on the user terminal device to change the position of the virtual water surface, the target water level of the field or water facility is set. program.

Images