System, program and system control method
Patent Information
- Application Number
- JP2019228800
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2019-12-19
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2039-12-19
AI Technical Summary
【0007】 本発明によれば農産物の生育状況等に関する有益な情報を提供できるシステムを提供できる。
Smart Images

Figure 0007578396000001 
Figure 0007578396000002 
Figure 0007578396000003
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device suitable for supporting agricultural work. [Background technology]
[0002] In conventional agriculture, the growth status of agricultural crops has traditionally been judged based on the experience, knowledge, and intuition of farmers. However, in recent years, systems that use IoT (Internet of Things) technology to obtain information from environmental sensors and image data to judge the growth status have been developed. However, even in such systems, it is ultimately the farmers who make the overall judgments about the image data collected, and the accumulated data is not fully utilized or shared as useful data. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2019-165655 A Summary of the Invention [Problem to be solved by the invention]
[0004] The above-mentioned Patent Document 1 describes an environmental information sensor that is installed in a cultivation facility where agricultural products are grown and acquires environmental information indicating the environment in the cultivation facility, and an imaging device that is installed in the cultivation facility and captures an image of the agricultural products.The Patent Document 1 also describes a determination device that determines a remaining integrated value, which is an integrated value of a predetermined environmental value until the appropriate harvest of the agricultural products, based on the environmental information and image information that is information obtained from the image of the agricultural products. However, the technology of Patent Document 1 only determines the harvest time and does not provide detailed information for optimal growth of agricultural crops during their growth. Therefore, even if the growth conditions change due to weather conditions, etc., no consideration is given to what measures should be taken to deal with such changes.
[0005] The present invention can provide useful information regarding the growth status of agricultural products, etc., in response to the above-mentioned problems of the conventional technology. system of provide The purpose is to: [Means for solving the problem]
[0006] In order to achieve the above object, the system of the present invention comprises: An image acquisition means for acquiring an image of a specific crop in a farm field; a generation means for generating a plurality of growth index data of the agricultural crop based on the image of the agricultural crop acquired by the image acquisition means; A storage means for storing a plurality of growth index data for a plurality of dates and times generated by the generation means; A control means for controlling display of a transition of a plurality of growth index data related to the specific crop during a period including a plurality of growth stages based on the plurality of growth index data for the plurality of dates and times stored in the storage means; a sensor device for acquiring environmental information data relating to an environment of the farm field; a recording control means for controlling recording of information based on the data measured by the sensor device in a header area of the image so as to associate the image with the information based on the measured data; and an instruction means for instructing to perform image analysis for obtaining growth index data of the agricultural crop, the image acquired by the image acquisition means being an analysis target. The recording control means controls so as to further record the growth index data based on the image analysis in a header area of the image. The present invention is characterized by the above. Effect of the Invention
[0007] According to the present invention, useful information on the growth status of agricultural products can be provided. system of offer can. [Brief description of the drawings]
[0008] [Figure 1] 1 is a diagram showing the overall configuration of a system using an information processing device according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a functional block diagram of an image analysis cloud server. [Diagram 3] 13 is a flowchart showing an operation flow centered around the control server. [Figure 4] FIG. 11 is a diagram showing an example of a display screen based on display data generated in an application server. [Diagram 5] FIG. 13 is a diagram showing another example of the display screen. [Figure 6] FIG. 13 is a diagram showing yet another example of the display screen. [Figure 7] FIG. 13 illustrates an example of a flowchart for association. [Figure 8] FIG. 13 is a continuation of the example flowchart for association. [Figure 9] FIG. 11 is a diagram showing an example of an associated table. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In each drawing, the same members or elements are given the same reference numerals, and duplicated descriptions will be omitted or simplified.
[0010] FIG. 1 is a diagram showing the overall configuration of a system using an information processing device according to an embodiment of the present invention. The overall configuration of a system using an information processing device according to an embodiment of the present invention will be described below with reference to FIG. A system using an information processing device in this embodiment includes a network camera 101, a plurality of sensor devices 102, a network (public line) 103, a control server 104, a data accumulation cloud server 105, and an application server 106.
[0011] It also includes a service provision management server 107, a billing settlement server 108, an information terminal 109 such as a tablet of a service subscriber, an image analysis cloud server 110, a model cloud server 111 that learns growth conditions, a weather information server 112, and the like. Although the embodiments are described using examples of a server and a cloud server, the server does not have to take the form of a server or a cloud server, and may be, for example, an electronic device having functions similar to those of each server or cloud server.
[0012] In the present embodiment, the information processing device is a device that includes at least the functions of the control server 104, and may also include some or all of the functions of the other servers 105 to 108, 110, 111, and so on. The network camera 101 may be a fixed camera or a camera mounted on a drone or the like, and has a network interface for sending captured images to a control server 104 or the like via a wired or wireless network 103.
[0013] The network camera 101 also receives control signals such as shooting instructions from a control server 104 via a network 103, and performs ON / OFF control, shooting operations, and sending video of the shot images to the control server based on the control signals. Here, the network camera 101 functions as an image acquisition means for acquiring images of specific agricultural products in the field. The image acquisition means includes, for example, a reading device for acquiring images by reading images of specific agricultural products recorded in advance on a recording medium.
[0014] The control server 104 also has a built-in CPU as a computer, and functions as a control means that controls the operation of the network cameras, sensor devices, and other devices in the system based on computer programs stored in the memory. In this embodiment, if the area of the agricultural field is large, a plurality of network cameras 101 and sensor devices 102 may be installed.
[0015] In addition, in this embodiment, the example of cultivating rice as the target crop (crop) is described, but the present invention can be applied to other plants of the Gramineae family as well. Gramineae plants (crops) include, for example, rice, barley (wheat, barley), millet, foxtail millet, barnyard millet, corn, etc. In addition to Gramineae plants, the present embodiment can also be applied to outdoor-grown spinach, etc. As will be described later, at least one network camera 101 is configured to capture images of crops at a given position in a field from above, as shown as the state of the field in FIG.
[0016] The network camera 101 may be a camera mounted on a drone, and a drone may be used to photograph crops at a plurality of predetermined positions (sample positions) in a farm field from above. Also, the grass height may be obtained by measuring the distance based on the left and right images photographed from above using a stereo camera or the like. Also, a camera capable of measuring the distance, such as a stereo camera, may be mounted on a drone or a mobile robot to photograph and measure the distance from above a fixed point.
[0017] Although the network 103 may be a wired LAN or a wireless LAN, in this embodiment, the network 103 is assumed to be a wireless LAN. The network 103 may be a mobile communication network provided by a telecommunications carrier. In that case, the network camera 101 can connect to a public line network by inserting a SIM card into a card adapter (not shown) in the main body of the network camera 101.
[0018] The multiple sensor devices 102 are connected to a control server 104 via a network 103, and transmit sensor data to the control server 104 in response to a sensor data acquisition request from the control server 104 for each sensor device. Alternatively, each sensor device may be configured to transmit sensor data to the control server 104 at different timings and at predetermined transmission intervals in accordance with a Low Power Wide Area (LPWA) communication standard or the like.
[0019] In this case, the control server 104 can select sensor data sent at a desired timing from among data sent from the sensor devices at a predetermined cycle. The sensor device 102 of this embodiment includes multiple types of sensor devices, and functions as an environmental information acquisition means for acquiring environmental information data related to the field environment. Specifically, for example, it is configured to be able to acquire data related to the latitude and longitude (and altitude) of the field. It further includes a sensor device for acquiring environmental information data related to the soil environment of the field.
[0020] The sensor device for acquiring data regarding the latitude and longitude (and altitude) of the field includes, for example, a GPS sensor, and this sensor device may be placed at multiple locations in the field or may be placed inside the network camera 101. Each network camera 101 has a camera ID (camera identification information) as unique identification information. The camera ID is displayed on the outside of the housing of the network camera 101 as characters or a QR code (registered trademark), or the like.
[0021] Similarly, each sensor device 102 has a sensor ID (sensor identification information) as unique identification information. In this embodiment, the sensor ID is displayed on the outside of the housing of the sensor device 102 as characters or a QR code (registered trademark), etc. It is desirable to link (associate) or pair the network camera 101 and the sensor device 102 in advance.
[0022] As a specific method, for example, a sensor ID consisting of characters or a QR code (registered trademark) displayed on the outside of the housing of the sensor device 102 is photographed by the network camera 101. This makes it possible to perform image recognition of the sensor ID of the photographed sensor device 102 and associate it with the camera ID of the network camera 101 that photographed it. At this time, for example, the sensor ID of the recognized sensor device 102 is written in the header area of the image captured by the network camera 101 .
[0023] This makes it possible to associate the sensor ID of the sensor device 102 with the camera ID of the network camera 101 when sending an image captured by the network camera 101 to the control server 104. Here, the means for writing to the header area of such an image file functions as an association means. Alternatively, the network camera 101, the sensor device 102, and the control server 104 may have application software in advance. Then, the user may register the sensor ID of the sensor device 102 and the camera ID of the network camera 101 in association with each other on the application software screen.
[0024] In that case, the camera ID and the sensor ID do not need to be displayed on the outside of each housing. In this case, the application software in which the user sets the correspondence between the camera identification information and the sensor identification information functions as the correspondence unit. Alternatively, the camera ID of the network camera 101 and the sensor ID of the sensor device 102 may be photographed simultaneously or sequentially by a separate camera provided on a smartphone or the like.
[0025] Then, by transmitting the image to the control server 104, it is possible to register the camera ID of the network camera 101 and the sensor ID of the sensor device 102 in association with each other. The flows in Figures 7 and 8 are detailed flowcharts of such a pairing (association, linking) method, which will be described later.
[0026] As a method of associating the camera ID with the sensor ID, instead of photographing a character ID or a QR code (registered trademark) displayed on the outside of the housing and performing image recognition as described above, pairing may be performed using, for example, Bluetooth (registered trademark) or NFC. By performing pairing in advance, for example, images taken by a specific camera among multiple network cameras can be linked (associated) with sensor data from a pre-associated sensor device and managed using a table on the control server.
[0027] Figures 7 and 8 are flow charts showing an example of a pairing method, and Figure 9 is a diagram showing an example of a table showing such association. Figures 7 to 9 will be described later. Then, for example, the sensor data of the associated sensor device can be efficiently recorded in the header area of the image file of an image captured by a specific network camera 101. Therefore, when there are a large number of network cameras 101 and sensor devices 102, the image data and the sensor data can be efficiently associated with each other.
[0028] In addition, the sensor data of the sensor device is written in a header area separate from that of the same image file, which is associated with the sensor device by writing the ID of the corresponding sensor device in part of the header area of the image file. The environmental information data regarding the soil environment of the field measured by the sensor device 102 includes data regarding at least one of the following: soil color, soil moisture content, the amount or ratio of several specific chemical substances (e.g., nitrogen, phosphate, potassium, etc.) in the soil, and the pH value of the soil.
[0029] The sensor device 102 may also be configured to acquire some data related to meteorological information of the field. That is, the sensor device 102 may be configured to detect (measure) data related to at least one of the following: altitude of the field, amount of precipitation, rainfall, amount of snowfall, temperature, humidity, water level of a field such as a paddy field, water temperature of a field such as a paddy field, illuminance, hours of sunshine, wind speed, and air pressure. The data storage cloud server 105 is a cloud server for storing data based on instructions from the control server 104.
[0030] Then, image data (image files) from the network camera 101 and sensor data from the sensor device 102 are accumulated (stored) in a state where they are linked to data on the date and time of their acquisition. Note that here, linking includes linking (associating) by writing (storing) various data in the header area of the image file. Note that the date and time data may be acquired, for example, from a CPU or the like in the network camera 101 or from the sensor device 102. Alternatively, the date and time data may be acquired from a CPU or the like in the data accumulation cloud server 105. The data accumulation cloud server 105 also acquires weather data for the date and time when the image was captured from the weather information server 112.
[0031] Based on instructions from the control server 104 , the data accumulation cloud server 105 sends captured images and sensor data from the network camera 101 and weather information data from the weather information server 112 to the image analysis cloud server 110 . The image analysis cloud server 110 performs image analysis on the images from the network camera 101 based on instructions from the control server 104 .
[0032] The model cloud server 111 statistically learns the relationship between the growth status and past images, growth index data, environmental information data (sensor data, weather data, etc.), dates, etc. of agricultural crops, and then generates and stores growth model data that associates the images, growth index data, environmental information data, dates, etc. with the growth status through learning. The image analysis cloud server 110 acquires growth model data linked to past growth index data and environmental information data from the model cloud server 111.
[0033] The image analysis cloud server 110 then analyzes the images of the crops obtained by the network camera 101, and generates, as the analysis results, growth index data of the crops. Furthermore, the image analysis cloud server 110 generates growth index data of the crops by comparing / referring to the growth index data, environmental information data (sensor data, meteorological data, etc.), and date and time data with the growth model data. That is, the image analysis cloud server 110 functions as an index data generating means for generating growth index data. In particular, the image analysis cloud server 110 functions as a (growth stage) data generating means for generating information on each growth stage (e.g., the tillering stage, etc.).
[0034] The growth index data (analysis results) generated by the image analysis cloud server 110 is sent to the data accumulation cloud server 105 and accumulated in a linked (associated) manner with the images, sensor data, weather data, and date and time. At that time, the growth index data, sensor data, weather data, date and time, etc. are written and accumulated (stored) in a predetermined header area of each image file of the agricultural crops, for example, in a header area of the image file defined by the EXIF (Exchangeable Image File Format) standard.
[0035] Specifically, it is desirable to write the data in a Maker Note area in the header area of the EXIF standard, which has a data type of UNDEFINED and no regulations regarding data structure or size. Then, the growth index data, sensor data, weather data, date and time, etc. can be assigned to be written in their respective header areas. In addition, when communication is performed using a separate data platform rather than the header area of the image file, the API format defined in the WAGRI standard may be used. Here, API stands for Application Programming Interface. This may enable the growth index data, sensor data, and weather data to be linked and shared, provided, and otherwise mutually used.
[0036] Here, WAGRI is the name of the Agricultural Data Collaboration Platform. Below, we will describe an example of using the header area of an image file. The data storage cloud server 105 functions as a storage means that stores image data, growth index data, environmental information data, meteorological information, etc. for multiple dates and times generated by the index data generation means in association with each date and time data.
[0037] The photographed image data of agricultural crops at multiple dates and times, growth index data, environmental information data, and information on the dates and times of their acquisition, etc., which are temporarily stored in the data storage cloud server 105, are sent to an application server 106. The application server 106 generates display data for displaying information on the growth process (growth stages) in a graph based on the growth index data at multiple dates and times and the environmental information data at multiple dates and times. The application server 106 functions as a display data generating means or a growth stage data generating means for this purpose.
[0038] In response to a weather information acquisition request from the control server 104, the weather information server 112 sends weather information to the data accumulation cloud server 105 as environmental information data. The weather information includes, for example, data on at least one of the weather, precipitation, rainfall, snowfall, temperature, humidity, sunshine hours, wind speed, and air pressure. As described above, some of the weather information, etc. can also be acquired from the sensor device 102. The weather information server 112 also functions as part of the environmental information acquisition means for acquiring environmental information data on the environment (weather) of the field.
[0039] A service provision management server 107 transmits information on what display screens, data, and services are to be provided to the contractor (contract user) based on billing and settlement information from a billing and settlement server 108 to the application server 106. It also transmits information on the contents of billing services to the billing and settlement server 108. The billing settlement server 108 communicates with the contractor's information terminal 109, obtains settlement information for the bill to the contractor from the contractor's information terminal 109, and sends information such as whether or not the settlement has been completed to the contractor's information terminal 109.
[0040] Here, the service provision management server 107, the billing settlement server 108, and the application server 106 function as a billing settlement means for making it possible to provide the accumulated growth index data and environmental information data to a specified terminal in accordance with a billing charge. It also functions as a billing and settlement means that enables data relating to the growth process generated based on growth index data for a plurality of dates and times and the environmental information data for the plurality of dates and times to be provided to the specified terminal in accordance with a charge.
[0041] When payment information is sent from the contractor's information terminal 109 to the billing and settlement server 108, the payment information is sent to the service provision and management server 107. Then, based on the payment information (whether or not payment has been completed, etc.), screen, data, and service information are sent to the application server 106.
[0042] The application server 106 responds by displaying the growth status of the information terminal 109 of the customer. The display data for graphing and displaying the growth process is sent to the contractor, and a display screen in which the growth process is graphed can be displayed on the contractor's information terminal 109. It goes without saying that the cloud servers 105 to 108, 110 to 112 and the information terminal 109 also have computers. FIG. 2 is a functional block diagram of the image analysis cloud server 110, and the functions of the image analysis cloud server 110 will be described with reference to FIG.
[0043] Reference numeral 1101 denotes an input unit that acquires captured image data, various sensor data, meteorological data, and date and time data of the acquisition of each data from the data accumulation cloud server 105 . A leaf color calculation unit 1102 calculates the color of the leaves of the crop. The color of the ears may also be calculated. A stalk number calculation unit 1103 calculates the number of stalks of the crop.
[0044] Leaf color may be expressed in terms of a unit called the Solid and Plant Analyzer Development (SPAD) value, which is said to be correlated with chlorophyll content within a certain range under certain conditions. In this embodiment, to calculate the number of stems in a field from an image, the leaf tips are treated as points, and the leaf tips (leaf tips) are learned and recognized by AI (artificial intelligence), the number of leaf tips (leaf tips) is counted, and the number of stems is calculated based on the results.
[0045] This is because a predetermined correlation can be found between the number of leaf tips and the number of stalks from past statistical data (learning data). For example, statistically, the number of stalks can be roughly calculated as about 1 / 3 of the number of leaf tips. 1104 is a grass height calculation unit that calculates the grass height (maximum height of the crop from the ground). For example, by placing one of the network cameras 101 to the side of the crop and taking an image together with a reference scale installed next to the crop, the grass height (maximum height of the crop from the ground) can be calculated.
[0046] In this embodiment, the data on the leaf color, the number of stems, and the plant height are used as the main growth index data for the crops.Further growth index data may include, for example, the vegetation coverage rate per unit area. In the case of a grass crop, data on at least one of the following may be included: the ratio of stalks with ears to the number of stalks, the ratio of ears that have turned yellow, the number of ears, the number of grains per ear, the inclination of the stalk, and the degree of curvature of the stalk. By increasing the variety of such growth index data, the accuracy of graphing and displaying the growth process can be improved.
[0047] In addition, the condition and maturity calculated from the swelling and color of the rice grains can be included in the growth index data. This makes it possible to show the percentage of whole grains. Here, the percentage of whole grains is the percentage of rice grains that are properly shaped, that is, that exist in a certain amount of brown rice, expressed as a percentage. Growth index data from a leaf color calculation unit 1102 , a stalk number calculation unit 1103 , a plant height calculation unit 1104 , etc., as well as sensor data, meteorological data, date and time data, etc. from an input unit 1101 are supplied to a growth stage determination unit 1105 .
[0048] At the same time, the data is also supplied to and stored in the model cloud server 111, which learns the growth status, and is used as learning data. The growth stage determination unit 1105 compares growth index data such as leaf color, number of stalks, plant height, sensor data, meteorological information data, date and time data, etc. with the growth model data in the model cloud server 111. This determines the growth stage (growth process) at the time the crop image was captured. Furthermore, by referring to the growth model data in the model cloud server 111, the predicted value of the harvest volume per unit area and the desired planned harvest date are also calculated.
[0049] Then, growth index data such as leaf color, number of stems, plant height, growth stage data, planned harvest date, harvest volume per unit area, etc. are output as analysis results to the data accumulation cloud server 105 via the output unit 1106. FIG. 4 is a diagram showing an example of a display screen based on the display data generated in the application server 106. As shown in FIG.
[0050] The application server 106 generates display data for graphing and displaying the growth process based on the growth index data for a plurality of dates and times and the environmental information data for the plurality of dates and times stored in the data storage cloud server 105. Furthermore, display data may be generated for displaying at least one of an image of the farm field or an image of the farm crop at a predetermined date and time together with the display of the growth process.
[0051] In addition, together with the display of the growth process, display data for displaying at least one of the expected harvest time and the expected yield per unit area may be generated. In addition, together with the display data of the growth process, display data for displaying information on the growth stage of the crop (for example, the tillering stage, etc.) may be generated. These display data are displayed, for example, as a graph as shown in Fig. 4 on the information terminal 109. The display contents of Fig. 4 will be described later.
[0052] Next, an operation flow centered on the control server 104 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an operation flow centered on the control server. In the system shown in Fig. 1, the flow in Fig. 3 starts by starting the control server 104 etc. In step S301, images of the farm field are taken at a predetermined cycle using the network camera 101. The predetermined cycle is set in advance in the control server 104 via an application on the information terminal 109, for example, to "take every morning at 10:00".
[0053] In step S302, a data acquisition request is sent to the sensor device 102. In step S303, a captured image and sensor data are acquired as a result of steps S301 and S302. In step S304, the capture date and time and the sensor data are written as metadata in a predetermined header area of the captured image data. In step S305, the captured image data in which the data has been written in the header is sent to the data accumulation cloud server 105, and the weather information data from the weather information server 112 is written in another header area of the captured image data.
[0054] In step S306, the image analysis cloud server 110 is instructed to transmit the image data in which various data has been written in the header, and to perform image analysis. In step S307, the image analysis cloud server 110 performs image analysis by comparing / referring to the growth model of the model cloud server 111. In step S308, the growth index data resulting from the analysis by the image analysis cloud server 110 is further added to the header area of the image, and is sent to the data accumulation cloud server 105.
[0055] As a result, the image file is stored in the data storage cloud server 105 with the date and time, environmental data (sensor data), and growth index data written in each header area. In step S309, it is determined whether or not a payment has been made from a specific contracted user, and if a payment has been made, the process proceeds to step S310. If no payment has been made, the process returns to step S301 and data storage continues. In step S310, an instruction is issued to the application server 106 to provide images for a specific period of time to contracted users who have paid the payment.
[0056] At this time, the image is provided with some or all of the above-mentioned data written in the header area. Depending on the amount of charge, some of the data written in the header area may be masked before being provided. In step S311, it is determined whether or not an option fee is to be paid as a charge. If the answer is No, the process proceeds to step S313, and if the answer is Yes, the process proceeds to step S312.
[0057] In step S312, the application server 106 generates display data for displaying the growth process, and provides the display data for displaying the growth process together with the images for the predetermined period to the contracted user. In step S313, the billing settlement server 108 stores the contract user's ID (identification number), payment history, and history regarding the provided data, etc.
[0058] 1 is turned off, and if it is not turned off, the process returns to step S301 and data accumulation is repeated. If it is determined in step S314 that the system is turned off, the flow ends. In this way, the system operator can provide useful accumulated growth model data to users who need it, and conversely, users can obtain useful accumulated growth model data by paying a fee.
[0059] Next, a display example of the growth process generated by the application server 106 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of a display screen based on display data generated by the application server. Figs. 5 and 6 are diagrams showing other examples of the display screen. The application server 106 acquires data required for the display screen from the data storage cloud server 105 .
[0060] 4 to 6, 400 is the display screen of the information terminal 109, 401 is a graph showing the growth process, and 402 is a display area showing the current date and time. 403 is a display area showing the name of the billing contractor (contract user name and user ID) and the like. Here, the billing contractor refers to a contractor who has made a contract to provide accumulated growth index data and environmental information data, and further growth process data, etc. to a specified terminal in response to a fee.
[0061] Reference numeral 404 denotes a display area displaying the type of agricultural crop, 405 denotes a scale of the number of stalks displayed on the vertical axis, 406 denotes a scale of the plant height displayed on the vertical axis, and 407 denotes a scale of the time axis in months displayed on the horizontal axis. Reference numeral 408 denotes data showing an outline of the rice growth process (growth stage), and 409 denotes a display showing examples of farm work recommended to be performed according to the growth stage. Reference numeral 420 denotes a scale of the SPAD value displayed on the vertical axis, and 421 denotes a scale of the leaf color before conversion to the SPAD value.
[0062] 410a is a graph showing the growth process of the plant height, and 410b is a graph showing a growth model of the plant height. 411a is a graph showing the growth process of the number of stems, and 411b is a graph showing a growth model of the number of stems. 412a is a graph showing the process of the SPAD value with growth, and 412b is a graph showing a growth model of the SPAD value. 422 displays an explanation of the growth stage (e.g., tillering stage) according to the growth process of the number of stems.
[0063] 410a, 411a, and 412a show graphs of values calculated from the currently captured image. Also, 410b, 411b, and 412b show graphs (growth models) that are modeled by accumulating past actual measurements and past calculated values and performing statistical processing. These graphs make it possible to easily compare and determine whether the values calculated from the captured image are in line with the modeled graphs (the difference is small) or are out of line.
[0064] Figures 5 and 6 are diagrams showing examples of display screens that can be switched from the graph display in Figure 4 by menu selection or the like. Reference numeral 413 in Figures 5 and 6 is a guide that highlights agricultural work that is currently recommended. The display example in Figure 5 displays "It is currently in the tillering stage. Please apply additional fertilizer," and the display example in Figure 6 displays "The current whole grain rate is 75%. Please harvest."
[0065] Reference numeral 414 denotes a display area for displaying the current growth stage, and 415 denotes a display area for displaying three main growth index data, namely leaf color, number of stems, and plant height, in numerical form. In this example, only three growth index data are displayed, but other growth index data may be displayed using, for example, a pull-down menu. In this embodiment, for example, the user can modify these numbers as appropriate. Reference numeral 416 denotes a display area for displaying the environmental information data in numerical form.
[0066] In this embodiment, memory, humidity, water level of the field, water temperature of the field, pH of the soil of the field, illuminance, wind speed, etc. are displayed as environmental information data, but other environmental information data may be displayed, for example, by using a pull-down menu or the like. 417 is a button for switching to a mode for correcting the numerical values displayed on the screen, and clicking or touching this button switches to a correction mode for correcting the numerical values displayed on the screen. Here, button 417 functions as a correction means for correcting part of the display data generated by the display data generating means through user input.
[0067] In the correction mode, the numerical values displayed in areas 415 and 416 can be changed. After making a change, clicking or touching button 417 again switches from the correction mode to the normal mode. 418 denotes an image of the farm field or an image of the crops displayed together with the graph, and at least one of the images can be displayed. 419 denotes a movable marker line, which is displayed at the current date and time position on the graph by default, but can also be moved by mouse or touch.
[0068] For example, if the stem is bent or fallen due to strong winds or weather conditions, the appropriate grass height may not be obtained from the captured image. In this correction mode, it is possible to re-enter values measured manually or values measured again after correcting the posture and taking a photograph. For environmental information, if the environmental information sensor fails, it is possible to input remeasurements or values from a normal sensor and then correct the information later.
[0069] In this way, if erroneous data is acquired as a result of image recognition, the user can correct the data, and the growth process graphs, etc. can be automatically corrected accordingly, thereby preventing the accuracy of the growth process graphs, etc. from decreasing. In Fig. 4, when marker line 419 is moved, the date display in area 402 changes accordingly. Furthermore, when the display screen is switched as shown in Fig. 5 or Fig. 6 by selecting a menu after marker line 419 is moved, the display contents in display areas 414 to 416 and the image in 418 also change to match the date and time.
[0070] In the above example, the graph of the growth process in Figure 4 and the images of the field and the display of each index value in Figures 5 and 6 are displayed on separate screens, but Figures 4 and 5, or Figures 4 and 6, may also be displayed on the same screen. Alternatively, for example, at least one of growth index data and environmental information data at a specific date and time may be displayed together with the type of agricultural crop along with a graph of the growth process.
[0071] In this way, in this embodiment, the contracted user can quickly understand the process of crop growth through graphs based on the crop images and environmental information data acquired from the field, etc. Even if he or she does not go to the field or is a beginner in farming, he or she can quickly and easily understand what farm work should be done at the current time. Next, FIG. 7 and FIG. 8 are flow charts showing an example of the above-mentioned pairing method, and FIG. 9 is a diagram showing an example of a table that makes such associations.
[0072] 7, in step S700, the information terminal 109 is powered on, and in step S701, an application for performing association is started. In step S702, a user ID is registered, in step S703, GPS location information is acquired, and in step S704, QR codes (registered trademark) of multiple sensor devices 102 are photographed with, for example, an attached camera or a smartphone camera.
[0073] In step S705, the IDs of the captured sensor devices 102 are registered and displayed on the screen. In step S706, the QR code (registered trademark) of the network camera 101 is photographed with an attached camera, a smartphone camera, or the like, and in step S707, the ID of the network camera 101 is registered and displayed on the screen. In step S708, the GPS location information acquired by the information terminal 109, the IDs of the sensor devices 102, and the ID of the network camera 101 are sent together to the control server 104.
[0074] In step S709, the initial setting operation is ended, and the process proceeds to step S710 in Fig. 8. In step S710, the control server 104 stores a table of the registered user ID, the GPS location information acquired by the information terminal 109, the ID of the network camera 101, and the IDs of the multiple sensor devices 102. Fig. 9(A) is a diagram showing an example of such an association table. Fig. 9(A) shows an example of a table that associates user IDs, information terminal IDs, GPS location information of sensor devices, sensor device IDs, and network camera IDs.
[0075] In this table example, only the user ID is different, and the other information is the same. Here, each user ID is subscribed to a different service plan, as shown in FIG. 9(B). In FIG. 9(B), user ID (0001) is subscribed to a plan that only monitors images of the farm field, and user ID (0002) is subscribed to a plan that allows the user to download all accumulated growth data at once. Here, a table showing the correspondence between user IDs and service plans, such as that shown in FIG. 9(B), is stored, for example, in the service provision management server 107.
[0076] After step S710, the process proceeds to step S711, where the flow shown in the flowchart of Fig. 3 is executed. In step S712, the application server 106 detects that the user has logged in to the application server, and in step S713, based on the login information, queries the service provision management server 107 about service information that the user can receive. Then, in step S714, if an instruction to provide a service is received from the control server 104 during the processing of step S711 by the control server 104, the process of step S714 is executed.
[0077] That is, in step S714, data for generating display data corresponding to the service information received as a result of the inquiry in step S713 is requested and obtained from the data accumulation cloud server 105. In step S715, the necessary data is obtained from the data accumulation cloud server 105, and in step S716, screen data for display is generated based on the obtained data.
[0078] The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to the above embodiments. Various modifications are possible based on the gist of the present invention, and these modifications are not excluded from the scope of the present invention. For example, in the above embodiment, environmental information data is accumulated in order to obtain growth process data with high accuracy, but in a simplified system, it is not necessary to accumulate environmental information data.
[0079] In addition, when graphing and displaying the growth process, display data for graphing and displaying the growth process may be generated based on growth index data (leaf color, number of stems, plant height, etc.) acquired from an image without taking into account the environmental information data. Moreover, in this case, display data for displaying the growth process may be generated using only the leaf color, number of stems, and plant height as growth index data.
[0080] In addition, in the embodiment, the growth process data is generated by referring to the model data learned in the model cloud server, but it is also possible to simply generate the growth process data using a function (table) or the like prepared in advance. In the embodiment, the process is divided and executed by multiple servers, but for example, the control server 104 may incorporate some or all of the functions of the data storage cloud server 105, application server 106, image analysis cloud server 110, model cloud server 111, etc.
[0081] Furthermore, it may also incorporate the functions of the service provision management server 107 and the billing settlement server 108. Alternatively, two or more of these seven servers 104 to 108, 110, 111, etc. may be appropriately integrated to reduce the number of servers. Furthermore, the subject to be photographed by the camera is not limited to agricultural crops, but may be any living thing, including humans.
[0082] A computer program for implementing all or part of the control in this embodiment may be provided to a computer such as each server constituting the information processing device via a network or various storage media. The computer (or CPU, MPU, etc.) in the server may then read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]
[0083] 101 Network Camera 102 Sensor Device 103 Network 104 Control Server 105 Data storage cloud server 106 Application Server 109 Information terminal 110 Image analysis cloud server
Claims
1. An image acquisition means for acquiring an image of a specific crop in a farm field; a generation means for generating a plurality of growth index data of the agricultural crop based on the image of the agricultural crop acquired by the image acquisition means; A storage means for storing a plurality of growth index data for a plurality of dates and times generated by the generation means; A control means for controlling display of a transition of a plurality of growth index data related to the specific crop during a period including a plurality of growth stages based on the plurality of growth index data for the plurality of dates and times stored in the storage means; a sensor device for acquiring environmental information data relating to an environment of the farm field; a recording control means for controlling recording of information based on the data measured by the sensor device in a header area of the image so as to associate the image with the information based on the measured data; and an instruction means for instructing to perform image analysis for obtaining growth index data of the agricultural crop, the image acquired by the image acquisition means being an analysis target. The system is characterized in that the recording control means controls so that the growth index data based on the image analysis is further recorded in a header area of the image.
2. 2. The system according to claim 1, wherein said control means controls so that a plurality of graphs each showing a transition of a plurality of growth index data during said period are displayed in correspondence with the same time axis.
3. The system described in claim 1 or 2, characterized in that the control means controls so as to display both the trends in multiple growth index data for the specific agricultural crop during the period and a model of the multiple growth index data.
4. An image acquisition means for acquiring an image of a specific crop in a farm field; a generating means for generating growth index data of the agricultural crop based on the image of the agricultural crop acquired by the image acquiring means; A storage means for storing the growth index data for the plurality of dates and times generated by the generation means; A control means for controlling displaying both a transition of the growth index data for the specific crop during a period including a plurality of growth stages and a model based on past statistics based on the growth index data for the plurality of dates and times stored in the storage means; a sensor device for acquiring environmental information data relating to an environment of the farm field; a recording control means for controlling recording of information based on the data measured by the sensor device in a header area of the image so as to associate the image with the information based on the measured data; and an instruction means for instructing to perform image analysis for obtaining growth index data of the agricultural crop, the image acquired by the image acquisition means being an analysis target. The system is characterized in that the recording control means controls so that the growth index data based on the image analysis is further recorded in a header area of the image.
5. 5. The system according to claim 4, wherein said control means controls so as to display a graph showing the transition of said growth index data during said period and a graph of said model for comparison.
6. The method further includes: acquiring environmental information data relating to an environment of the farm field; The storage means further stores the environmental information data for the plurality of dates and times acquired by the environmental information acquisition means, The system according to any one of claims 1 to 5, characterized in that the control means further performs control to display the transition of the growth index data based on the environmental information data for the multiple dates and times.
7. 7. The system according to claim 6, wherein the environmental information acquisition means acquires environmental information data based on measurement data measured by a sensor device that measures an environment of the farm field.
8. 8. The system according to claim 1, wherein said storage means also stores images of the crops taken at the plurality of dates and times.
9. The system described in any one of claims 1 to 8, characterized in that the control means further controls to display at least one of the growth index data and environmental information data regarding the field environment at a specific date and time included in the period.
10. The system according to claim 9, characterized in that the control means controls to display at least one of an image of the field or an image of the crop at the specific date and time together with at least one of the growth index data and the environmental information data.
11. The specific crop is a grass family plant, The growth index data includes data on the number of stalks of the agricultural crop, The system described in any one of claims 1 to 10, characterized in that the generation means counts the number of leaf tips of the crop based on the image of the crop acquired by the image acquisition means, and identifies the number of stems of the crop based on the counted number of leaf tips.
12. The system according to claim 11 , wherein the growth index data includes data on the number of stems, plant height, and leaf color of the crop.
13. The system according to any one of claims 1 to 12, characterized in that the growth index data includes data relating to at least one of the following: vegetation coverage rate per unit area, the ratio of stalks with ears to the number of stalks, the ratio of ears that have turned yellow, the number of ears, the number of grains per ear, the inclination of the stem, and the degree of curvature of the stem.
14. The system according to any one of claims 1 to 13, characterized in that the control means further controls so as to display at least one of a forecast of harvest time and a forecast of harvest amount per unit area.
15. A system described in any one of claims 1 to 14, characterized in that the generation means generates growth index data for the crop by comparing growth index data for multiple dates and times stored in the storage means and environmental information data regarding the field environment with growth model data.
16. The system described in any one of claims 1 to 15, characterized in that it has a processing means for processing the growth index data stored in the storage means so as to provide it to a specified terminal depending on the billing status.
17. The system according to any one of claims 6, 7, 9, 10 and 15, characterized in that the environmental information data includes data relating to at least one of the following items related to the field: soil color, soil moisture content, proportions of a number of predetermined chemical substances in the soil, soil pH value, meteorological information, latitude and longitude, altitude, weather, precipitation, rainfall, snowfall, temperature, humidity, field water level, field water temperature, illuminance, hours of sunshine, wind speed and air pressure.
18. 18. The system according to claim 1, wherein the storage means stores the growth index data generated based on an image of the crop in association with the image.
19. 19. The system according to claim 18, wherein said storage means stores said growth index data in a header area of a file of said image in accordance with the EXIF standard format.
20. The system according to claim 18, characterized in that the storage means stores the growth index data in accordance with an API format of a WAGRI standard data platform.
21. The system according to any one of claims 1 to 20, characterized in that the image acquisition means acquires images taken by a network camera or a camera capable of measuring distances that captures images of the agricultural products.
22. A program for causing at least one computer to function as each of the means of the system according to any one of claims 1 to 21.
23. An image acquisition step of acquiring an image of a specific crop in a farm field; a generation step of generating a plurality of growth index data of the crop based on the image of the crop acquired by the image acquisition step; A storage step of storing a plurality of growth index data for a plurality of dates and times generated by the generation step; a control step of performing control to display a transition of a plurality of growth index data related to the specific crop during a period including a plurality of growth stages, based on the plurality of growth index data for the plurality of dates and times accumulated by the accumulation step; an environmental information data acquisition step for acquiring environmental information data relating to an environment of the farm field; a recording control step of controlling to record information based on the data acquired by the environmental information data acquisition step in a header area of the image so as to associate the image with the information based on the acquired data; and an instruction step of instructing to perform image analysis for obtaining growth index data of the agricultural crop, using the image acquired by the image acquisition step as an analysis target. A system control method, characterized in that the recording control step includes controlling so that the growth index data based on the image analysis is further recorded in a header area of the image.
24. An image acquisition step of acquiring an image of a specific crop in a farm field; a generating step of generating growth index data of the crop based on the image of the crop acquired by the image acquiring step; A storage step of storing the growth index data for the plurality of dates and times generated by the generation step; a control step of performing control for displaying both a transition of the growth index data for the specific crop during a period including a plurality of growth stages and a model based on past statistics, based on the growth index data for the plurality of dates and times accumulated by the accumulation step; an environmental information data acquisition step for acquiring environmental information data relating to an environment of the farm field; a recording control step of controlling to record information based on the data acquired by the environmental information data acquisition step in a header area of the image so as to associate the image with the information based on the acquired data; and an instruction step of instructing to perform image analysis for obtaining growth index data of the agricultural crop, using the image acquired by the image acquisition step as an analysis target. A system control method, characterized in that the recording control step includes controlling so that the growth index data based on the image analysis is further recorded in a header area of the image.
Citation Information
Patent Citations
Stereoscopic video photographing monitoring system
JP1987061492A
Sliver doffing order device for fine spinning creel
JP1987062929A
Plant growth system and information-providing service
JP2005013057A
Method and program for determining reaping schedule
JP2019033720A
Determination system, determination apparatus, determination method, and computer program
JP2019165655A
