A Portable Automatic Detection Device and Method for Crop Water Stress Index

Abstract

The present application discloses a portable automatic detection device and method for crop water stress index, which relates to the field of crop water stress index detection. The device includes: an image acquisition module for acquiring image data of the crop canopy and image data of a wet reference surface; a temperature acquisition module for acquiring the ambient air temperature of the crop; a main control module, connected to the image acquisition module and the temperature acquisition module respectively, for determining the image data of the crop canopy area and the image data of the wet reference surface area based on the image data of the crop canopy and the image data of the wet reference surface by using a built-in instance segmentation model, and calculating the crop water stress index according to the image data of the crop canopy area, the image data of the wet reference surface area and the ambient air temperature. The present application can improve the real-time performance and effectiveness of crop water stress index detection.

Description

Technical Field

[0001] This application relates to the field of crop water stress index detection, and particularly to a portable automatic crop water stress index detection device and method. Background Art

[0002] Under the background of the increasingly intensifying global warming and water resource shortage, agricultural production faces huge challenges. Crop water stress will directly affect the growth, yield and quality of crops. Therefore, the demand for real-time detection and management of crop water stress status is becoming increasingly urgent.

[0003] Traditional crop water stress detection equipment has the following problems: Some equipment collects single types of data, relying on single temperature data and unable to comprehensively reflect the degree of crop water stress. The data processing and analysis functions are relatively lagging, lacking support for automatic data upload and intelligent analysis, and it is difficult to achieve real-time and efficient data sharing and decision-making guidance. Summary of the Invention

[0004] The purpose of this application is to provide a portable automatic crop water stress index detection device and method, which can improve the real-time and effectiveness of crop water stress index detection.

[0005] To achieve the above purpose, this application provides the following solutions:

[0006] In the first aspect, this application provides a portable automatic crop water stress index detection device, including:

[0007] An image acquisition module, configured to acquire image data of the crop canopy and image data of the wet reference surface;

[0008] A temperature acquisition module, configured to acquire the ambient air temperature of the crop;

[0009] A main control module, respectively connected to the image acquisition module and the temperature acquisition module, configured to perform segmentation on the basis of the image data of the crop canopy and the image data of the wet reference surface by using a built-in instance segmentation model to obtain image data of the crop canopy area and image data of the wet reference surface area, and calculate a crop water stress index according to the image data of the crop canopy area, the image data of the wet reference surface area, and the ambient air temperature; wherein, the instance segmentation model is a trained YOLOv5-seg model.

[0010] In the second aspect, this application provides a method for automatically detecting a crop water stress index, including:

[0011] Collect the ambient air temperature of the crop, the image data of the crop canopy, and the image data of the wet reference surface; the image data includes thermal infrared images and visible light images, the visible light images include left visible light images and right visible light images, and the thermal infrared images include grayscale images and temperature matrices;

[0012] Based on the image data of the crop canopy and the image data of the wet reference surface, use an instance segmentation model for detection to obtain the image data of the crop canopy area and the image data of the wet reference surface area, and calculate the water stress index of the crop according to the image data of the crop canopy area, the image data of the wet reference surface area, and the ambient air temperature; wherein, the instance segmentation model is the trained YOLOv5-seg model.

[0013] According to the specific embodiments provided by the present application, the following technical effects are disclosed in the present application:

[0014] The present application provides a portable crop water stress index automatic detection device and method. Through the image acquisition module and the temperature acquisition module, the image data of the crop canopy and the wet reference surface and the ambient air temperature can be obtained simultaneously. The instance segmentation model set by the main control module is used to segment the crop canopy area and the wet reference surface area, and the crop water stress index is calculated in real time, improving the real-time performance and effectiveness of the crop water stress index. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0016] Figure 1 It is a schematic diagram of the overall structure of a portable crop water stress index automatic detection device in an embodiment of the present application;

[0017] Figure 2 It is a rear view of the overall structure of a portable crop water stress index automatic detection device in an embodiment of the present application;

[0018] Figure 3 It is a schematic diagram of a mask image of the crop canopy area provided by an embodiment of the present application;

[0019] Figure 4 It is a schematic diagram of a mask image of the wet reference surface area provided by an embodiment of the present application;

[0020] Figure 5 It is a schematic diagram of the registration effect of the left visible light image and the right visible light image provided by an embodiment of the present application;

[0021] Figure 6 Flowchart of a method for automatically detecting crop water stress index provided in an embodiment of the present application. Detailed implementation manners

[0022] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0023] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0024] The embodiment of the present application provides a portable device for automatically detecting crop water stress index. The portable device for automatically detecting crop water stress index includes: an image acquisition module, a temperature acquisition module, and a main control module.

[0025] The image acquisition module is used to acquire image data of the crop canopy and image data of the wet reference surface. As Figure 1 shown, the image acquisition module includes a thermal infrared camera 1 and binocular visible light cameras 2 arranged on the left and right sides of the thermal infrared camera 1. The binocular visible light cameras 2 can acquire high-resolution visible light images of the crop canopy and the wet reference surface. The binocular visible light cameras 2 are respectively a left visible light camera and a right visible light camera. Therefore, the acquired image data includes thermal infrared images and visible light images. The visible light images include left visible light images and right visible light images. The thermal infrared images include grayscale images and temperature matrices.

[0026] Specifically, the thermal infrared camera 1 is connected to the main control module through a USB3.0 interface, and is used to capture thermal infrared images of the crop canopy and the wet reference surface, capture the temperature distribution information on the leaf surface to obtain the temperature matrices of the crop canopy and the wet reference surface, and transmit the acquired grayscale images and corresponding temperature matrices to the main control module for analysis and processing; the binocular visible light cameras 2 are symmetrically arranged on the left and right sides of the thermal infrared camera 1 and are connected to the main control module through USB2.0 interfaces, and are used to capture visible light images of the crop canopy and the wet reference surface. The visible light images include left visible light images and right visible light images.

[0027] In an exemplary embodiment, the wet reference surface is a simulation device placed beside the crop for simulating the temperature when the stomata of the crop leaves are fully opened and in a full transpiration state.

[0028] The simulation device consists of a plastic box, a polystyrene foam board, a water-absorbent non-woven fabric and a viscose blended fabric, and a polyester non-woven fabric. The plastic box contains water, and the polystyrene foam board is covered above the plastic box. The surface of the polystyrene foam board is wrapped with a water-absorbent non-woven fabric and a viscose blended fabric, and the outermost layer is covered with a polyester non-woven fabric. This structure enables the materials on the polystyrene foam board to continuously absorb water, effectively reducing water evaporation, thereby maintaining a lower temperature, simulating the temperature of crop leaves when the stomata are fully open and in a full transpiration state, that is, the wet reference surface temperature, with the unit of degree Celsius (°C).

[0029] The design of the binocular cameras in the binocular visible light camera 2 improves the image depth perception ability, facilitating the acquisition of three-dimensional structure information, contributing to canopy segmentation and the calculation of crop water stress index, so as to enhance the comprehensive analysis of crop stress conditions.

[0030] In an exemplary embodiment, before using the thermal infrared camera 1 and the binocular visible light camera 2 to collect image data, they also need to be calibrated.

[0031] Specifically, use the thermal infrared camera 1 and the binocular visible light camera 2 to simultaneously take a set of black and white checkerboard images including thermal infrared images, left visible light images and right visible light images, and use the Zhang Zhengyou calibration method for calibration.

[0032] The Zhang Zhengyou calibration method calculates the internal parameters and external parameters of the binocular visible light camera by taking multiple black and white checkerboard images at different angles and using the least squares method. The internal parameters and external parameters only depend on the optical characteristics and relative positions of the camera. Therefore, the process of calculating the internal parameters and external parameters will not affect the processing time of registering the thermal infrared image and the visible light image. The calibration uses a special alumina black and white checkerboard calibration plate. The front panel is a black checkerboard made of alumina, and the back panel is a glass substrate. When the back of the calibration plate is uniformly heated, the different thermal conductivities and reflectivities of the two materials cause the calibration plate to form a clearly defined black and white checkerboard image in both the thermal infrared image and the visible light image.

[0033] The calibration is divided into two groups: one group is the calibration of the binocular visible light camera, and the internal parameters and external parameters of the left visible light camera and the right visible light camera are obtained; the other group is the calibration between the left visible light camera in the binocular visible light camera and the thermal infrared camera, and the internal parameters and external parameters of the left visible light camera and the thermal infrared camera are obtained. The internal parameters include focal length, principal point and distortion coefficient, and the external parameters include rotation matrix and translation vector.

[0034] The internal parameters of the camera mainly represent the mapping from the camera coordinate system to the image coordinate system, and the external parameters of the camera describe the conversion from the world coordinate system to the camera coordinate system. Among them, the rotation matrix describes the direction of the coordinate axes of the world coordinate system relative to the camera coordinate axes, and the translation matrix describes the position of the origin of space in the camera coordinate system.

[0035] The temperature acquisition module is used to acquire the ambient air temperature of the crop. The temperature acquisition module is composed of a temperature sensor 3, which is connected to the main control module through the RS-485 interface and is used to acquire the ambient air temperature in real time, providing necessary data support for subsequent calculation of the Crop Water Stress Index (CWSI).

[0036] The main control module is respectively connected to the image acquisition module and the temperature acquisition module. The main control module uses the MODBUS RTU (Remote Terminal Unit) protocol to communicate with the temperature acquisition module to realize real-time reading of temperature data and sending of instructions, facilitating subsequent calculation of the crop water stress index. Based on the image data of the crop canopy and the image data of the wet reference surface, the main control module uses the built-in instance segmentation model to determine the image data of the crop canopy area and the image data of the wet reference surface area, and calculates the crop water stress index according to the image data of the crop canopy area, the image data of the wet reference surface area and the ambient air temperature. Specifically, the instance segmentation model is the trained YOLOv5-seg model.

[0037] In an exemplary embodiment, the thermal infrared camera 1 and the binocular visible light camera 2 are aligned with the crop and photographed from the upper side of the crop to obtain the image data of the complete crop canopy, and at the same time, the image data of the wet reference surface of the same crop is obtained. The visible light images of the crop canopy and the wet reference surface obtained by shooting are first scaled and cropped, and the resolution of the visible light image is made consistent with the resolution of the thermal infrared image through appropriate scale transformation. On this basis, the visible light images of the crop canopy and the wet reference surface are labeled to obtain a training set and trained to obtain the trained YOLOv5-seg model. The specific steps are as follows:

[0038] S3.1. Data annotation: Use the Segment Anything Model (SAM) to automatically annotate the acquired visible light images of the crop canopy and the wet reference surface, and the labels for annotation are the crop canopy area and the wet reference surface area.

[0039] The SAM model is a highly versatile segmentation model. It uses an image encoder to extract features from the input image and generate a one-time image embedding. The image encoder divides the image into multiple small patches and converts the color values of each patch into a vector representation as the input sequence for the Transformer. According to the prompts provided by the user (such as points, boxes, text, etc.), a prompt encoder is used to generate prompt embeddings. The prompt encoder can convert different types of prompts into a unified vector representation for subsequent processing. Combining the outputs of the image encoder and the prompt encoder, a mask decoder is used to predict the segmentation mask. The segmentation mask represents the probability that each pixel in the image belongs to the foreground or the background, thus achieving precise segmentation of the objects in the image.

[0040] By introducing the SAM model, the crop canopy area and the wet reference surface area in the visible light images of the collected crop canopy and wet reference surface can be automatically labeled quickly and accurately, and standard data with high precision and consistency can be generated, providing a reliable data basis for the training of the YOLOv5-seg model.

[0041] S3.2, Model Training: The visible light images of the crop canopy and wet reference surface with labels are divided into a training set, a test set, and a validation set in the ratio of 7:2:1. The training set is used to train the YOLOv5-seg model to optimize the parameters of the YOLOv5-seg model. The test set is used to preliminarily verify the performance of the YOLOv5-seg model, and the validation set is used to evaluate the final effect of the YOLOv5-seg model. The YOLOv5-seg model after multiple rounds of iterative optimization can accurately segment the crop canopy area and the wet reference surface area in the visible light images of the crop canopy and wet reference surface.

[0042] S3.3, Model Conversion: After training, the trained YOLOv5-seg model is converted into the RKNN (Rockchip Neural Network) format for adaptation to embedded devices.

[0043] Specifically, the trained YOLOv5-seg model is saved in the.pt format and needs to be converted into the ONNX (Open Neural Network Exchange) format. ONNX is an open neural network exchange format that can perform model conversion and sharing between different deep learning frameworks. In order to deploy the trained YOLOv5-seg model on the main control module, it is necessary to further convert the ONNX format into the RKNN format. RKNN is a neural network inference engine designed specifically for Rockchip series chips, providing efficient neural network inference capabilities for adaptation to embedded devices.

[0044] S3.4. Model Embedding: Embed the trained YOLOv5-seg model as an instance segmentation model into the main control module, enabling it to automatically segment the crop canopy area and the wet reference surface area from the real-time collected image data in the field environment, obtaining the mask images of the crop canopy area and the wet reference surface area, as shown in Figure 3 and Figure 4 respectively. Use image preprocessing technology based on erosion operation to optimize the boundaries of the mask images, remove boundary noise, and ensure the stability of the generated points.

[0045] Determine the image data of the crop canopy area and the image data of the wet reference surface area by combining the optimized mask images of the crop canopy area and the wet reference surface area with the image data of the crop canopy and the image data of the wet reference surface.

[0046] In an exemplary embodiment, the main control module specifically includes a data acquisition unit, an instance segmentation unit, a feature registration unit, a position confirmation unit, a first calculation unit, a second calculation unit, and an index calculation module.

[0047] The data acquisition unit is used to acquire the ambient air temperature, the image data of the crop canopy, and the image data of the wet reference surface; the image data includes thermal infrared images and visible light images, and the visible light images include left visible light images and right visible light images.

[0048] The instance segmentation unit is connected to the data acquisition unit. The instance segmentation unit is used to determine the image data of the crop canopy area and the image data of the wet reference surface area based on the image data of the crop canopy and the image data of the wet reference surface by using an instance segmentation model.

[0049] Specifically, use the instance segmentation model in the instance segmentation unit to segment the visible light images in the image data of the crop canopy and the visible light images in the image data of the wet reference surface, obtaining the mask images of the crop canopy area and the wet reference surface area. Determine the image data of the crop canopy area and the image data of the wet reference surface area based on the mask images of the crop canopy area, the mask images of the wet reference surface area, the image data of the crop canopy, and the image data of the wet reference surface.

[0050] The feature registration unit is connected to the instance segmentation unit. The feature registration unit is used to register the left visible light image and the right visible light image in the image data of the crop canopy area to obtain a pair of feature points.

[0051] Specifically, the feature registration unit includes an initial feature point confirmation subunit and a feature point pair determination subunit.

[0052] The initial feature point confirmation subunit is used to divide the left visible light image and the right visible light image of the crop canopy area into a plurality of regularly distributed grid points at a preset step length, obtaining the initial feature point sets of the left visible light image and the right visible light image; wherein, the image feature at the position of each grid point is an initial feature point.

[0053] The feature point pair determination subunit is connected to the initial feature point confirmation subunit. The feature point pair determination subunit is used to register the initial feature points of the left visible light image and the right visible light image of the crop canopy area according to the geometric relationship constraint conditions, obtaining feature point pairs; wherein, the geometric relationship constraint condition is the coordinate difference of the initial feature points in the horizontal and vertical directions.

[0054] Specifically, through the geometric constraint matching algorithm, with the coordinate difference in the horizontal and vertical directions as the constraint condition, the initial feature point pairs are matched one by one in the initial feature point sets of the left visible light image and the right visible light image of the crop canopy area, ensuring that the matched initial feature point pairs satisfy the spatial geometric relationship. Finally, the initial feature point pairs that do not conform to the geometric constraints are excluded, the matched initial feature point pairs are retained, and the matched initial feature point pairs are used as the matching point pairs to complete the registration, as Figure 5 shown. At the same time, all the matched point pairs are visualized and the coordinates of the matched point pairs are obtained. The matched point pairs are used as the feature point pairs, and finally the feature point pairs of the regular grid of the left visible light image and the right visible light image of the crop canopy area are obtained.

[0055] The position confirmation unit is connected to the instance segmentation unit and the feature registration unit. The position confirmation unit is used to determine the position of the feature point pair on the grayscale image of the crop canopy area according to the position of the feature point pair on the visible light image of the crop canopy area.

[0056] Specifically, using the above calibration to obtain the internal parameters and external parameters of the left visible light camera and the right visible light camera, combined with the position of the feature point pair on the visible light image of the crop canopy area, the world coordinates of the feature point pair can be obtained. Through the world coordinates of the feature point pair and the internal parameters and external parameters of the left visible light camera and the thermal infrared camera obtained by calibration, the feature points in the feature point pair can be back-projected onto the grayscale image of the crop canopy area. The specific principle is as follows:

[0057] Assume that the left visible light camera coordinate system in the binocular visible light camera coincides with the world coordinate system, and the coordinates of a point in the world coordinate system are , and its coordinates in the left visible light camera coordinate system and the image coordinate system are and respectively. Its coordinates in the right visible light camera coordinate system and the image coordinate system in the binocular visible light camera are and .

[0058] In a known binocular visible light camera, the relationships between the left visible light camera coordinate system, the right visible light camera coordinate system, and the world coordinate system are given by equations (1) and (2) respectively:

[0059] (1);

[0060] (2);

[0061] Wherein, and are depth factors.

[0062] (3);

[0063] (4);

[0064] (5);

[0065] (6);

[0066] Wherein, is the focal length of the left camera in the left visible light camera of the binocular visible light camera; is the focal length of the right camera in the right visible light camera of the binocular visible light camera; is the principal point of the right camera in the right visible light camera of the binocular visible light camera; and are respectively the rotation matrix and the translation matrix of the right camera of the right visible light camera relative to the left camera of the left visible light camera in the binocular visible light camera; and are respectively the internal parameter matrix of the left visible light camera and the internal parameter matrix of the right visible light camera; to are all the element values of the internal parameter matrix of the left visible light camera; to are all the element values of the internal parameter matrix of the right visible light camera.

[0067] Substitute equations (3) and (4) into equations (1) and (2), and after rearrangement, equation (7) is obtained. If a pair of point coordinates in the image coordinate systems of the left camera and the right camera in the binocular visible light camera are known, the coordinates of a point in the world coordinate system can be solved through equation (7).

[0068] (7);

[0069] Through the above calculations, the world coordinate is obtained. Through the external parameters (rotation matrix and translation vector ), convert this point from the world coordinate system to the thermal infrared camera coordinate system , and use the internal parameter matrix of the thermal infrared camera to project the point in the thermal infrared camera coordinate system onto the grayscale image coordinate system. The conversion relationship is shown in Equation (8):

[0070] (8);

[0071] (9);

[0072] where is the depth factor; is the internal parameter matrix of the thermal infrared camera; is the focal length of the thermal infrared camera; is the principal point of the thermal infrared camera.

[0073] Through Equation (8) and Equation (9), the grayscale image coordinates can be obtained, which is the pixel position of the feature point on the thermal grayscale image. Similarly, the pixel position of the centroid position of the wet reference surface area on the grayscale image can also be obtained.

[0074] The first calculation unit is connected to the position confirmation unit and the instance segmentation unit. The first calculation unit is used to calculate the average temperature of the crop canopy area according to the position of the feature point pair on the grayscale image of the crop canopy area and the temperature matrix of the crop canopy area.

[0075] Specifically, according to the position of the feature point on the grayscale image of the crop canopy area, locate the corresponding temperature value in the temperature matrix of the crop canopy area, traverse the temperature matrix of the crop canopy area, and average all the temperature values corresponding to the position of the feature point to obtain the average temperature of the crop canopy area, providing data support for CWSI calculation.

[0076] The second calculation unit is connected to the instance segmentation unit. The second calculation unit is used to determine the centroid position of the grayscale image of the wet reference surface area according to the visible light image in the image data of the wet reference surface area, and calculate the temperature of the wet reference surface area according to the centroid position and the temperature matrix of the wet reference surface area.

[0077] The index calculation module is connected to the data acquisition unit, the first calculation unit, and the second calculation unit. The index calculation module is used to calculate the crop water stress index according to the ambient air temperature, the average temperature of the crop canopy area, and the temperature of the wet reference surface area.

[0078] Specifically, the crop water stress index is calculated using Equation (10):

[0079] (10);

[0080] Wherein, is the crop water stress index, is the average temperature of the crop canopy area, is the temperature of the wet reference surface area, is the temperature of the dry reference surface.

[0081] The temperature of the dry reference surface is the temperature of the dry reference surface when the simulated crop stomata are completely closed and transpiration does not occur. , is the obtained ambient air temperature.

[0082] Calculating the crop water stress index by the above portable crop water stress index automatic detection device can improve the real-time performance and effectiveness of the detection of the crop water stress index.

[0083] In an exemplary embodiment, the portable crop water stress index automatic detection device further includes a wireless communication and positioning module, a display module, a power supply module, a heat dissipation module, a grip handle, and a trigger switch.

[0084] The wireless communication and positioning module is connected to the main control module. The wireless communication and positioning module is connected to the main control module through a USB interface. Specifically, as Figure 1 shown, the wireless communication and positioning module uses a WiFi antenna 4, which is arranged on the top of the portable crop water stress index automatic detection device. The wireless communication and positioning module is used to obtain the geographical location information of the portable crop water stress index automatic detection device, that is, the Global Positioning System (GPS) information, and receive the water stress index sent by the main control module, and send the geographical location information and the water stress index to the cloud server to realize the remote real-time transmission and positioning function of data. The wireless communication and positioning module makes the crop water stress index correspond to the geographical location information one by one, which is convenient for data archiving, analysis, and future retrieval and use.

[0085] The display module is connected to the main control module. As Figure 2 shown, the display module uses a 4.3-inch LCD touch screen 8 and is connected to the main control module through a MIPI interface. The designed embedded QT Creator interface is user-friendly. The display module displays the visible light image and the grayscale image sent by the main control module in real time, and can also display the crop water stress index. Based on this, the visualization of the visible light image, the grayscale image, and the crop water stress index is realized.

[0086] The heat dissipation module is connected to the main control module through a USB interface. The heat dissipation module is set as a small fan. When the temperature inside the portable crop water stress index automatic detection device reaches a certain level, the fan will automatically start, effectively preventing the abnormal operation of the thermal infrared camera 1 due to overheating and ensuring that the thermal infrared camera 1 can work stably in a high-temperature environment.

[0087] The power supply module is connected to the main control module. The power supply module supplies power to the main control module through a power interface, and then provides electrical energy for the display module, temperature acquisition module, image acquisition module, wireless communication and positioning module, and heat dissipation module. As Figure 1 shown, the power supply module is set at the bottom of the portable crop water stress index automatic detection device. A detachable rechargeable battery 7 with a capacity of 5000 mAh is used, which can provide portable power for the device. When the battery needs to be replaced or replenished, the user can quickly replace the battery to ensure that the device can continue to work.

[0088] As Figure 1 shown, the portable crop water stress index automatic detection device is equipped with a USB interface 4 on the side, which is convenient for connecting external devices and data transmission. As Figure 1 shown, a holding handle 6 is set. The holding handle 6 conforms to the ergonomic design, is convenient for hand-held operation, is suitable for use in multiple outdoor scenarios, and a trigger switch 5 is set on the holding handle 6. The user can synchronously collect image data and ambient air temperature by pressing the trigger switch 5. Specifically, the trigger switch 5 uses a GPIO (General Purpose Input Output) button, which is convenient for users to operate.

[0089] The above-mentioned portable crop water stress index automatic detection device can be held by hand, is convenient to move, and has good portability and an independent power supply design, and can adapt to the complex application requirements in the field environment.

[0090] Using the portable crop water stress index automatic detection device to calculate the crop water stress index, the crop water stress index can be used to evaluate the current water status of the crop, quantify the crop water stress level, and provide a scientific basis for precise irrigation. The present invention is applicable to the real-time monitoring of crop water stress in the field environment and has guiding significance for field crop water irrigation.

[0091] Based on the same inventive concept, the embodiment of the present application also provides a method for automatically detecting the crop water stress index. The implementation solutions provided by this method to solve problems are similar to the implementation solutions recorded in the above-mentioned device. Therefore, the specific limitations in one or more embodiments of the method for automatically detecting the crop water stress index provided below can refer to the limitations on the method for automatically detecting the crop water stress index in the above text, and will not be repeated here.

[0092] In an exemplary embodiment, as Figure 6 shown, a method for automatically detecting a crop water stress index is provided, including:

[0093] Step 101, collect the ambient air temperature of the crop, the image data of the crop canopy, and the image data of the wet reference surface.

[0094] Step 102, based on the image data of the crop canopy and the image data of the wet reference surface, use an instance segmentation model to perform segmentation to obtain the image data of the crop canopy area and the image data of the wet reference surface area, and calculate the crop water stress index according to the image data of the crop canopy area, the image data of the wet reference surface area, and the ambient air temperature; wherein, the instance segmentation model is a trained YOLOv5-seg model.

[0095] In an exemplary embodiment, Step 102 includes Steps 201 - 207:

[0096] Step 201, obtain the ambient air temperature, the image data of the crop canopy, and the image data of the wet reference surface; wherein, the image data includes a thermal infrared image and a visible light image, the visible light image includes a left visible light image and a right visible light image, and the thermal infrared image includes a grayscale image and a temperature matrix.

[0097] Step 202, based on the image data of the crop canopy and the image data of the wet reference surface, use an instance segmentation model to determine the image data of the crop canopy area and the image data of the wet reference surface area.

[0098] Step 203, register the left visible light image and the right visible light image in the image data of the crop canopy area to obtain a pair of feature points.

[0099] Step 204, determine the position of the pair of feature points on the grayscale image of the crop canopy area according to the position of the pair of feature points on the visible light image of the crop canopy area.

[0100] Step 205, calculate the average temperature of the crop canopy area according to the position of the pair of feature points on the grayscale image of the crop canopy and the temperature matrix of the crop canopy area.

[0101] Step 206, determine the centroid position of the grayscale image of the wet reference surface area according to the visible light image in the image data of the wet reference surface area, and calculate the temperature of the wet reference surface area according to the centroid position and the temperature matrix of the wet reference surface area.

[0102] Step 207, calculate the crop water stress index according to the ambient air temperature, the average temperature of the crop canopy area, and the temperature of the wet reference surface area.

[0103] In an exemplary embodiment, step 203 specifically includes steps 301 - 302:

[0104] Step 301, divide the left visible light image and the right visible light image of the crop canopy area into a plurality of regularly distributed grid points according to a preset step size, to obtain an initial feature point set of the left visible light image and the right visible light image of the crop canopy area; wherein, the image feature at the position of each grid point is an initial feature point.

[0105] Step 302, register the initial feature points of the left visible light image and the right visible light image of the crop canopy area according to the geometric relationship constraint condition, to obtain feature point pairs; wherein, the geometric relationship constraint condition is the coordinate difference of the initial feature points in the horizontal and vertical directions.

[0106] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0107] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0108] The databases involved in the various embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the various embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0109] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0110] Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A portable crop water stress index automatic detection device, characterized in that: The portable crop water stress index automatic detection device comprises: An image acquisition module is used to acquire image data of the crop canopy and image data of the wet reference surface; wherein the image data includes a thermal infrared image, a left visible light image and a right visible light image, and the thermal infrared image includes a grayscale image and a temperature matrix; Temperature collection module, used to collect the ambient air temperature of crops; The main control module is connected to the image acquisition module and the temperature acquisition module respectively, and is used to segment the image data of the crop canopy area and the image data of the wet reference surface area using the built-in instance segmentation model based on the image data of the crop canopy area and the image data of the wet reference surface area, and calculate the crop water stress index according to the image data of the crop canopy area, the image data of the wet reference surface area and the ambient air temperature; wherein the instance segmentation model is a trained YOLOv5-seg model; The main control module specifically includes: An instance segmentation unit is used to determine the image data of the crop canopy region and the image data of the wet reference surface region by using an instance segmentation model based on the image data of the crop canopy and the image data of the wet reference surface; A feature registration unit is connected to the instance segmentation unit and is used to register the left visible light image and the right visible light image in the image data of the crop canopy area to obtain a feature point pair; the feature registration unit includes an initial feature point confirmation subunit and a feature point pair determination subunit; the initial feature point confirmation subunit is used to divide the left visible light image and the right visible light image of the crop canopy area into a plurality of regularly distributed grid points according to a preset step length, and obtain an initial feature point set of the left visible light image and the right visible light image; wherein the image feature at the location of each grid point is an initial feature point; the feature point pair determination subunit is connected to the initial feature point confirmation subunit, and the feature point pair determination subunit is used to register the initial feature points of the left visible light image and the right visible light image of the crop canopy area according to geometric relationship constraints to obtain a feature point pair; wherein the geometric relationship constraints are the coordinate differences of the initial feature points in the horizontal and vertical directions; The position confirmation unit is connected to the instance segmentation unit and the feature registration unit, and is used to determine the position of the feature point pair on the grayscale image of the crop canopy area according to the position of the feature point pair on the visible light image of the crop canopy area.

2. The portable crop water stress index automatic detection device according to claim 1, characterized in that: The main control module further includes: a first calculation unit connected to the position confirmation unit and the instance segmentation unit, and configured to calculate an average temperature of the crop canopy region according to the position of the feature point pair on the grayscale image of the crop canopy region and a temperature matrix of the crop canopy region; a second calculation unit, connected to the instance segmentation unit, for determining a centroid position of a grayscale image of the wet reference surface region according to a visible light image in the image data of the wet reference surface region, and calculating a temperature of the wet reference surface region according to the centroid position and a temperature matrix of the wet reference surface region; The index calculation module is connected to the first calculation unit and the second calculation unit, and is used to calculate the crop water stress index according to the ambient air temperature, the average temperature of the crop canopy area and the temperature of the wet reference surface area.

3. The portable crop water stress index automatic detection device according to claim 2, characterized in that: The portable crop water stress index automatic detection device also includes: The wireless communication positioning module is connected to the main control module, and is used to obtain the geographical location information of the portable crop water stress index automatic detection device and receive the crop water stress index sent by the main control module, and send the geographical location information and the crop water stress index to the cloud server.

4. The portable crop water stress index automatic detection device according to claim 3, characterized in that: The portable crop water stress index automatic detection device also includes: The display module is connected to the main control module and is used to receive the visible light image and grayscale image sent by the main control module and display them in real time.

5. The portable crop water stress index automatic detection device according to claim 4, characterized in that: The portable crop water stress index automatic detection device also includes: The power module is connected to the main control module and is used to provide power to the main control module, the display module, the temperature acquisition module, the wireless communication positioning module, and the image acquisition module.

6. The portable crop water stress index automatic detection device according to claim 1, characterized in that: The image acquisition module includes a thermal infrared camera and binocular visible light cameras arranged on the left and right sides of the thermal infrared camera.

7. A method for automatically detecting crop water stress index, applied to the portable automatic detection device for crop water stress index according to any one of claims 1 to 6, characterized in that: The crop water stress index automatic detection method comprises: Collecting the ambient air temperature of the crop, the image data of the crop canopy and the image data of the wet reference surface; wherein the image data includes a thermal infrared image, a left visible light image and a right visible light image, and the thermal infrared image includes a grayscale image and a temperature matrix; Based on the image data of the crop canopy and the image data of the wet reference surface, an instance segmentation model is used to segment the image data of the crop canopy area and the image data of the wet reference surface area, and the crop water stress index is calculated according to the image data of the crop canopy area, the image data of the wet reference surface area and the ambient air temperature; wherein the instance segmentation model is a trained YOLOv5-seg model; the left visible light image and the right visible light image of the crop canopy area are registered to obtain feature point pairs of the left and right visible light images, specifically including: Dividing the left visible light image and the right visible light image of the crop canopy region into a plurality of regularly distributed grid points according to a preset step length, and obtaining an initial feature point set of the left visible light image and the right visible light image of the crop canopy region; wherein the image feature at the position of each grid point is an initial feature point; The initial feature points of the left visible light image and the right visible light image of the crop canopy area are aligned according to geometric relationship constraints to obtain feature point pairs; wherein the geometric relationship constraints are the coordinate differences of the initial feature points in the horizontal and vertical directions.

8. The method for automatically detecting crop water stress index according to claim 7, characterized in that: The crop water stress index is calculated based on the image data of the crop canopy area, the image data of the wet reference surface area and the ambient air temperature, specifically including: Obtain image data of ambient air temperature, crop canopy, and wet reference surface; Based on the image data of the crop canopy and the image data of the wet reference surface, an instance segmentation model is used to determine the image data of the crop canopy region and the image data of the wet reference surface region; Registering a left visible light image and a right visible light image in the image data of the crop canopy region to obtain a pair of feature points; Determining the position of the feature point pair on the grayscale image of the crop canopy region according to the position of the feature point pair on the visible light image of the crop canopy region; Calculating the average temperature of the crop canopy region according to the positions of the feature point pairs on the grayscale image of the crop canopy region and the temperature matrix of the crop canopy region; Determine the centroid position of the grayscale image of the wet reference surface area according to the visible light image in the image data of the wet reference surface area, and calculate the temperature of the wet reference surface area according to the centroid position and the temperature matrix of the wet reference surface area; The crop water stress index is calculated based on the ambient air temperature, the average temperature of the crop canopy area and the temperature of the wet reference surface area.

9. The method for automatically detecting crop water stress index according to claim 7, characterized in that: The calculation formula of crop water stress index is: ； in, is the crop water stress index, is the average temperature of the crop canopy area, is the temperature of the wet reference surface area, is the temperature of the dry reference surface.

Images