Crop growth vigor detection device and method based on canopy temperature and light information

Abstract

The invention provides a device and a method for detecting crop growth based on canopy temperature and light information. The device comprises an information acquisition module, a wireless communication module, a power supply module and a terminal processing module, the information acquisition module is used for acquiring crop canopy information and surrounding environment information and transmitting the crop canopy information and the surrounding environment information to the terminal processing module through the wireless communication module; the power supply module is respectively connected with the information acquisition module and the wireless communication module; and the terminal processing module constructs a crop three-dimensional model according to the crop canopy information and the surrounding environment information, and obtains comprehensive growth information from the constructed crop three-dimensional model. The crop canopy information acquisition device can realize acquisition of crop growth information, has portability, can adapt to nondestructive detection of crop growth information in various cultivation environments such as a field and a greenhouse, greatly improves the convenience and practicability of crop growth detection, and is beneficial to solving the problem that the application of an existing crop canopy information acquisition device in a complex environment is limited.

Description

Technical Field

[0001] The invention belongs to the field of crop comprehensive growth information detection, and in particular relates to a crop growth detection device and method based on canopy temperature and light information. Background Art

[0002] In the development process of our country, crops are extremely important. They are the foundation of the national economy and play a fundamental role in social stability and national independence. After years of development, my country's crop planting technology has made great progress and improvement, effectively improving the yield and quality of crops. However, in the process of crop cultivation and management, it is easy to be affected by climatic conditions, water and fertilizer conditions, and pests and diseases, resulting in problems such as reduced production efficiency, reduced quality, and aggravated pests and diseases, which affect the stable improvement of crop yield and quality. How to improve crop yield and quality has become an important issue in the crop production process. Traditional crop detection mainly involves regular manual inspection of crop growth conditions, which is difficult to deeply understand the internal information of crops, has poor reliability, and is difficult to accurately obtain crop characterization information such as growth, and cannot achieve scientific management and refined agricultural production. Therefore, there is an urgent need to accurately monitor the growth, nutritional information and growth process of crops to provide technical support for digital breeding and cultivation management.

[0003] A rice field moisture control method based on crop canopy growth patent publication number: CN112219657A, the invention installs a crop canopy monitoring module in the rice field and installs light intensity sensors inside and outside the canopy, which are respectively fixed on a monitoring bracket to collect information. The crop canopy growth information collection and installation sensors are complex and not portable, making it difficult to realize multi-environment crop canopy information collection.

[0004] A plant health status detection method based on thermal imaging technology Patent Publication No.: CN103674998A, based on thermal imaging technology, obtains plant leaf temperature information through thermal imaging sensors, and evaluates plant growth status by analyzing the temperature data of thermal infrared images of leaves, which has certain limitations. A single sensor cannot obtain high-throughput information of plants and cannot fully reflect the growth of plants, resulting in inaccurate evaluation results. Summary of the invention

[0005] In response to the above technical problems, the present invention provides a crop growth detection device and method based on canopy temperature and light information to achieve the acquisition of crop growth information. The device and method are portable and can adapt to the non-destructive detection of crop growth information in various cultivation environments such as fields and greenhouses. This greatly improves the convenience and practicality of crop growth detection, and is conducive to solving the problem of limited application of existing crop canopy information collection devices in complex environments.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A crop growth detection device based on canopy temperature and light information, comprising an information collection module, a wireless communication module, a power supply module and a terminal processing module;

[0008] The information collection module is used to collect crop canopy information and surrounding environment information, and transmit the crop canopy information and surrounding environment information to the terminal processing module through the wireless communication module;

[0009] The power supply module is connected to the information acquisition module and the wireless communication module respectively;

[0010] The terminal processing module constructs a three-dimensional crop model according to crop canopy information and surrounding environment information, and obtains comprehensive growth information from the constructed three-dimensional crop model.

[0011] In the above scheme, the information acquisition module includes a depth camera, a multispectral camera, a visible light camera, a thermal imaging camera, a temperature and humidity sensor, and an illumination sensor;

[0012] The temperature and humidity sensor is used to obtain ambient temperature and humidity information and transmit it to the terminal processing module;

[0013] The light intensity sensor is used to obtain solar radiation information and transmit it to the terminal processing module;

[0014] The depth camera is used to obtain a three-dimensional image of the crop canopy and transmit it to a terminal processing module;

[0015] The visible light camera is used to obtain visible light images of crop canopies and transmit them to the terminal processing module;

[0016] The multispectral camera is used to obtain crop canopy spectral images and transmit them to the terminal processing module;

[0017] The thermal infrared camera is used to obtain thermal infrared images of crop canopies and transmit them to a terminal processing module.

[0018] Further, it also includes an upper shell, a handle, a first bottom box, a second bottom box, a bottom cover and a top cover;

[0019] One end of the handle is connected to the bottom of the upper shell, and the other end is connected to the upper part of the first bottom box, and the lower part of the first bottom box is connected to the upper part of the second bottom box; the bottom cover is arranged on the first bottom box, and the top cover is arranged on the upper shell;

[0020] The visible light camera and the thermal imaging camera are arranged in the upper housing and are located in the same horizontal plane and facing the same direction;

[0021] The multispectral camera is disposed in the first bottom box and faces the same direction as the visible light camera and the thermal imaging camera;

[0022] The depth camera is arranged in a depth camera housing, the depth camera housing is connected to the lower part of the upper housing through a card slot, and the depth camera, the visible light camera and the thermal imaging camera face the same direction;

[0023] The temperature and humidity sensor is installed on the bottom cover, and the light intensity sensor is arranged on the top cover.

[0024] Furthermore, the wireless communication module includes a microcontroller and a wireless transmission module; the microcontroller is connected to the information acquisition module and the wireless transmission module respectively, and the wireless transmission module uploads the crop canopy information in the microcontroller to the terminal processing module in the form of wireless transmission; the microcontroller and the wireless transmission module are installed in the second bottom box;

[0025] The power module includes a power button, a power supply port and a power module; the power module is connected to the information acquisition module and the wireless communication module respectively; the power button, the power supply port and the power module are arranged on the upper housing;

[0026] The terminal processing module includes a host computer and a cloud; the cloud is connected to the display interface of the host computer and the lower computer respectively; the cloud constructs a three-dimensional crop model according to the received crop canopy information and surrounding environment information, obtains comprehensive growth information from the constructed three-dimensional crop model, and transmits it to the display interface of the host computer and the display interface of the lower computer for human-computer interaction and result output display; the display interface of the lower computer is arranged on the upper shell.

[0027] A detection method according to the crop growth detection device based on canopy temperature and light information comprises the following steps:

[0028] The information collection module collects crop canopy information and surrounding environment information, and transmits the crop canopy information and surrounding environment information to the terminal processing module through the wireless communication module; the terminal processing module constructs a three-dimensional crop model based on the crop canopy information and surrounding environment information, and obtains comprehensive growth information from the constructed three-dimensional crop model;

[0029] Specific:

[0030] The information acquisition module is aimed at the crop canopy, synchronously acquires high-throughput information of the crop canopy, and transmits the information as raw data to the microcontroller of the wireless communication module;

[0031] The microcontroller performs edge computing processing on the raw data, saves the processed image information, and transmits the processed image information to the cloud via a wireless transmission module;

[0032] The cloud performs data processing and analysis, constructs a three-dimensional crop model and obtains comprehensive growth information, and displays the obtained comprehensive growth information on the display interface of the upper computer and the display interface of the lower computer, thereby completing the detection of the comprehensive growth information of the crop.

[0033] In the above scheme, collecting crop canopy high-throughput information specifically involves parallel acquisition of high-throughput information of the crop's entire growth cycle through the information collection module:

[0034] Use temperature and humidity sensors to obtain ambient temperature and humidity information, use light sensors to obtain solar radiation intensity, use depth cameras to obtain three-dimensional images of crop canopies, use visible light cameras to obtain visible light images of crop canopies, use multispectral cameras to obtain spectral images of crop canopies, and use thermal infrared cameras to obtain thermal infrared images of crop canopies.

[0035] In the above scheme, the edge computing processing is specifically as follows: removing the image background and noise of the three-dimensional image, visible light image, spectral image and thermal infrared image acquired by the information acquisition module, extracting the target image, and then acquiring the processed crop image information and saving it in the microcontroller.

[0036] In the above scheme, the construction of the crop three-dimensional model specifically includes the following steps:

[0037] Spatial registration: Obtain the three-dimensional image, visible light image, spectral image and thermal infrared image of the crop canopy with calibration plate collected by the information acquisition module, and perform spatial coordinate registration;

[0038] Image Fusion:

[0039] Visible light-multi-spectral fusion: The spectral image is unmixed using the non-negative matrix decomposition method, and the end-member spectral curve is obtained by adding the sparse constraint of the abundance matrix to the unmixing. According to the spectral absorption curve of the visible light image and the end-member spectral curve obtained by unmixing, the mixing coefficient of each pixel in the image is solved by constrained quadratic programming using the function library in MATLAB. The end-member spectral curve obtained by unmixing and the mixing coefficient of each pixel are linearly mixed to obtain a visible light-multi-spectral fusion image with high spatial resolution and multi-spectral image.

[0040] High-dimensional image fusion: Use thermal infrared images to obtain edge and temperature target areas, and obtain target areas from visible light-multispectral fusion images at the same time. Select the generative adversarial network as the fusion model, take the visible light-multispectral target area and thermal infrared target area images as input, and use the structural similarity index and peak signal-to-noise ratio as part of the loss function to guide model learning; by designing a weighted fusion strategy, assign weights to the input image contributions to generate a fused image, and evaluate the image quality by calculating the structural similarity, peak signal-to-noise ratio and spectral distortion of the fused image, while performing visual evaluation to ensure the integrity of the image information; sharpen the fused image to obtain a high-dimensional vector data image;

[0041] Image-point cloud integration: By unifying the three-dimensional image data and high-dimensional vector data images into a reference coordinate system, the high-dimensional vector data images and the three-dimensional image position information are aligned. The high-dimensional vector data images and the three-dimensional images are matched with each other using a stereo matching algorithm. The pixel information of each pixel point of the high-dimensional vector data image is mapped to the three-dimensional image through the position relationship. The three-dimensional image is combined with the pixel information of each point of the high-dimensional vector data image to establish a connection between the spatial position points of the crop and the pixels of the high-dimensional vector data image. The image and point cloud are integrated to obtain an image-point cloud image.

[0042] Furthermore, the spatial registration is specifically as follows: the acquisition information acquisition module acquires the three-dimensional image, visible light image, spectral image and thermal infrared image of the crop canopy with the calibration plate; the visible light camera, thermal infrared camera and multi-spectral camera shoot the image with the calibration plate, and the pixel coordinates of each corner point are acquired by the corner point detection algorithm, and the camera internal parameters are acquired by knowing the size of the calibration plate; the feature point matching algorithm is used to acquire the correspondence between the three-dimensional coordinates and the pixel coordinates in the images, images and images, and the camera external parameters are calibrated to perform the spatial coordinate registration;

[0043] Furthermore, the method of obtaining the comprehensive crop growth information is as follows:

[0044] According to the image-point cloud image, the color texture, crown area, plant height, plant width, leaf inclination, leaf length, leaf width, and leaf area of ​​the crop plants are obtained, thereby obtaining plant type parameters and disease information;

[0045] According to the image-point cloud image, the temperature distribution of the crop plants is obtained, and the crown-air temperature difference characteristics of the crop plants are obtained in combination with the ambient temperature information;

[0046] The spectral index in different bands is obtained based on the visible light-multispectral image of the crop canopy to obtain information on chlorophyll content, water status and photosynthesis efficiency.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The crop growth detection device based on canopy temperature and light information of the present invention adopts a multi-source imaging unit integrated design, has the characteristics of portability, ease of use and high throughput, and can be effectively applied to crop production management and digital breeding in different cultivation scenarios;

[0049] The present invention can collect high-throughput information of crops, construct a high-dimensional vector feature data set by combining depth images with color images, thermal infrared images and multispectral images, obtain comprehensive growth information of crops, and provide a scientific basis for crop growth management;

[0050] The present invention uses edge computing technology to process data from depth images, visible light images, multispectral images, and thermal infrared images acquired by multiple sensors to obtain target images, reduce redundancy, and improve the efficiency of later image processing;

[0051] The present invention adopts wireless transmission technology to realize wireless transmission of sensor data, facilitates real-time data collection and remote transmission, and can quickly analyze and process the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a three-dimensional schematic diagram of the detection device of the present invention;

[0053] Figure 2 It is an exploded view of the three-dimensional structure of the detection device of the present invention;

[0054] Figure 3 It is a schematic diagram of the buttons and display end surface on the back of the detection device of the present invention;

[0055] Figure 4 It is a schematic diagram of the end face of the lens of the detection device of the present invention;

[0056] Figure 5 It is a schematic diagram of the side end surface of the detection device of the present invention;

[0057] Figure 6 The present invention is a method and device operation diagram for detecting crop growth based on canopy temperature and light information according to an embodiment of the present invention.

[0058] In the figure, 1. light intensity sensor, 2. power module, 3. power button, 4. lower computer display interface, 5. communication button, 6. power port, 7. upper shell, 8. handle, 9. first bottom box, 10. wireless transmission module, 11. second bottom box, 12 microcontroller, 13. multispectral camera, 14. bottom cover, 15. temperature and humidity sensor, 16. card slot, 17. depth camera, 18. depth camera housing, 19. thermal infrared camera, 20. visible light camera, 21. top cover, 22. calibration board, 23. upper computer display interface, 24. cloud. DETAILED DESCRIPTION

[0059] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0060] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "front", "back", "left", "right", "up", "down", "axial", "radial", "vertical", "horizontal", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0061] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0062] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0063] Figure 1-5 The figure shows a preferred embodiment of the crop growth detection device based on canopy temperature and light information of the present invention, wherein the crop growth detection device based on canopy temperature and light information comprises an information acquisition module, a wireless communication module, a power supply module and a terminal processing module;

[0064] The information collection module is used to collect crop canopy information and surrounding environment information, and transmit the crop canopy information and surrounding environment information to the terminal processing module through the wireless communication module;

[0065] The power supply module is connected to the information acquisition module and the wireless communication module respectively, and is used for powering the entire device;

[0066] The terminal processing module constructs a three-dimensional crop model according to crop canopy information and surrounding environment information, and obtains comprehensive growth information from the constructed three-dimensional crop model.

[0067] The information acquisition module includes a depth camera 17, a multi-spectral camera 13, a visible light camera 20, a thermal imaging camera 19, a temperature and humidity sensor 15 and an illumination sensor 1; the information acquisition module includes a depth camera 17, a multi-spectral camera 13, a visible light camera 20, a thermal imaging camera 19, a temperature and humidity sensor 15 and an illumination sensor 1, which are used to collect crop canopy information and surrounding environment information, and transmit the crop canopy information and surrounding environment information to the cloud 24 of the terminal processing module through the wireless communication module.

[0068] The temperature and humidity sensor 15 is used to obtain ambient temperature and humidity information and transmit it to the terminal processing module;

[0069] The light intensity sensor 1 is used to obtain solar radiation information and transmit it to the terminal processing module;

[0070] The depth camera 17 is used to obtain a three-dimensional image of the crop canopy and transmit it to a terminal processing module;

[0071] The visible light camera 20 is used to obtain visible light images of crop canopies and transmit them to the terminal processing module;

[0072] The multispectral camera 13 is used to obtain crop canopy spectral images and transmit them to the terminal processing module;

[0073] The thermal infrared camera 19 is used to obtain thermal infrared images of crop canopies and transmit them to the terminal processing module.

[0074] The crop growth detection device based on canopy temperature and light information also includes an upper shell 7, a handle 8, a first bottom box 9, a second bottom box 11, a bottom cover 14 and a top cover 21;

[0075] One end of the handle 8 is connected to the bottom of the upper shell 7, and the other end is connected to the upper part of the first bottom box 9, and the lower part of the first bottom box 9 is connected to the upper part of the second bottom box 11; the bottom cover 14 is arranged on the first bottom box 9, and the top cover 21 is arranged on the upper shell 7;

[0076] The visible light camera 20 and the thermal imaging camera 19 are arranged in the upper housing 7 and are located in the same horizontal plane and facing the same direction;

[0077] The multispectral camera 13 is disposed in the first bottom box 9 and faces the same direction as the visible light camera 20 and the thermal imaging camera 19;

[0078] The depth camera 17 is disposed in a depth camera housing 18 , and the depth camera housing 18 is connected to the lower portion of the upper housing 7 via a card slot 16 , and the depth camera 17 faces the same direction as the visible light camera 20 and the thermal imaging camera 19 ;

[0079] The temperature and humidity sensor 15 is installed on the bottom cover 14 , and the light intensity sensor 1 is arranged on the top cover 21 .

[0080] The wireless communication module includes a microcontroller 12 and a wireless transmission module 10; the microcontroller 12 is connected to the information acquisition module and the wireless transmission module 10 respectively, and the wireless transmission module 10 uploads the crop canopy information in the microcontroller 12 to the terminal processing module in the form of wireless transmission; the microcontroller 12 and the wireless transmission module 10 are installed in the second bottom box 11.

[0081] Preferably, the wireless transmission module 10 is a long-range 5G module.

[0082] The power module includes a power button 3, a power supply port 6 and a power module 2; the power module is respectively connected to the information acquisition module and the wireless communication module for powering the entire device; the power button 3, the power supply port 6 and the power module 2 are arranged on the upper shell 7.

[0083] The terminal processing module includes a host computer and a cloud 20; the cloud 20 is connected to the host computer and the lower computer display interface 4 respectively; the cloud 20 constructs a three-dimensional crop model based on the received crop canopy information and surrounding environment information, obtains comprehensive growth information from the constructed three-dimensional crop model, and transmits it to the display interface 23 of the host computer and the lower computer display interface 4 for human-computer interaction and result output display; the lower computer display interface 4 is arranged on the upper shell 7.

[0084] Preferably, the host computer is a computer, and the display interface 23 of the host computer is a computer screen.

[0085] It also includes a communication button 5 , which is mounted on the upper housing 7 and connected to the microcontroller 12 , and the microcontroller 12 communicates with the host computer through the communication button 5 .

[0086] A detection method according to the detection device for crop growth based on canopy temperature and light information is characterized by comprising the following steps:

[0087] The information collection module collects crop canopy information and surrounding environment information, and transmits the crop canopy information and surrounding environment information to the terminal processing module through the wireless communication module; the terminal processing module constructs a three-dimensional crop model based on the crop canopy information and surrounding environment information, and obtains comprehensive growth information from the constructed three-dimensional crop model;

[0088] Specific:

[0089] Turn on the power 3 and wait for the system to fully start;

[0090] Press the communication button 5 to establish communication between the host computer and the microcontroller 12;

[0091] Adjust the crop growth detection device based on canopy temperature and light information to an appropriate height and angle so that the information collection module is aligned with the crop canopy, synchronously collects high-throughput information of the crop canopy, and transmits this information as raw data to the microcontroller 12 of the wireless communication module;

[0092] The microcontroller 12 performs edge computing processing on the raw data, saves the processed image information, and transmits the processed image information to the cloud 24 via a wireless transmission module;

[0093] The cloud 24 performs data processing and analysis, constructs a three-dimensional crop model and obtains comprehensive growth information, and displays the obtained comprehensive growth information on the display interface 23 of the host computer and the display interface 4 of the slave computer, thereby completing the detection of the comprehensive growth information of the crop.

[0094] Preferably, the synchronous acquisition is specifically as follows: the unified clock source of the microcontroller 12 is used as the system clock, the information acquisition module is connected to the microcontroller 12, and the acquisition command is sent by the microcontroller 12 as the acquisition signal for information acquisition, and the time compensation algorithm is used inside the software to control the synchronous acquisition of the information end of the information acquisition module to achieve the purpose of synchronous acquisition.

[0095] The adjustment of the detection device for crop growth based on canopy temperature and light information to an appropriate height and angle is specifically as follows: holding the handle 8 to move the detection device to the crop canopy, and adjusting the height so that the crop is located in the acquisition field of view and the image is clear.

[0096] Collecting crop canopy high-throughput information specifically involves parallel acquisition of high-throughput information of the crop's entire growth cycle through the information collection module:

[0097] The temperature and humidity sensor 15 is used to obtain the ambient temperature and humidity information, the illuminance sensor 1 is used to obtain the solar radiation intensity, the depth camera 17 is used to obtain the three-dimensional image of the crop canopy, the visible light camera 20 is used to obtain the visible light image of the crop canopy, the multispectral camera 13 is used to obtain the spectral image of the crop canopy, and the thermal infrared camera 19 is used to obtain the thermal infrared image of the crop canopy.

[0098] The edge computing processing is specifically as follows: removing the image background and noise of the three-dimensional image, visible light image, spectral image and thermal infrared image acquired by the information acquisition module, extracting the target image, and then acquiring the processed crop image information and storing it in the microcontroller 12.

[0099] The construction of the crop three-dimensional model specifically comprises the following steps:

[0100] Spatial registration: The information acquisition module acquires images of the crop canopy with the calibration plate, image, image, image; the visible light camera 20, the thermal infrared camera 19 and the multispectral camera 13 take images with the calibration plate 22, and the pixel coordinates of each corner point are acquired by the corner point detection algorithm, and the camera's intrinsic parameters are acquired by knowing the size of the calibration plate 22; the feature point matching algorithm is used to acquire the correspondence between the three-dimensional coordinates and the pixel coordinates in the images, images and images, and the camera extrinsic parameters are calibrated to perform spatial coordinate registration;

[0101] Image Fusion:

[0102] Visible light-multi-spectral fusion: The spectral image is unmixed using the non-negative matrix decomposition method, and the end-member spectral curve is obtained by adding the sparse constraint of the abundance matrix to the unmixing. According to the spectral absorption curve of the visible light image and the end-member spectral curve obtained by unmixing, the mixing coefficient of each pixel in the image is solved by constrained quadratic programming using the function library in MATLAB. The end-member spectral curve obtained by unmixing and the mixing coefficient of each pixel are linearly mixed to obtain a visible light-multi-spectral fusion image with high spatial resolution and multi-spectral image.

[0103] High-dimensional image fusion:

[0104] The edge and temperature target areas are obtained from the thermal infrared image, and the target area is obtained from the visible light-multispectral fusion image. The generative adversarial network is selected as the fusion model, and the visible light-multispectral target area and thermal infrared target area images are used as input. The structural similarity index and peak signal-to-noise ratio are used as part of the loss function to guide the model to learn to maintain image details; by designing a weighted fusion strategy, weights are assigned to the input image contributions to generate a fused image. The weight allocation is determined by the smaller the value of the loss function under different weights. The image quality is evaluated by calculating the structural similarity, peak signal-to-noise ratio and spectral distortion of the fused image, and a visual evaluation is performed to ensure the integrity of the image information; the fused image is sharpened to enhance the details and obtain a high-dimensional vector data image;

[0105] Image-Point Cloud Integration:

[0106] By unifying the three-dimensional image data and high-dimensional vector data images into a reference coordinate system, the registration of the high-dimensional vector data images and the three-dimensional image position information is achieved, and the feature matching of the high-dimensional vector data images and the three-dimensional images is performed using a stereo matching algorithm. The pixel information of each pixel point of the high-dimensional vector data image is mapped to the three-dimensional image through the position relationship, and the three-dimensional image is combined with the pixel information of each point of the high-dimensional vector data image to establish a connection between the spatial position points of the crop and the pixels of the high-dimensional vector data image, thereby achieving the fusion of the image and the point cloud and obtaining an image-point cloud image.

[0107] The spatial registration is specifically:

[0108] The acquisition information acquisition module acquires the three-dimensional image, visible light image, spectral image and thermal infrared image of the crop canopy with the calibration plate 22; the visible light camera 20, the thermal infrared camera 19 and the multi-spectral camera 13 shoot the image with the calibration plate 22, and obtain the pixel coordinates of each corner point through the corner point detection algorithm, and obtain the camera's internal parameters by knowing the size of the calibration plate 22; the feature point matching algorithm is used to obtain the correspondence between the three-dimensional coordinates and the pixel coordinates in the image, image and image, calibrate the camera's external parameters, and perform spatial coordinate registration;

[0109] The specific method of obtaining the comprehensive crop growth information is as follows:

[0110] According to the image-point cloud image, the color texture, crown area, plant height, plant width, leaf inclination, leaf length, leaf width, leaf area, etc. of the crop plant are obtained, so as to obtain plant type parameters and disease information;

[0111] According to the image-point cloud image, the temperature distribution of the crop plants is obtained, and the crown-air temperature difference characteristics of the crop plants are obtained in combination with the ambient temperature information;

[0112] The spectral index in different bands is obtained based on the visible light-multispectral image of the crop canopy to obtain information on chlorophyll content, water status and photosynthesis efficiency.

[0113] The present invention obtains three-dimensional images, visible light images, thermal infrared images, multi-spectral images, environmental temperature and humidity, and solar radiation intensity of crop canopies through an information acquisition module. The data is uploaded to a terminal processing module through a wireless transmission module. The data collected from the crop canopy are analyzed and three-dimensionally reconstructed to obtain comprehensive growth information of crops in different growth stages.

[0114] The present invention has strong adaptability and can cope with the crop growth conditions under various complex environmental conditions. Secondly, by integrating multi-sensor fusion technology, storage technology, and remote processing technology, it is portable and can adapt to various cultivation environments such as fields and greenhouses. It can realize non-destructive detection of crop growth information, greatly improving the convenience and practicality of crop growth detection, and is conducive to solving the problem of limited application of existing crop canopy information collection devices in complex environments.

[0115] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0116] The series of detailed descriptions listed above are only specific descriptions of feasible embodiments of the present invention. They are not intended to limit the scope of protection of the present invention. All equivalent embodiments or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A crop growth detection device based on canopy temperature and light information, characterized in that: It includes information collection module, wireless communication module, power supply module and terminal processing module; The information collection module is used to collect crop canopy information and surrounding environment information, and transmit the crop canopy information and surrounding environment information to the terminal processing module through the wireless communication module; The power supply module is connected to the information acquisition module and the wireless communication module respectively; The terminal processing module constructs a three-dimensional crop model according to crop canopy information and surrounding environment information, and obtains comprehensive growth information from the constructed three-dimensional crop model.

2. The crop growth detection device based on canopy temperature and light information according to claim 1, characterized in that: The information acquisition module comprises a depth camera (17), a multi-spectral camera (13), a visible light camera (20), a thermal imaging camera (19), a temperature and humidity sensor (15) and a light intensity sensor (1); The temperature and humidity sensor (15) is used to obtain environmental temperature and humidity information and transmit it to the terminal processing module; The light intensity sensor (1) is used to obtain solar radiation information and transmit it to the terminal processing module; The depth camera (17) is used to obtain a three-dimensional image of the crop canopy and transmit it to a terminal processing module; The visible light camera (20) is used to obtain visible light images of crop canopies and transmit them to a terminal processing module; The multispectral camera (13) is used to obtain a spectral image of the crop canopy and transmit it to a terminal processing module; The thermal infrared camera (19) is used to obtain thermal infrared images of crop canopies and transmit them to a terminal processing module.

3. The crop growth detection device based on canopy temperature and light information according to claim 2 is characterized in that: It also includes an upper shell (7), a handle (8), a first bottom box (9), a second bottom box (11), a bottom cover (14) and a top cover (21); One end of the handle (8) is connected to the bottom of the upper shell (7), and the other end is connected to the upper part of the first bottom box (9); the lower part of the first bottom box (9) is connected to the upper part of the second bottom box (11); the bottom cover (14) is arranged on the first bottom box (9), and the top cover (21) is arranged on the upper shell (7); The visible light camera (20) and the thermal imaging camera (19) are arranged in the upper housing (7) and are located in the same horizontal plane and facing the same direction; The multispectral camera (13) is arranged in the first bottom box (9) and faces the same direction as the visible light camera (20) and the thermal imaging camera (19); The depth camera (17) is arranged in a depth camera housing (18), the depth camera housing (18) is connected to the lower part of the upper housing (7) via a card slot (16), and the depth camera (17) faces the same direction as the visible light camera (20) and the thermal imaging camera (19); The temperature and humidity sensor (15) is installed on the bottom cover (14), and the light intensity sensor (1) is arranged on the top cover (21).

4. The crop growth detection device based on canopy temperature and light information according to claim 3 is characterized in that: The wireless communication module comprises a microcontroller (12) and a wireless transmission module (10); the microcontroller (12) is connected to the information acquisition module and the wireless transmission module (10) respectively, and the wireless transmission module (10) uploads the crop canopy information in the microcontroller (12) to the terminal processing module in the form of wireless transmission; the microcontroller (12) and the wireless transmission module (10) are installed in the second bottom box (11); The power module comprises a power button (3), a power supply port (6) and a power module (2); the power module is connected to the information collection module and the wireless communication module respectively; the power button (3), the power supply port (6) and the power module (2) are arranged on the upper housing (7); The terminal processing module comprises a host computer and a cloud (20); the cloud (20) is connected to the host computer and the display interface (4) of the lower computer respectively; the cloud (20) constructs a three-dimensional crop model according to the received crop canopy information and surrounding environment information, obtains comprehensive growth information from the constructed three-dimensional crop model, and transmits it to the display interface (23) of the host computer and the display interface (4) of the lower computer for human-computer interaction and result output display; the display interface (4) of the lower computer is arranged on the upper shell (7).

5. A detection method of a crop growth detection device based on canopy temperature and light information according to any one of claims 1 to 4, characterized in that: The following steps are involved: The information collection module collects crop canopy information and surrounding environment information, and transmits the crop canopy information and surrounding environment information to the terminal processing module through the wireless communication module; the terminal processing module constructs a three-dimensional crop model based on the crop canopy information and surrounding environment information, and obtains comprehensive growth information from the constructed three-dimensional crop model; Specific: The information collection module is aimed at the crop canopy, synchronously collects high-throughput information of the crop canopy, and transmits the information as raw data to the microcontroller (12) of the wireless communication module; The microcontroller (12) performs edge computing processing on the raw data, stores the processed image information, and transmits the processed image information to the cloud (24) via a wireless transmission module; The cloud (24) performs data processing and analysis, constructs a three-dimensional crop model and obtains comprehensive growth information, and displays the obtained comprehensive growth information on the display interface (23) of the upper computer and the display interface (4) of the lower computer, thereby completing the detection of the comprehensive growth information of the crop.

6. The detection method of the crop growth detection device based on canopy temperature and light information according to claim 5, characterized in that: Collecting crop canopy high-throughput information specifically involves parallel acquisition of high-throughput information of the crop's entire growth cycle through the information collection module: A temperature and humidity sensor (15) is used to obtain environmental temperature and humidity information, a light intensity sensor (1) is used to obtain solar radiation intensity, a depth camera (17) is used to obtain a three-dimensional image of the crop canopy, a visible light camera (20) is used to obtain a visible light image of the crop canopy, a multispectral camera (13) is used to obtain a spectral image of the crop canopy, and a thermal infrared camera (19) is used to obtain a thermal infrared image of the crop canopy.

7. The detection method of the crop growth detection device based on canopy temperature and light information according to claim 5, characterized in that: The edge computing process specifically includes: removing the image background and noise of the three-dimensional image, visible light image, spectral image and thermal infrared image acquired by the information acquisition module, extracting the target image, and then acquiring the processed crop image information and storing it in the microcontroller (12).

8. The detection method of the crop growth detection device based on canopy temperature and light information according to claim 5, characterized in that: The construction of the crop three-dimensional model specifically comprises the following steps: Spatial registration: obtaining a three-dimensional image, a visible light image, a spectral image and a thermal infrared image of the crop canopy with a calibration plate (22) collected by the information collection module, and performing spatial coordinate registration; Image Fusion: Visible light-multi-spectral fusion: The spectral image is unmixed using the non-negative matrix decomposition method, and the end-member spectral curve is obtained by adding the sparse constraint of the abundance matrix to the unmixing. According to the spectral absorption curve of the visible light image and the end-member spectral curve obtained by unmixing, the mixing coefficient of each pixel in the image is solved by constrained quadratic programming using the function library in MATLAB. The end-member spectral curve obtained by unmixing and the mixing coefficient of each pixel are linearly mixed to obtain a visible light-multi-spectral fusion image with high spatial resolution and multi-spectral image. High-dimensional image fusion: Use thermal infrared images to obtain edge and temperature target areas, and obtain target areas from visible light-multispectral fusion images at the same time. Select the generative adversarial network as the fusion model, take the visible light-multispectral target area and thermal infrared target area images as input, and use the structural similarity index and peak signal-to-noise ratio as part of the loss function to guide model learning; by designing a weighted fusion strategy, assign weights to the input image contributions to generate a fused image, and evaluate the image quality by calculating the structural similarity, peak signal-to-noise ratio and spectral distortion of the fused image, while performing visual evaluation to ensure the integrity of the image information; sharpen the fused image to obtain a high-dimensional vector data image; Image-point cloud integration: By unifying the three-dimensional image data and high-dimensional vector data images into a reference coordinate system, the high-dimensional vector data images and the three-dimensional image position information are aligned. The high-dimensional vector data images and the three-dimensional images are matched with each other using a stereo matching algorithm. The pixel information of each pixel point of the high-dimensional vector data image is mapped to the three-dimensional image through the position relationship. The three-dimensional image is combined with the pixel information of each point of the high-dimensional vector data image to establish a connection between the spatial position points of the crop and the pixels of the high-dimensional vector data image. The image and point cloud are integrated to obtain an image-point cloud image.

9. The detection method of the crop growth detection device based on canopy temperature and light information according to claim 8, characterized in that: The spatial registration specifically comprises: acquiring an information acquisition module to acquire a three-dimensional image, a visible light image, a spectral image and a thermal infrared image of a crop canopy with a calibration plate (22); a visible light camera (20), a thermal infrared camera (19) and a multi-spectral camera (13) to shoot an image with a calibration plate (22), and acquiring the pixel coordinates of each corner point by using a corner point detection algorithm, and acquiring the camera's internal parameters by knowing the size of the calibration plate (22); and utilizing a feature point matching algorithm to acquire the correspondence between the three-dimensional coordinates and the pixel coordinates in images, images and images, calibrate the camera's external parameters, and perform spatial coordinate registration.

10. The detection method of the crop growth detection device based on canopy temperature and light information according to claim 8, characterized in that: The specific method of obtaining the comprehensive crop growth information is as follows: According to the image-point cloud image, the color texture, crown area, plant height, plant width, leaf inclination, leaf length, leaf width, and leaf area of ​​the crop plants are obtained, thereby obtaining plant type parameters and disease information; According to the image-point cloud image, the temperature distribution of the crop plants is obtained, and the crown-air temperature difference characteristics of the crop plants are obtained in combination with the ambient temperature information; The spectral index in different bands is obtained based on the visible light-multispectral image of the crop canopy to obtain information on chlorophyll content, water status and photosynthesis efficiency.

Images