Plant growth distribution state detection method and device

Abstract

The present invention provides a method and device for detecting the distribution state of plant growth, the method comprising: receiving a multispectral plant image, ambient light intensity, point cloud data and a sky image; using the ambient light intensity, point cloud data and sky image to correct the multispectral plant image, and obtain a corrected image pixel brightness value and a corrected reflectivity; according to the corrected image pixel brightness value and the corrected reflectivity, and in combination with the point cloud data and a pre-constructed crop nutrient tissue detection model, obtain a nutrient tissue distribution detection result that characterizes the crop growth distribution state. The present invention uses ambient light intensity, point cloud data and sky image to correct the multispectral plant image to avoid interference to the multispectral plant image due to the tilt, bend and twist of the leaf surface, and detects the crop nutrient tissue distribution according to the corrected spectrum combined with the model, thereby improving the detection accuracy of the plant growth distribution state.

Description

Technical Field

[0001] The present invention relates to the field of computer vision processing technology, and in particular to a method and device for detecting plant growth distribution status. Background Art

[0002] With the development of smart agriculture, visual management and intelligent decision-making are provided for agricultural production, and visual management and intelligent decision-making require a quick and accurate grasp of the growth status of crops. Chlorophyll is an important pigment for photosynthesis in crops and plays a core role in the light absorption process of photosynthesis. The chlorophyll content of crop leaves is an important parameter for judging the photosynthetic rate, growth and nutritional status of crops. Therefore, it is crucial to obtain a spatial three-dimensional model of crops to obtain phenotypic information at key growth nodes of crops, and then combine it with the distribution of chlorophyll content in crops to evaluate the nutrient distribution, nutrient transport and growth status of crops.

[0003] At present, most people use two methods to obtain crop nutrient distribution. One is to use PhenoSpex's FieldScan system to obtain crop hyperspectral data and three-dimensional point cloud data through high-throughput multispectral three-dimensional laser, and calculate crop height, 3D leaf area, projected leaf area, digital biomass, leaf angle, leaf area index, chlorophyll content and green light nutrition indicators. The other is to use MicaSense-RedEdge imaging multispectral equipment, which can collect multispectral image information in the 475nm, 560nm, 668nm, 840nm and 717nm bands through multiple cameras with different bandpass filters, so as to use plane images to detect the distribution of plant phenotypic nutrient tissues.

[0004] However, the various parameters calculated in the first method mentioned above are independent, and there is no connection between the data, which can easily cause errors in nutrient detection. At the same time, the entire system requires huge, expensive and professional large-scale site facilities to support it, and cannot be well applied to actual agricultural production. The system of the other method cannot obtain the three-dimensional spatial model of the crop, and cannot generate the spatial posture information of the crop, the sun, and the measuring device, which can easily cause large differences in the distribution of chlorophyll content. Summary of the invention

[0005] The present invention provides a method and device for detecting the growth distribution state of a plant, which is used to solve the defect of large error in crop nutrient distribution detection in the prior art, and timely and accurately detects the phenotypic nutrient tissue distribution of the plant by acquiring the phenotypic information of the crop and the spatial distribution information of the nutrition.

[0006] The present invention provides a method for detecting plant growth distribution status, comprising: receiving a multispectral plant image, ambient light intensity, point cloud data and a sky image; correcting the multispectral plant image using the ambient light intensity, the point cloud data and the sky image to obtain a corrected image pixel brightness value and a corrected reflectivity; and obtaining a nutrient tissue distribution detection result characterizing the crop growth distribution status according to the corrected image pixel brightness value and the corrected reflectivity, in combination with the point cloud data and a pre-constructed crop nutrient tissue detection model.

[0007] According to a method for detecting plant growth distribution status provided by the present invention, the multispectral plant image is corrected using the ambient light intensity, the point cloud data and the sky image, including: matching the point cloud data with the plant spectral image, and segmenting the point cloud data using a pre-acquired first segmentation mask to obtain crop point cloud data for characterizing a crop part; dividing the crop part in the multispectral plant image into a plurality of regions with the points constituting the crop in the multispectral plant image as the center and in combination with a preset area range; and correcting the crop part in each region according to the ambient light intensity, the crop point cloud data and the sky image to obtain a corrected DN value and a corrected reflectivity.

[0008] According to a method for detecting the plant growth distribution status provided by the present invention, the crop parts in each area are corrected according to the ambient light intensity, the crop point cloud data and the sky image, so as to obtain the corrected image pixel brightness value and the corrected reflectivity, including: for any area, according to the center point and the crop point cloud data, obtaining the average normal vector of the center point in the area; according to the sky image, obtaining the solar radiation angle and the solar azimuth angle corresponding to the area; according to the average normal vector, the solar radiation angle and the solar azimuth angle, obtaining the mirror reflection light vector corresponding to the center point; according to the mirror reflection light vector, the ambient light intensity and the crop point cloud data, obtaining the corrected image pixel brightness value corresponding to the center point; according to the corrected image pixel brightness value corresponding to the center point and the ambient light intensity, correcting the preset center point in the corresponding area to obtain the corrected reflectivity of the corresponding center point.

[0009] According to a method for detecting plant growth distribution status provided by the present invention, the multispectral plant image includes plant spectral images of multiple different channels, and the spectral channel pixels corresponding to the plant spectral images are arranged in a preset arrangement. Before dividing the crop part in the multispectral plant image into multiple areas, it also includes: according to pre-specified channels, the corresponding plant spectral images are combined to generate a color image; according to crop color characteristics, the color image is segmented to generate a second segmentation mask; according to the second segmentation mask, the plant spectral image corresponding to the pre-specified channel is segmented to obtain the crop part in the multispectral plant image.

[0010] According to a plant growth distribution state detection method provided by the present invention, crop point cloud data for characterizing a crop part is obtained, including: projecting the point cloud data along the optical axis direction of a laser radar used to obtain the point cloud data to obtain point cloud reflectivity data; normalizing the point cloud reflectivity data to obtain a point cloud reflectivity distribution map with values ​​distributed in a preset value interval; obtaining a reflectivity distribution map based on the multispectral plant image and the ambient light intensity; performing image matching between the point cloud reflectivity distribution map and the reflectivity distribution map, and segmenting the point cloud data using a pre-acquired first segmentation mask, and extracting the crop part in the point cloud data to obtain crop point cloud data.

[0011] According to a method for detecting plant growth distribution status provided by the present invention, after dividing the crop part in the multispectral plant image into multiple regions, plant phenotypic data for characterizing the crop growth distribution status is obtained, and the plant phenotypic data includes at least one of a leaf surface average leaf inclination parameter, a crop leaf area index, and a crop plant height parameter. The plant phenotypic data is obtained, including: traversing the crop leaf surface in the multispectral image, and selecting the leaf surface edge region according to a preset selection rule, to obtain the average leaf normal vector of the portion where the crop leaf and the stem are connected and the average normal vector of the adjacent stem surface; obtaining the leaf surface average leaf inclination parameter according to the average leaf normal vector of the portion where the crop leaf and the stem are connected and the average normal vector of the adjacent stem surface; and / or, dividing the corresponding region into multiple triangular regions according to the center point, and obtaining the area of ​​each triangular region of the canopy leaf corresponding to the crop point cloud data; obtaining the crop leaf area index according to the area of ​​each triangular region of the canopy leaf corresponding to the crop point cloud data and the pre-acquired canopy minimum circumscribed rectangle area; and / or, obtaining the crop plant height parameter according to the crop point cloud data.

[0012] According to a method for detecting plant growth distribution status provided by the present invention, before receiving a multispectral plant image, ambient light intensity, point cloud data and a sky image, the method includes: using an inertial measurement module to obtain a pitch angle, a roll angle, a yaw angle and an acceleration; obtaining a current measurement posture according to the pitch angle, the roll angle, the yaw angle and the acceleration; and adjusting the current measurement posture according to the multispectral plant image, the ambient light intensity, the point cloud data and the sky image to be obtained.

[0013] The present invention also provides a plant growth distribution status detection device, comprising: an imaging spectrum sensor module for acquiring multi-spectral plant images, an uplink environment spectrum sensor module for acquiring ambient light intensity, a laser radar module for acquiring point cloud data, a fisheye camera module for acquiring sky images and a central processing module, and also includes a program or instruction stored and run on the central processing module, and the program or instruction is executed by the central processing module to perform the steps of the plant growth distribution status detection method as described in any of the above items.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned plant growth distribution state detection methods are implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned plant growth distribution state detection methods are implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned plant growth distribution state detection methods are implemented.

[0017] The plant growth distribution state detection method provided by the present invention utilizes the ambient light intensity, point cloud data and sky images to correct the multispectral plant image to avoid interference to the multispectral plant image due to the tilt, bending and twisting of the leaf surface, and detects the crop nutrient tissue distribution according to the corrected spectrum combination model, thereby improving the detection accuracy of the plant growth distribution state, so as to quickly and accurately obtain the phenotypic data of the crop and the spatial nutrient distribution status of the crop, thereby facilitating the subsequent comprehensive analysis of the growth status of the crop, and fertilizing and watering in a targeted manner according to different crop growth conditions, thereby reducing the waste of chemical fertilizers and environmental pollution.

[0018] The plant growth distribution state detection device provided by the present invention integrates a laser radar, a fisheye camera, an imaging spectrum sensor module and an uplink environmental spectrum sensor module to fuse the corresponding acquired data, corrects the complex chaotic scattering caused by the complex structure of the crop canopy under natural growth conditions and the interference of the reflected spectrum caused by the tilt, bending and twisting of the leaf surface, thereby improving the detection accuracy of the crop nutrient distribution, and quickly and conveniently reconstructs the three-dimensional structure of field crops through the laser radar and IMU modules to obtain the phenotypic information of the crops; in addition, the structure is compact and the equipment is lightweight, which is convenient for application in actual farmland environment conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a schematic diagram of the process of the plant growth distribution state detection method provided by the present invention;

[0021] Figure 2 It is a schematic diagram of the arrangement of spectral channels of the imaging multi-spectral sensor provided by the present invention;

[0022] Figure 3 It is a schematic diagram of data collection provided by the present invention;

[0023] Figure 4 is a schematic diagram of the average normal vector provided by the present invention;

[0024] Figure 5 It is a schematic diagram of the sun irradiation angle provided by the present invention;

[0025] Figure 6 is a schematic diagram of the mirror reflection light vector provided by the present invention;

[0026] Figure 7 This is one of the structural schematic diagrams of the plant growth distribution state detection device provided by the present invention;

[0027] Figure 8 This is the second structural schematic diagram of the plant growth distribution state detection device provided by the present invention;

[0028] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] Figure 1 A schematic flow chart of a method for detecting plant growth distribution status according to the present invention is shown, and the method comprises:

[0031] S11, receives multispectral plant images, ambient light intensity, point cloud data, and sky images;

[0032] S12, using the ambient light intensity, point cloud data and sky image, correcting the multispectral plant image to obtain a corrected image pixel brightness value and a corrected reflectance;

[0033] S13, obtaining a nutrient tissue distribution detection result representing the growth distribution state of the crop according to the corrected image pixel brightness value and the corrected reflectivity, and in combination with the point cloud data and a pre-built crop nutrient tissue detection model.

[0034] It should be noted that S1N in this specification does not represent the order of the plant growth distribution state detection method. Figure 2-Figure 6 The plant growth distribution state detection method of the present invention is described.

[0035] Step S11, receiving a multispectral plant image, ambient light intensity, point cloud data and a sky image.

[0036] It should be noted that the executor of this embodiment is the central processing module. Therefore, before receiving the multispectral plant image, ambient light intensity, point cloud data and sky image, it includes: using the imaging spectral sensor module to obtain the multispectral plant image, and sending the obtained multispectral plant image to the central processing module; using the uplink ambient spectral sensor module to obtain the ambient light intensity, and sending the obtained ambient light intensity to the central processing module; using the laser radar module to obtain point cloud data, and sending the obtained point cloud data to the central processing module; using the fisheye camera module to obtain the sky image, and sending the obtained sky image to the central processing module.

[0037] It should be noted that the multispectral plant image includes plant spectral images of multiple preset channels, and the spectral channel pixels corresponding to the plant spectrum are arranged in a preset order. The preset channel can be set according to the chlorophyll sensitive band and the matching requirements with the laser radar. The preset arrangement can be set according to the preset band and actual design requirements or prior experience. The ambient light intensity includes an ambient spectral image of multiple bands, and the bands of the ambient light intensity correspond to the preset channels one by one.

[0038] In a possible implementation, the preset bands include 450nm, 550nm, 650nm, 680nm, 730nm, 760nm, 810nm, 860nm and 905nm bands, and the corresponding spectral channel pixels are arranged on the CMOS of the imaging sensor module according to a 3×3 rule, referring to Figure 2 The resolution of each band plant spectral image is 680*680, and the frame rate is 90fps.

[0039] In an optional embodiment, before receiving the multispectral plant image, ambient light intensity, point cloud data and sky image, it includes: using an inertial measurement (IMU) module to obtain the pitch angle, roll angle, yaw angle and acceleration; obtaining the current measurement posture according to the pitch angle, roll angle, yaw angle and acceleration; adjusting the current measurement posture according to the multispectral plant image to be obtained, the ambient light intensity, point cloud data and sky image. It should be noted that the IMU module can be built into the laser radar. In addition, when the IMU module uses a gyroscope, it needs to be calibrated before measurement. The calibration method can refer to the existing calibration method, such as picking up the device and circling it three times in the shape of an "8" in space. No further limitation is made here.

[0040] Step S12, using the ambient light intensity, point cloud data and sky image, the multispectral plant image is corrected to obtain the corrected image pixel brightness value and corrected reflectivity.

[0041] In this embodiment, the multispectral plant image is corrected using ambient light intensity, point cloud data and sky image, including: matching the point cloud data with the plant spectral image, and segmenting the point cloud data using a pre-acquired first segmentation mask to obtain crop point cloud data for characterizing the crop part; dividing the crop part in the multispectral plant image into multiple areas with the points constituting the crop in the multispectral plant image as the center and in combination with a preset area range; and correcting the crop part in each area according to the ambient light intensity, the crop point cloud data and the sky image to obtain a corrected DN value and a corrected reflectivity.

[0042] Specifically, crop point cloud data for characterizing crop parts are obtained, including: projecting the point cloud data along the optical axis direction of the laser radar used to obtain the point cloud data to obtain point cloud reflectivity data; normalizing the point cloud reflectivity data to obtain a point cloud reflectivity distribution map with values ​​distributed in a preset value range; obtaining a reflectivity distribution map based on a multispectral plant image and ambient light intensity; performing image matching on the point cloud reflectivity distribution map and the reflectivity distribution map, and using a pre-acquired first segmentation mask to segment the point cloud data, and extracting the crop part in the point cloud data to obtain crop point cloud data.

[0043] It should be noted that, based on the multispectral plant image and the ambient light intensity, a reflectance distribution map is obtained, including: taking the point constituting the crop in the multispectral plant image as the center, and combining with a preset area range, dividing the crop part in the multispectral plant image into multiple areas; for each area, respectively, obtaining the reflectance based on the multispectral plant image and the ambient light intensity; and obtaining a reflectance distribution map based on the reflectance corresponding to each area.

[0044] The reflectivity is expressed as:

[0045]

[0046] Among them, R Band (x, y) represents the reflectance of the point (x, y) in the plant spectrum image of the band band, DN Band (x, y) represents the image pixel brightness (DN) value of the point (x, y) in the corresponding band environmental spectrum image, Mask (x, y) represents the mask value of the point (x, y), DNs Band The corresponding band uplink environmental spectral sensor data specifically includes the values ​​of each band of the corresponding uplink environmental spectral sensor, and the bands of the uplink environmental spectral sensor correspond to the bands of the imaging spectral sensor one by one. It should be noted that the mask value can be determined according to the second segmentation mask below, which will not be repeated here.

[0047] In addition, the first segmentation mask can be obtained by referring to the second segmentation mask below, which will not be repeated here.

[0048] refer to Figure 3 ,Since the bending and wrinkles of the leaves are distributed in different areas of the leaves, and the complex ,structure of the crop canopy under natural growth conditions, it is easy to cause cluttered ,scattering, it is necessary to divide the crop part in the multispectral plant image into multiple ,areas to correct the reflectance spectrum image of each area separately, which reduces interference and improves the ,accuracy of crop nutrition diagnosis.

[0049] In an optional embodiment, after the crop part in the multispectral plant image is divided into multiple regions, it includes: obtaining crop phenotypic data characterizing the growth distribution state of the crop. Specifically, the crop phenotypic data includes at least one of the average leaf inclination parameter of the leaf surface, the crop leaf area index and the crop plant height parameter, and the corresponding crop phenotypic data is obtained, including: traversing the crop leaf surface in the multispectral image, and selecting the edge area of ​​the leaf surface according to the preset selection rule, to obtain the average leaf normal vector of the portion where the crop leaf and the stem are connected and the average normal vector of the adjacent stem surface; according to the average leaf normal vector of the portion where the crop leaf and the stem are connected and the average normal vector of the adjacent stem surface, the average leaf inclination parameter of the leaf surface is obtained; and / or, according to the center point, the corresponding area is divided into multiple triangular areas, and the area of ​​each triangular area of ​​the crop point cloud data corresponding to the canopy leaf is obtained; according to the area of ​​each triangular area of ​​the crop point cloud data corresponding to the canopy leaf and the pre-acquired canopy minimum circumscribed rectangle area, the crop leaf area index is obtained; and / or, according to the crop point cloud data, the crop plant height parameter is obtained.

[0050] More specifically, the average leaf inclination parameter of the leaf surface is expressed as:

[0051]

[0052] Among them, LA represents the average leaf inclination angle parameter of the leaf surface, represents the average leaf normal vector of the part where the crop leaf connects to the stem, Represents the average normal vector of adjacent stem surfaces. It should be added that the preset selection rule can be set according to actual design requirements or prior experience, for example, a 3×2 pixel area can be selected.

[0053] The leaf area index of crops is expressed as:

[0054]

[0055] Among them, LAI represents the crop leaf area index, ST i It represents the area of ​​each triangular region of the crop point cloud data corresponding to the canopy leaves, and SR represents the area of ​​each triangular region of the crop point cloud data corresponding to the canopy leaves and the pre-acquired minimum circumscribed rectangle area of ​​the canopy.

[0056] In addition, it should be noted that the crop height parameters can be directly obtained based on the point cloud data obtained by the lidar, which will not be further explained here.

[0057] In addition, according to the ambient light intensity, crop point cloud data and sky image, the crop parts in each area are corrected respectively to obtain the corrected image pixel brightness value and corrected reflectivity, including: for any area, according to the center point and crop point cloud data, the average normal vector of the center point in the area is obtained; according to the sky image, the solar radiation angle and solar azimuth angle of the corresponding area are obtained; according to the average normal vector, solar radiation angle and solar azimuth angle, the mirror reflection light vector of the corresponding center point is obtained; according to the mirror reflection light vector, ambient light intensity and crop point cloud data, the corrected image pixel brightness value of the corresponding center point is obtained; according to the corrected image pixel brightness value and ambient light intensity of the corresponding center point, the preset center point in the corresponding area is corrected to obtain the corrected reflectivity of the corresponding center point. Specifically,

[0058] First, for any area, the average normal vector of the center point in the area is obtained according to the center point and the crop point cloud data, including: for any area, the area is divided into multiple triangular areas according to the center point; the normal vector of each triangle is obtained according to the point cloud data and the center point; the average normal vector of the center point is obtained according to the normal vector of each triangle.

[0059] In one possible implementation, reference Figure 4 , assuming that the center point of any region is p n , we get triangle p n p1p2, p n p2p3、p n p3p4 and p n p4p1, and then get triangle p n Normal vector n of p1p2 n12 、Triangle p n Normal vector n of p2p3 n23 、Triangle p n Normal vector n of p3p4 n34 and triangle p n Normal vector n of p4p1 n41 The average normal vector of the center point is expressed as:

[0060]

[0061] In addition, the multispectral plant image includes spectral images of multiple different channels, and the spectral channel pixels corresponding to the spectral images are arranged in a preset order. Before the crop part in the multispectral plant image is divided into multiple areas, it also includes: according to the pre-specified channels, the corresponding spectral images are combined to generate an RGB color image; the color image is segmented according to the crop color characteristics to generate a second segmentation mask; according to the second segmentation mask, the spectral image corresponding to the pre-specified channel is segmented to obtain the crop part in the multispectral plant image. It should be supplemented that the plant spectral image and the point cloud can be matched and segmented using the near-infrared band. The data correspondence between the point cloud and the spectral image is established by compressing the lidar reflectivity point cloud into a two-dimensional plane image and matching it with the spectral image.

[0062] In addition, the sun irradiation angle and sun azimuth angle of the corresponding area are obtained according to the sky image. Figure 5 , the sun's angle, expressed as:

[0063]

[0064]

[0065] θ′ v =tan -1 (r / f)

[0066]

[0067] Among them, θ v represents the angle of sun exposure; a, b and c1 represent parameters, which can be obtained experimentally based on prior experience; θ′ v represents the angle of refracted light, r represents O I S i Length, S i represents the image of the sun on the fisheye camera, f represents the focal length of the lens, x i and i are the centroid coordinates of the image point.

[0068] The solar azimuth angle is expressed as:

[0069]

[0070] Among them, θ h Represents the solar azimuth angle, θ hi represents the azimuth of the image point, x i and i are the centroid coordinates of the image point.

[0071] It should be noted that the steps of obtaining the average normal vector of the center point in each area and obtaining the solar radiation angle and solar azimuth angle of the corresponding area can be performed in any order, and only need to be performed before obtaining the mirror reflection light vector of the corresponding center point.

[0072] Secondly, refer to Figure 6 , according to the average normal vector, the sun's irradiation angle and the sun's azimuth angle, the mirror reflection light vector corresponding to the center point is obtained, including:

[0073] First, according to the sun's illumination angle and the sun's azimuth angle, the incident vector is obtained, which is expressed as:

[0074]

[0075] in, represents the incident vector; θ h Represents the irradiation light and the coordinate axis X d The angle between the sun and the sun is the angle between the sun and the sun. v Represents the irradiation light and the coordinate axis Z d The angle between the sun and the sky is the angle of the sun.

[0076] Second, according to the incident vector and the average normal vector, the mirror reflection light vector is obtained, which is expressed as:

[0077]

[0078] in, represents the specular reflection light vector, represents the incident vector, θ h Represents the solar azimuth angle, θ v represents the sun's angle, represents the average normal vector, Represents the average normal vector With Z d The angle of the axis, Represents the average normal vector With X d The angle of the axis.

[0079] Secondly, according to the specular reflection light vector, ambient light intensity and crop point cloud data, the corrected image pixel brightness value corresponding to the center point is obtained, including:

[0080] First, according to the mirror reflection light vector and Z d Axis, get the mirror reflection light and the device receiving direction Z d The angle ω v , expressed as:

[0081]

[0082] Second, according to the mirror reflection light vector and X d Axis, get the mirror reflection light and the horizontal position of the device X d Angle ω h , expressed as:

[0083]

[0084] It should be noted that the first and second steps may be performed in any order, and only need to be performed before the third step.

[0085] Third, according to the mirror reflection light and the device receiving direction Z d The angle ω v , mirror reflection light and equipment horizontal position X d Angle ω h , ambient light intensity and crop point cloud data, and obtain the pixel brightness (DN) value of the corrected image, expressed as:

[0086]

[0087] Among them, DNc Band (x, y) represents the brightness (DN) value of the corrected image pixel; DN Band Indicates the DN value of the spectral image point (x, y) of the corresponding band; DNs Band represents the uplink environmental spectral sensor data of the corresponding band; σ represents the mirror reflection correction coefficient, which can be obtained by prior experience or experiment; ε[ω v ,ω h , d(x, y)] represents the scattering correction function; c2, d and e represent parameters, which can be obtained by prior experience or experiments; d(x, y) represents the distance from the object point (x, y) to the lidar.

[0088] It should be noted that Among them, t(x, y) is the flight time of the measured point (x, y), and the point cloud data corresponding to the point (x, y) is expressed as PC(x, y, z), where the c3 parameter can be obtained by prior experience or experiment.

[0089] Finally, according to the brightness value of the corrected image pixel and the ambient light intensity of the corresponding center point, the preset center point in the corresponding area is corrected to obtain the corrected reflectivity of the corresponding center point, which is expressed as:

[0090]

[0091] Among them, Rc Band (x, y) represents the corrected reflectance, DNc Band(x, y) represents the brightness (DN) value of the corrected image pixel, Mask (x, y) represents the mask value of the point (x, y), which can be determined according to the second segmentation mask, DNs Band Corresponding to the uplink environmental spectral sensor data in the band, PC (x, y, z, Band) represents the spectral latitude of the point cloud data amplification.

[0092] Step S13, obtaining a nutrient tissue distribution detection result representing the growth distribution state of the crop according to the corrected image pixel brightness value and the corrected reflectivity, in combination with the point cloud data and a pre-constructed crop nutrient tissue detection model.

[0093] In this embodiment, the nutrient tissue distribution detection results include the spatial distribution of crop chlorophyll content, the spatial distribution of nitrogen content and the spatial distribution of crop water content, wherein:

[0094] The spatial distribution of chlorophyll content is expressed as:

[0095]

[0096] Among them, LCC (x, y, z) is the spatial distribution of chlorophyll content, KLCC Band Represents the coefficient of the corresponding band, Rc Band (x, y) represents the corrected reflectance, and gLCC represents a constant. It should be noted that KLCC Band and gLCC can be obtained by prior experience or experiments.

[0097] The spatial distribution of nitrogen content is expressed as:

[0098] N(x,y,z)=KN*LCC(x,y,z)+gN

[0099] Among them, N(x, y, z) represents the spatial distribution of nitrogen content, KN represents the coefficient of the corresponding band, LCC(x, y, z) represents the spatial distribution of chlorophyll content, and gN represents a constant. It should be noted that KN and gN can be obtained by prior experience or experiments.

[0100] The spatial distribution of crop moisture content is expressed as:

[0101]

[0102] Where W(x, y, z) represents the spatial distribution of crop moisture content, KW Band Represents the coefficient of the corresponding band, Rc Band (x, y) represents the corrected reflectivity, and gW represents a constant. It should be noted that KW Band and gW can be obtained by prior experience or experiments.

[0103] In an optional embodiment, after obtaining the nutrient tissue distribution detection results characterizing the growth distribution status of the crop, it also includes: using the Internet of Things module to send the obtained nutrient tissue distribution detection results and / or plant phenotypic data to a remote PC, so as to facilitate timely understanding of the growth status of the crop and provide data support for generating crop fertilization and watering prescription maps.

[0104] In an optional embodiment, after obtaining the nutrient tissue distribution detection result characterizing the growth distribution state of the crop, it also includes: using a data storage module to store and transfer the nutrient tissue distribution detection result and / or plant phenotype data.

[0105] In an optional embodiment, the method further includes: managing charging and discharging of the battery using the power module, and monitoring the battery charge.

[0106] In an optional embodiment, the method further includes: using a GPS positioning module to obtain the GPS position coordinates of the crop to facilitate subsequent precision agricultural management.

[0107] In summary, the embodiments of the present invention utilize ambient light intensity, point cloud data, and sky images to correct multispectral plant images to avoid interference to the multispectral plant images due to the tilt, bending, and twisting of leaf surfaces, and detect the distribution of crop nutritional tissues based on the corrected spectral combination model, thereby improving the detection accuracy of plant growth distribution status, so as to quickly and accurately obtain crop phenotypic data and crop spatial nutritional distribution status, thereby facilitating subsequent comprehensive analysis of crop growth conditions, and targeted fertilization and watering according to different crop growth conditions, thereby reducing fertilizer waste and environmental pollution.

[0108] Figure 7-Figure 8 A structural schematic diagram of a plant growth distribution status detection device is shown, which is characterized in that it includes an imaging spectrum sensor module 1 for acquiring multi-spectral plant images, an uplink environment spectrum sensor module 2 for acquiring ambient light intensity, a laser radar module 3 for acquiring point cloud data, a fisheye camera module 4 for acquiring sky images and a central processing module 5, and also includes a program or instruction stored and run on the central processing module, and the program or instruction is executed by the central processing module as the steps of any of the above plant growth distribution status detection methods.

[0109] In an alternative embodiment, reference Figure 7 , the device also includes: an Internet of Things module 6, which sends the nutritional tissue distribution detection results and / or plant phenotype data obtained by the central processing module 5 to a remote PC.

[0110] In an alternative embodiment, continue to refer to Figure 7The device also includes: a data storage module 7 for storing and transferring the nutrient tissue distribution detection results and / or plant phenotype data.

[0111] In an alternative embodiment, continue to refer to Figure 7 The device also includes: a power module 8, which manages the charging and discharging of the battery and monitors the battery power.

[0112] In an alternative embodiment, continue to refer to Figure 7 The device also includes: a GPS positioning module 9 for acquiring the crop position coordinates.

[0113] refer to Figure 8 In an optional embodiment, the device includes a shell and an imaging spectrum sensor module 1, an uplink environment spectrum sensor module 2, a laser radar 3, an IMU module and a fisheye camera module 4 arranged on the shell, the imaging spectrum sensor module 1 and the laser radar module 3 are arranged on the first surface of the shell, the IMU module is built into the laser radar module 2, the uplink spectrum sensor module 2 and the fisheye camera module 4 are arranged on the second surface of the shell, and the first surface and the second surface are opposite.

[0114] In an optional embodiment, the second surface is further provided with a USB interface 10 , an SD card slot 11 , a display screen 12 and operation buttons 13 .

[0115] In summary, the embodiments of the present invention integrate laser radar, fisheye camera, imaging spectrum sensor module and uplink environmental spectrum sensor module to fuse the corresponding acquired data, correct the complex messy scattering caused by the complex structure of the crop canopy under natural growth conditions and the interference of the reflected spectrum caused by the tilt, bending and twisting of the leaf surface, thereby improving the detection accuracy of crop nutrient distribution, and quickly and conveniently reconstructing the three-dimensional structure of field crops through laser radar and IMU modules to obtain crop phenotypic information; in addition, the structure is compact and the equipment is lightweight, which is convenient for application in actual farmland environment conditions.

[0116] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9As shown, the electronic device may include: a processor 91, a communication interface 92, a memory 93 and a communication bus 94, wherein the processor 91, the communication interface 92 and the memory 93 communicate with each other through the communication bus 94. The processor 91 may call the logic instructions in the memory 93 to execute the plant growth distribution state detection method, which includes: receiving a multi-spectral plant image, ambient light intensity, point cloud data and a sky image; correcting the multi-spectral plant image using the ambient light intensity, the point cloud data and the sky image to obtain a corrected image pixel brightness value and a corrected reflectivity; according to the corrected image pixel brightness value and the corrected reflectivity, combined with the point cloud data and a pre-constructed crop nutrition tissue detection model, obtaining a nutrition tissue distribution detection result characterizing the crop growth distribution state.

[0117] In addition, the logic instructions in the above-mentioned memory 93 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0118] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the plant growth distribution status detection method provided by the above-mentioned methods, and the method includes: receiving multispectral plant images, ambient light intensity, point cloud data and sky images; using the ambient light intensity, point cloud data and sky images to correct the multispectral plant images to obtain corrected image pixel brightness values ​​and corrected reflectances; according to the corrected image pixel brightness values ​​and corrected reflectances, and in combination with the point cloud data and a pre-constructed crop nutritional tissue detection model, obtain nutritional tissue distribution detection results that characterize the crop growth distribution status.

[0119] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the plant growth distribution status detection method provided by the above-mentioned methods, the method comprising: receiving a multispectral plant image, ambient light intensity, point cloud data and a sky image; correcting the multispectral plant image using the ambient light intensity, point cloud data and the sky image to obtain a corrected image pixel brightness value and a corrected reflectivity; obtaining a nutritional tissue distribution detection result characterizing the crop growth distribution status based on the corrected image pixel brightness value and the corrected reflectivity, and in combination with the point cloud data and a pre-constructed crop nutritional tissue detection model.

[0120] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0121] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting plant growth distribution status, characterized in that: include: Receive multispectral plant images, ambient light intensity, point cloud data, and sky images; Correcting the multispectral plant image using the ambient light intensity, the point cloud data and the sky image to obtain a corrected image pixel brightness value and a corrected reflectivity; According to the corrected image pixel brightness value and the corrected reflectivity, and in combination with the point cloud data and a pre-constructed crop nutrient tissue detection model, a nutrient tissue distribution detection result characterizing the crop growth distribution state is obtained; in, Before receiving multispectral plant images, ambient light intensity, point cloud data, and sky images, including: Use the inertial measurement module to obtain pitch angle, roll angle, yaw angle and acceleration; Obtaining a current measurement posture according to the pitch angle, the roll angle, the yaw angle and the acceleration; The current measurement posture is adjusted according to the multispectral plant image to be acquired, the ambient light intensity, the point cloud data and the sky image.

2. The method for detecting plant growth distribution status according to claim 1, characterized in that: The method of correcting the multispectral plant image by using the ambient light intensity, the point cloud data and the sky image comprises: Matching the point cloud data with the multispectral plant image, and segmenting the point cloud data using a pre-acquired first segmentation mask to obtain crop point cloud data for characterizing a crop part; Taking the point constituting the crop in the multispectral plant image as the center and combining with a preset area range, dividing the crop part in the multispectral plant image into a plurality of areas; The crop parts in each area are corrected according to the ambient light intensity, the crop point cloud data and the sky image to obtain a corrected DN value and a corrected reflectivity.

3. The method for detecting plant growth distribution status according to claim 2, characterized in that: According to the ambient light intensity, the crop point cloud data and the sky image, the crop parts in each area are corrected respectively to obtain a corrected image pixel brightness value and a corrected reflectivity, including: For any area, according to the center point and the crop point cloud data, an average normal vector of the center point in the area is obtained; According to the sky image, obtaining the sun illumination angle and the sun azimuth angle corresponding to the area; Obtaining a mirror reflection light vector corresponding to the center point according to the average normal vector, the sun illumination angle and the sun azimuth angle; Obtaining a corrected image pixel brightness value corresponding to the center point according to the specular reflection light vector, the ambient light intensity and the crop point cloud data; According to the brightness value of the corrected image pixel corresponding to the central point and the ambient light intensity, the preset central point in the corresponding area is corrected to obtain the corrected reflectivity of the corresponding central point.

4. The method for detecting plant growth distribution status according to claim 3, characterized in that: The multispectral plant image includes plant spectral images of multiple different channels, and the spectral channel pixels corresponding to the plant spectral images are arranged in a preset arrangement. Before the crop part in the multispectral plant image is divided into multiple regions, the method further includes: According to the pre-specified channels, the corresponding plant spectral images are combined to generate a color image; performing image segmentation on the color image according to crop color features to generate a second segmentation mask; The plant spectral image corresponding to the pre-specified channel is segmented according to the second segmentation mask to obtain the crop part in the multi-spectral plant image.

5. The method for detecting plant growth distribution status according to claim 2, characterized in that: The crop point cloud data used to characterize the crop part is obtained, including: Projecting the point cloud data along the optical axis direction of the laser radar used to obtain the point cloud data to obtain point cloud reflectivity data; Normalizing the point cloud reflectivity data to obtain a point cloud reflectivity distribution map whose values ​​are distributed in a preset value range; Obtaining a reflectance distribution map according to the multispectral plant image and the ambient light intensity; The point cloud reflectivity distribution map is image matched with the reflectivity distribution map, and the point cloud data is segmented using a pre-acquired first segmentation mask, and the crop part in the point cloud data is extracted to obtain crop point cloud data.

6. The method for detecting plant growth distribution status according to claim 2, characterized in that: After dividing the crop part in the multispectral plant image into a plurality of regions, obtaining plant phenotypic data for characterizing the growth distribution state of the crop, wherein the plant phenotypic data includes at least one of a leaf average inclination parameter, a crop leaf area index, and a crop plant height parameter, then obtaining the plant phenotypic data includes: Traversing the crop leaf surface in the multispectral plant image, and selecting the edge area of ​​the leaf surface according to a preset selection rule, to obtain the average leaf normal vector of the portion where the crop leaf and the stem are connected and the average normal vector of the adjacent stem surface; Obtaining an average leaf inclination angle parameter of the leaf surface according to an average leaf normal vector of the portion where the crop leaf and the stem are connected and an average normal vector of the adjacent stem surface; and / or, According to the center point, the corresponding area is divided into a plurality of triangular areas, and the area of ​​each triangular area of ​​the canopy leaf corresponding to the crop point cloud data is obtained; Obtaining a crop leaf area index according to the area of ​​each triangular region of the canopy leaf corresponding to the crop point cloud data and the pre-acquired canopy minimum circumscribed rectangle area; and / or, The crop plant height parameters are obtained according to the crop point cloud data.

7. A plant growth distribution state detection device, characterized in that: It includes an imaging spectral sensor module for acquiring multi-spectral plant images, an uplink environmental spectral sensor module for acquiring ambient light intensity, a laser radar module for acquiring point cloud data, a fisheye camera module for acquiring sky images and a central processing module, and also includes a program or instruction stored and run on the central processing module, wherein the program or instruction is executed by the central processing module according to the steps of the plant growth distribution state detection method as described in any one of claims 1 to 6.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the plant growth distribution status detection method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the plant growth distribution status detection method according to any one of claims 1 to 6 are implemented.

Images