Plant phenotype light spectrum compensation device, method, electronic device and storage medium

Abstract

The plant phenotype spectrum compensation device, method, electronic equipment and storage medium provided by the application belong to the imaging technical field and comprise: a carrying main body for carrying a sensor array, a multispectral camera and a main control device; the sensor array is used for collecting spectrum data and light intensity data and sending the spectrum data and the light intensity data to the main control device; the multispectral camera is used for collecting multispectral images of plant phenotypes and sending the multispectral images to the main control device; and the main control device is used for performing intensity compensation correction on the multispectral images based on the spectrum data and the light intensity data. The plant phenotype spectrum compensation device, method, electronic equipment and storage medium provided by the application can use the spectrum data and the light intensity data of the environment at the image collection time under varying light environments, and then perform real-time compensation correction on the collected images, thereby improving the accuracy of spectrum high-throughput acquisition and overcoming errors caused by light and environmental changes.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of imaging technology, in particular to a plant phenotype spectrum compensation device, method, electronic device and storage medium. BACKGROUND

[0002] In recent years, spectral imaging technology has become a research hotspot in the field of plant phenomics at home and abroad.

[0003] The multispectral imaging system is widely used in unmanned aerial vehicles, phenotyping robots and orbital phenotyping platforms and other automated high-throughput plant phenotyping platforms. Under different platforms and different environments, the operation mode and data post-processing scheme of the multispectral imaging system during continuous acquisition are obviously different, and the demand for environmental compensation information and compensation correction also has great differences.

[0004] However, most multispectral cameras are used in short-time high-altitude long-distance scenarios, and the correction method for multispectral images is not accurate in near-ground environments. SUMMARY

[0005] The plant phenotype spectrum compensation device, method, electronic device and storage medium provided by the present application solve the problem of inaccurate correction of multispectral images in near-ground environments in the prior art, improve the accuracy of spectral high-throughput acquisition, and overcome errors caused by changes in light and environment.

[0006] The present application provides a plant phenotype spectrum compensation device, comprising:

[0007] a sensor array, a multispectral camera, a main control device and a carrying body;

[0008] The carrying body is used to carry the sensor array, the multispectral camera and the main control device;

[0009] The sensor array is used to collect spectral data and light intensity data, and send the spectral data and the light intensity data to the main control device;

[0010] The multispectral camera is used to collect multispectral images of plant phenotypes, and send the multispectral images to the main control device;

[0011] The main control device is used to perform intensity compensation correction on the multispectral images based on the spectral data and the light intensity data.

[0012] According to the plant phenotype spectrum compensation device provided by the present application, the sensor array comprises a plurality of spectral sensors and a plurality of light intensity sensors, and the main control device comprises an MCU.

[0013] The plurality of spectral sensors are specifically configured to continuously collect spectral data of the light environment and send the spectral data to the MCU.

[0014] The plurality of light intensity sensors are specifically configured to continuously collect light intensity data of the light environment and send the light intensity data to the MCU.

[0015] The MCU is configured to determine target spectral data in the spectral data and target light intensity data in the light intensity data in the case that the variation of the spectral data is not less than a spectral threshold or the variation of the light intensity data is not less than a light intensity threshold.

[0016] The target spectral data in the spectral data and the target light intensity data in the light intensity data are determined according to a preset time interval.

[0017] The device is further configured to generate a trigger signal and a timestamp sequence at a preset frequency, send the trigger signal to the multispectral camera, and send the timestamp sequence to the multispectral camera, wherein the trigger signal is used to trigger the multispectral camera to collect a multispectral image, and the timestamp sequence corresponds to the multispectral image in a one-to-one manner.

[0018] The master control device further comprises a host computer, and communication between the MCU and the host computer is realized based on ESP-NOW wireless communication.

[0019] The host computer is configured to receive the target spectral data, the target light intensity data and the timestamp sequence sent by the MCU, and a multispectral image sent by the multispectral camera, and add the timestamp sequence to the multispectral image.

[0020] The multispectral image is intensity-compensated and corrected according to the target spectral data and the target light intensity data.

[0021] The host computer is built-in with a spectral radiation correction model.

[0022] The host computer is specifically configured to input the target spectral data and the target light intensity data into the spectral radiation correction model to obtain radiation correction parameters output by the spectral radiation correction model.

[0023] The multispectral image is intensity-compensated and corrected by using the correction parameters.

[0024] The spectral radiation correction model is obtained by training a preset neural network model based on a sample data set with a radiation correction parameter label.

[0025] The plant phenotype spectrum compensation device provided by the application further comprises a power supply device;

[0026] The power supply device comprises a battery pack, a voltage reduction module, a communication module, and a display screen.

[0027] The battery pack supplies power to the MCU and the sensor array through the voltage reduction module.

[0028] The communication module is configured to send the power information of the battery pack to the MCU.

[0029] The display screen is configured to display the power information of the battery pack.

[0030] The application further provides a plant phenotype spectrum compensation method, comprising the following steps:

[0031] receiving a multispectral image of a plant phenotype collected by a multispectral camera, and receiving spectrum data and light intensity data sent by a sensor array;

[0032] performing intensity compensation correction on the multispectral image based on the spectrum data and the light intensity data.

[0033] The plant phenotype spectrum compensation method provided by the application performs intensity compensation correction on the multispectral image based on the spectrum data and the light intensity data, and comprises the following steps:

[0034] In a case where it is determined that the change amount of the spectrum data is greater than a spectrum threshold value, or it is determined that the change amount of the light intensity data is greater than a light intensity threshold value, target spectrum data is determined in the spectrum data, and target light intensity data is determined in the light intensity data; or,

[0035] According to a preset time interval, the target spectrum data is determined in the spectrum data, and the target light intensity data is determined in the light intensity data.

[0036] According to the target spectrum data and the target light intensity data, intensity compensation correction is performed on the multispectral image.

[0037] The plant phenotype spectrum compensation method provided by the application performs intensity compensation correction on the multispectral image based on the target spectrum data and the target light intensity data, and comprises the following steps:

[0038] The target spectrum data and the target light intensity data are input into a spectrum radiation correction model to obtain radiation correction parameters output by the spectrum radiation correction model.

[0039] The multispectral image is subjected to intensity compensation correction by using the correction parameters.

[0040] The spectral radiation correction model is obtained by training a preset neural network model based on a sample data set with a radiation correction parameter label.

[0041] The spectral radiation correction model is constructed based on a neural network model.

[0042] The present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the plant phenotype spectral compensation method according to any one of the above when executing the program.

[0043] The present application also provides a non-transitory computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the plant phenotype spectral compensation method according to any one of the above.

[0044] The present application also provides a computer program product, comprising a computer program, wherein the computer program is executable on a processor to implement the plant phenotype spectral compensation method according to any one of the above.

[0045] The plant phenotype spectral compensation device, method, electronic device and storage medium provided by the present application can use the spectral data and light intensity data of the environment at the image acquisition time to compensate and correct the collected images in real time under varying light environments, so that the corrected multispectral images are as close as possible to the same light environment, and the accuracy of spectral high-throughput acquisition is improved, and the errors caused by light and environmental changes are overcome. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the present application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0047] Figure 1 is one of the structural schematic diagrams of the plant phenotype spectral compensation device provided by the present application;

[0048] Figure 2 is the second structural schematic diagram of the plant phenotype spectral compensation device provided by the present application;

[0049] Figure 3 is the timing schematic diagram of the trigger signal provided by the present application;

[0050] Figure 4 is the flow schematic diagram of the plant phenotype spectral compensation method provided by the present application;

[0051] Figure 5Fig. 1 is a structural schematic diagram of an electronic device provided by the present application.

[0052] Reference signs:

[0053] 110: sensor array; 120: multi-spectral camera; 130: master device; 140: mounted body. DETAILED DESCRIPTION

[0054] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0055] At present, the acquisition of plant spectral information in the near-ground environment mostly depends on self-propelled phenotyping unmanned vehicles and track-type phenotyping platforms. Compared with traditional unmanned aerial vehicles, the single plant and plot supplement of large-area rapid acquisition of spectral data has the advantages of more complete and detailed resolution order improvement, and the disadvantages of shadow problems caused by large differences in environmental light changes due to long time consumption, calibration difficulties and the need for real-time compensation.

[0056] The existing multi-spectral radiation calibration and compensation scheme generally adopts special material single-frame exposure calibration and uses light intensity sensors to capture solar radiation to exclude the interference of environmental light on data acquisition. However, in the long-time data acquisition process in the near-ground environment, there are inconsistencies in the visual attitude between the light intensity sensor and the worked plant, and the long acquisition time, large differences in solar height variation and obvious cloud changes result in large errors, which are not conducive to accurately extracting plant spectral characteristics and bring more inconvenience to later data processing.

[0057] How to realize accurate correction and accurate data acquisition of multi-spectral high-resolution imaging in long-time near-ground environment has become a difficulty in plant phenotype acquisition technology. The existing technology does not propose a relatively mature real-time environmental compensation correction scheme for high-throughput plant spectral acquisition technology in long-time near-ground complex environment.

[0058] In view of the current lack of near-ground plant phenotype platform spectral real-time compensation correction system scheme and the large error of near-ground spectral imaging data in complex field environment in China, a spectral detection compensation correction device suitable for long-time complex and changeable field and near-ground plant phenotype platform in greenhouse is invented.

[0059] RedEdge-MX five-channel agricultural multispectral is widely used in plant phenotype research, and a dynamic light scattering (DLS) light intensity detection + calibration plate reflectivity calibration scheme is provided for a high-altitude remote unmanned aerial platform for spectral radiation calibration and compensation. The DLS is used to obtain the ambient light intensity during imaging exposure, and the light intensity correction is realized through the dark current correction coefficient, the uplink light radiation flux of each waveband, the downlink light radiation flux and the reference calibration waveband.

[0060] The embodiments of the present application provide a plant phenotype spectrum compensation device, method, electronic equipment and storage medium. Figures 1 to 5 The embodiments of the present application provide a plant phenotype spectrum compensation device, method, electronic equipment and storage medium.

[0061] Figure 1 is one of the structural diagrams of the plant phenotype spectrum compensation device provided by the present application, as Figure 1 shown, comprising:

[0062] a sensor array 110, a multispectral camera 120, a master control device 130 and a carrying body 140;

[0063] The carrying body is used to carry the sensor array, the multispectral camera and the master control device.

[0064] The sensor array is used to collect spectral data and light intensity data, and send the spectral data and the light intensity data to the master control device.

[0065] The multispectral camera is used to collect multispectral images of plant phenotypes, and send the multispectral images to the master control device.

[0066] The master control device is used to compensate and correct the multispectral images based on the spectral data and the light intensity data.

[0067] The carrying body is a near-ground motion device such as an unmanned vehicle or a robot, which is used for near-ground data collection of plants, can carry a large-capacity battery pack, has the characteristics of long working time and high motion stability, and needs to compensate and correct the images in real time because the light environment changes constantly during work.

[0068] The sensor array can include a plurality of sensors for collecting light intensity data and spectral data of a plurality of channels in the environment.

[0069] The master control device can be a device with control function, such as a computer or a single-chip microcomputer, etc., and the master control device can connect the optocoupler isolation input end of the multispectral camera through the general purpose input / output (GPIO) to trigger the multispectral camera to collect images.

[0070] The spectral data is represented by the number of collected photons, with the unit of count; and the light intensity data is the light intensity of the environment, with the unit of Lux.

[0071] The carrying body 140 carries the sensor array 110, the multispectral camera 120 and the main control device 130, drives along a set route on the land to be collected, and in the driving process, the multispectral camera collects images of plants in the near-ground environment under the control of the central control module according to a preset frequency, and sends the collected multispectral images to the main control device; and the sensor array collects light environment parameters such as spectral data and light intensity data in real time, and sends the collected light environment data to the main control device.

[0072] After receiving the multispectral image data and the light environment data, the main control device corresponds the spectral data, the light intensity data and the multispectral image according to the time sequence relationship, and adjusts and corrects the parameters such as brightness, hue and gain of the multispectral image by using the spectral data and the light intensity data, so that the corrected multispectral image is approximately in the same light environment.

[0073] The plant phenotype spectral compensation device provided by the application can use the spectral data and the light intensity data of the environment at the image collection time to compensate and correct the collected images in real time, so that the corrected multispectral image is approximately in the same light environment, and the accuracy of spectral high-throughput acquisition is improved, and the errors caused by light and environmental changes are overcome.

[0074] Optionally, the sensor array comprises a plurality of spectral sensors and a plurality of light intensity sensors; and the main control device comprises an MCU.

[0075] The plurality of spectral sensors are specifically configured to continuously collect spectral data of the light environment, and send the spectral data to the MCU.

[0076] The plurality of light intensity sensors are specifically configured to continuously collect light intensity data of the light environment, and send the light intensity data to the MCU.

[0077] The MCU is configured to determine target spectral data in the spectral data and target light intensity data in the light intensity data in the case that the variation of the spectral data is not less than a spectral threshold value, or the variation of the light intensity data is not less than a light intensity threshold value.

[0078] The target spectral data is determined in the spectral data and the target light intensity data is determined in the light intensity data according to a preset time interval.

[0079] Also configured to generate a trigger signal and a timestamp sequence at a preset frequency, send the trigger signal to the multi-spectrum camera, and use the trigger signal to trigger the multi-spectrum camera to capture a multi-spectrum image; the timestamp sequence corresponds to the multi-spectrum image one-to-one.

[0080] The sensor array includes a plurality of spectral sensors, such as an AS7265 sensor integrating a total of 18 spectral channel bands of AS72651, AS72652, and AS72653, and an AS7341 sensor having 11 spectral channels in the visible-near infrared region, and the bands collected by the two sensors can complement each other to collect as many bands as possible, thereby being able to more accurately compensate and correct the image.

[0081] The sensor array also includes a plurality of high-dynamic digital ambient light sensors and other light intensity sensors, such as a TSL2591 sensor.

[0082] The optical parameters of the sensor array can include spectral band information and sensor installation positions, the spectral band information determines the band that each sensor focuses on acquiring, and the installation of a plurality of spectral sensors with different bands and different light intensity sensors can acquire high-quality and comprehensive spectral data and light intensity data, thereby increasing the robustness of the data.

[0083] Each sensor in the sensor array supports an IIC communication protocol. In addition, considering the detection field of view of the sensor, an outer shell is designed and 3D printed for each sensor, the outer shell is covered with 1 / 4 inch polytetrafluoroethylene (PTFE) patch material, and the sensor can be assembled at any position of the 2020 aluminum profile using M3 screws to select the installation position.

[0084] The microcontroller unit (MCU) can be an ESP32 single-chip microcomputer.

[0085] The smaller the spectral threshold and light intensity threshold, the higher the accuracy; the preset time interval, spectral threshold, and light intensity threshold can be flexibly set according to the accuracy requirement, and the shorter the preset time interval, the higher the accuracy. For example, the preset time interval is set to 1 second.

[0086] Since the sensor array collects spectral data and light intensity data at a very high frequency and sends the collected data to the MCU, there is a certain degree of data redundancy, and the MCU needs to select the target spectral data and target light intensity data required for plant phenotype spectral compensation and correction from the data, and the selection method includes at least the following two methods:

[0087] The MCU is set as a condition trigger, and the change of adjacent two data of the light intensity data and the spectrum data or the cumulative change amount in a period of time is perceived, and if any change amount is not less than a set value, the MCU determines that the light intensity data and the spectrum data at this moment are target spectrum data and target light intensity data, and automatically uploads to the host, or the target spectrum data and the target light intensity data can be uploaded through a set timer.

[0088] The shielding of the cloud layer or the tree shade will cause the change amount of the light intensity data and the spectrum data to be large.

[0089] In addition, the output end of the MCU is connected with the opto-isolating input end of the multi-spectrum camera, the output end of the multi-spectrum camera is connected with the host, the MCU determines the target spectrum data and the target light intensity data from the spectrum data and the light intensity data in a polling manner according to a preset time interval, generates a time stamp sequence, and generates a trigger signal and sends the trigger signal to the multi-spectrum camera.

[0090] The preset frequency can be flexibly set according to actual needs, for example, 1 second / time, and the preset frequency can be synchronized with the acquisition time of the target spectrum data and the target light intensity data in the polling.

[0091] According to the plant phenotype spectrum compensation device provided by the application, effective light environment data is acquired in a polling or condition triggering manner, and a basis is provided for compensation and correction of the multi-spectrum image.

[0092] Optionally, the host control device further comprises a host, and communication between the MCU and the host is realized based on ESP-NOW wireless communication.

[0093] The host is used for receiving the target spectrum data, the target light intensity data and the time stamp sequence sent by the MCU, and the multi-spectrum image sent by the multi-spectrum camera, and adding the time stamp sequence to the multi-spectrum image.

[0094] According to the target spectrum data and the target light intensity data, the multi-spectrum image is subjected to intensity compensation and correction.

[0095] The host can be provided with an operating system, for example, a Linux system.

[0096] For example, the MCU is an ESP32, the ESP8266 receiving end of the ESP32 sends the time stamp sequence, the target spectrum data and the target light intensity data to the host through a UART serial port, the host and the ESP8266 receiving end are connected and communicated through a USB / RS232 to TTL module, and the ESP8266 receiving end and the ESP32 are connected and communicated through an ESP-NOW wireless communication technology.

[0097] The timestamp sequence, target spectrum data and target light intensity data are communicated and parsed with a host in a message transmission protocol, such as a JSON protocol, and contain spectrum data of 18 channels collected by the AS7265, 11 spectrum channel data of the AS7341 and data of two dynamic digital ambient light sensors.

[0098] The multispectral image collected by the multispectral camera is sent to the host through an RJ45 connector, the host reads the ambient spectrum data such as the target spectrum data and the target light intensity data through a Qt host computer to establish a parsing thread, and the target spectrum data, the target light intensity data and the multispectral image are accurately corresponded through the timestamp sequence.

[0099] According to the plant phenotype spectrum compensation device provided by the application, wireless communication between the MCU and the host can effectively avoid shadows caused by wired connection, improve the accuracy of the multispectral image, and also realize efficient transmission.

[0100] Optionally, the host is built-in with a spectrum radiation correction model.

[0101] The host is specifically configured to input the target spectrum data and the target light intensity data into the spectrum radiation correction model, and obtain radiation correction parameters output by the spectrum radiation correction model.

[0102] The multispectral image is intensity-compensated and corrected by using the correction parameters.

[0103] The spectrum radiation correction model is obtained by training a preset neural network model based on a sample data set with a radiation correction parameter label.

[0104] The spectrum radiation correction model can be constructed based on a preset neural network model such as a convolutional neural network (CNN).

[0105] The radiation correction parameters can include gain, synchronous radiation spectrum intensity parameters corresponding to spectrum imaging one by one, field of view angle, incident azimuth angle, atmospheric spectrum absorption parameters and scattering parameters, and captured instantaneous light intensity parameters.

[0106] The sample data set includes sample spectrum data and sample light intensity data corresponding to the radiation correction parameter label.

[0107] A sample data set with a radiation correction parameter label is taken as a training sample, and a plurality of training samples can be obtained.

[0108] The preset neural network model is trained by using a plurality of training samples, model parameters in the preset neural network model are adjusted according to each output result of the preset neural network model, and finally the training process of the preset neural network model is completed, and the trained preset neural network model is used as the spectral radiation correction model.

[0109] For the multispectral image A, the target spectral data and the target light intensity number corresponding to the multispectral image A are input into the spectral radiation correction model, and the radiation correction parameter of the multispectral image A output by the spectral radiation correction model can be obtained, and the intensity compensation correction of the multispectral image A is performed by using the radiation correction parameter.

[0110] According to the plant phenotype spectrum compensation device provided by the application, the radiation correction parameter output by the neural network model is used for compensating and correcting the multispectral image, so that the corrected multispectral image is approximately in the same light environment, the accuracy of spectral high-throughput acquisition is improved, and the error caused by light and environmental changes is overcome.

[0111] Optionally, the device further comprises a power supply device;

[0112] The power supply device comprises a battery pack, a voltage reduction module, a communication module, and a display screen;

[0113] The battery pack supplies power to the MCU and the sensor array through the voltage reduction module;

[0114] The communication module is configured to send the power information of the battery pack to the MCU;

[0115] The display screen is configured to display the power information of the battery pack.

[0116] The power supply device can be two groups of 18650 lithium batteries and a voltage reduction module. The lithium batteries supply direct current to the MCU and the sensor array through the voltage reduction module. The current remaining amount of the lithium batteries is displayed on an organic light-emitting diode (OLED) screen. The communication module sends the power information to the host computer. The power supply device can be charged with a DC 5V through a Type-C interface.

[0117] According to the plant phenotype spectrum compensation device provided by the application, the components in the device are powered by the power supply device, so that the normal operation of the device is ensured.

[0118] Figure 2 is a structural schematic diagram of the plant phenotype spectrum compensation device provided by the application, as shown in Figure 2 comprises:

[0119] The ESP32 expands 1 IIC of the ESP32 to 8 through a TCA9548A multiplexer, greatly improves the expansion performance, and connects the AS7265, the AS7341 and two TSL2591 sensors to the TCA9548 IIC multiplexer through wires.

[0120] Meanwhile, the ESP32 stores configuration information in an internal EEPROM (Electrically Erasable Programmable Read Only Memory), which is used for power-off storage and initialization of the signal configuration.

[0121] In the lithium battery charging and discharging module, the Type-C interface lithium battery charging protection plate output is connected to a step-down module to provide corresponding DC power supply for the sensors and the ESP32, and the OLED display screen is connected to the power detection module through an SPI bus.

[0122] The present application provides a near-ground plant phenotype platform multi-spectral detection compensation correction device, which has the advantages of simple operation, high practicability, high precision, low cost, wide application in near-ground high-throughput spectral data acquisition, and greatly improved effectiveness and accuracy of spectral data in near-ground environment.

[0123] The plant phenotype spectral compensation device can also be used independently as an environmental spectrum sensor, and the ESP32 can emit an 802.11n wireless hotspot.

[0124] The gain value of the spectral radiometric correction model lookup table can be adjusted, and multiple sets of associated parameters can be established. The effective range is adjustable, with a maximum value (default 4095, range 0-4095) and a minimum value (default 0, range 0-4032).

[0125] This invention provides a near-ground plant phenotyping platform multispectral detection compensation and correction device, which combines environmental spectrum with dynamic light data and introduces it into plant spectral imaging radiometric correction. It uses a microcontroller and sensors to realize the external trigger acquisition function of the multispectral camera, with complete timing and closed-loop control.

[0126] Figure 3 This is a timing diagram of the trigger signal provided by the present invention, as shown below. Figure 3 As shown, the trigger timing diagram of the plant phenotypic spectral compensation method device during operation is as follows: When the spectral radiation correction signal is generated, the ESP32 sends the real-time light environment data collected by the four sensors to the ESP8266 receiver. At this time, the ESP8266 immediately generates a GPIO signal to generate a corresponding pulse signal to the optically isolated input interface of the multispectral camera, triggering the spectral camera to expose and collect data. After the acquisition of one frame of data is completed, the multispectral camera sends a corresponding pulse signal to the ESP8266 receiver to indicate that the compensation for that frame has ended.

[0127] The ESP32 achieves high real-time coupling with multispectral camera image acquisition through external trigger mode, and accurately corresponds calibration compensation data with imaging files through timestamp sequence matching. It outputs radiometrically corrected multispectral imaging data in real time through the compensation correction model, and achieves a complete compensation correction closed loop through strict timing control. It is easy to operate and can conveniently and quickly realize real-time spectral imaging correction in the process of high-throughput plant phenotyping. It overcomes the difficulties of spectral radiometric correction caused by changes in ambient light during long-term acquisition operations, greatly reduces the workload and difficulty of post-processing, and improves the acquisition efficiency and data accuracy of high-throughput phenotyping platforms.

[0128] The plant phenotypic spectral compensation method provided by the present invention is described below. The plant phenotypic spectral compensation method described below can be referred to in correspondence with the plant phenotypic spectral compensation device described above.

[0129] Figure 4 This is a schematic flowchart of the plant phenotypic spectral compensation method provided by the present invention, as shown below. Figure 4 As shown, including but not limited to the following steps:

[0130] First, in step S1, the multispectral image of the plant phenotype acquired by the multispectral camera, as well as the spectral data and light intensity data sent by the sensor array, are received.

[0131] The plant phenotype spectrum compensation method provided in the embodiments of the present application can be implemented by an electronic device or a software or a functional module or a functional entity capable of implementing the plant phenotype spectrum compensation method in the electronic device, and the electronic device in the embodiments of the present application includes but is not limited to a master device. It should be noted that the above execution subject does not constitute a limitation on the present application.

[0132] The master device can be a device with a control function, such as a computer or a single-chip microcomputer, etc., and the master device can connect the opto-coupler isolation input end of the multispectral camera through a general-purpose input / output port (GPIO) to trigger the multispectral camera to perform image acquisition.

[0133] During the image acquisition process, the light environment is constantly changing, and therefore it is necessary to compensate and correct the image in real time.

[0134] The sensor array can include a plurality of sensors for acquiring light intensity data and a plurality of channel spectrum data in the environment, respectively.

[0135] The spectrum data is represented by the number of collected photons, and the unit is count; the light intensity data is the illumination intensity of the environment, and the unit is Lux.

[0136] The carrying body 140 carries the sensor array 110, the multispectral camera 120 and the master device 130, and drives along a set route in a land to be acquired, and in the driving process, the multispectral camera acquires images of plants in the near-ground environment under the control of the master module according to a preset frequency, and sends the acquired multispectral images to the master device; and the sensor array acquires light environment parameters such as spectrum data and light intensity data in real time, and sends the acquired light environment data to the master device.

[0137] Further, in step S2, the multispectral image is intensity compensated and corrected based on the spectrum data and the light intensity data.

[0138] After receiving the multispectral image data and the light environment data, the master device corresponds the spectrum data, the light intensity data and the multispectral image according to the time sequence relationship, and adjusts and corrects the brightness, hue, gain and other parameters of the multispectral image by using the spectrum data and the light intensity data, so that the corrected multispectral image is approximately in the same light environment.

[0139] The plant phenotype spectrum compensation method provided in the present application can use the spectrum data and the light intensity data of the environment at the image acquisition time to compensate and correct the acquired image in real time under the changing light environment, so as to make the corrected multispectral image approximately in the same light environment, and improve the accuracy of spectrum high-throughput acquisition and overcome the error caused by light and environment changes.

[0140] Optionally, based on the spectral data and the light intensity data, the multispectral image is corrected for intensity compensation, comprising:

[0141] In a case where the variation of the spectral data is greater than a spectral threshold or the variation of the light intensity data is greater than a light intensity threshold, target spectral data is determined in the spectral data and target light intensity data is determined in the light intensity data; or,

[0142] According to a preset time interval, the target spectral data is determined in the spectral data and the target light intensity data is determined in the light intensity data.

[0143] According to the target spectral data and the target light intensity data, the multispectral image is corrected for intensity compensation.

[0144] Optionally, the correction for intensity compensation of the multispectral image according to the target spectral data and the target light intensity data comprises:

[0145] The target spectral data and the target light intensity data are input into a spectral radiation correction model to obtain a radiation correction parameter output by the spectral radiation correction model;

[0146] The multispectral image is corrected for intensity compensation by using the correction parameter;

[0147] The spectral radiation correction model is obtained by training a preset neural network model based on a sample data set with a radiation correction parameter label;

[0148] The spectral radiation correction model is constructed based on a neural network model.

[0149] The smaller the spectral threshold and the light intensity threshold, the higher the accuracy. The preset time interval, the spectral threshold and the light intensity threshold can be flexibly set according to the accuracy requirement. The shorter the preset time interval, the higher the accuracy. For example, the preset time interval is set to 1 second.

[0150] Since the sensor array collects spectral data and light intensity data at a very high frequency and sends the collected data to the MCU, there is a certain degree of data redundancy. The MCU needs to filter out the target spectral data and the target light intensity data required for plant phenotype spectral compensation correction. The filtering method includes at least the following two methods:

[0151] The MCU senses the variation of the adjacent two data of the light intensity data and the spectral data, or the cumulative variation in a period of time. If any variation is not less than a set value, the MCU determines that the light intensity data and the spectral data at this moment are the target spectral data and the target light intensity data, and automatically uploads them to the host. The timer can also be set to realize timed uploading.

[0152] In addition, the output end of the MCU is connected with the opto-isolator input end of the multi-spectrum camera, the output end of the multi-spectrum camera is connected with the host computer, the MCU determines target spectrum data and target light intensity data from the spectrum data and the light intensity data at a preset time interval, generates a time stamp sequence, and generates a trigger signal and sends the trigger signal to the multi-spectrum camera.

[0153] The spectrum radiation correction model can be constructed based on a preset neural network model such as a CNN.

[0154] The radiation correction parameters can include a gain, a synchronous radiation spectrum intensity parameter corresponding to the spectrum imaging in a one-to-one manner, a field of view angle, an incident azimuth angle, an atmospheric spectrum absorption parameter and a scattering parameter, and a captured instantaneous light intensity parameter.

[0155] The sample data set includes sample spectrum data and sample light intensity data corresponding to the radiation correction parameter label.

[0156] A sample data set with a radiation correction parameter label is taken as a training sample, and a plurality of training samples can be obtained.

[0157] The preset neural network model is trained by using the plurality of training samples, the model parameters in the preset neural network model are adjusted according to each output result of the preset neural network model, and finally the training process of the preset neural network model is completed, and the trained preset neural network model is taken as the spectrum radiation correction model.

[0158] For the multi-spectrum image A, the target spectrum data and the target light intensity data corresponding to the multi-spectrum image A are input into the spectrum radiation correction model, and the radiation correction parameters of the multi-spectrum image A can be obtained, and the multi-spectrum image A is compensated and corrected by using the radiation correction parameters.

[0159] According to the plant phenotype spectrum compensation method provided by the application, by collecting effective light environment data, and then using the radiation correction parameters output by the neural network model to compensate and correct the multi-spectrum image, the corrected multi-spectrum image is approximately in the same light environment, and the accuracy of spectrum high-throughput acquisition is improved, and the error caused by light and environmental changes is overcome.

[0160] Figure 5 is a structural schematic diagram of an electronic device provided by the application, as Figure 5As shown, the electronic device can include a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 complete mutual communication through the communications bus 540. The processor 510 can invoke a logical instruction in the memory 530 to execute a plant phenotype spectrum compensation method, which includes receiving a multispectral image of a plant phenotype collected by a multispectral camera and spectral data and light intensity data sent by a sensor array; and performing intensity compensation correction on the multispectral image based on the spectral data and the light intensity data.

[0161] In addition, the logical instruction in the memory 530 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0162] On the other hand, the present application also provides a computer program product, which includes a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program is executed by a processor, so that the computer can execute the plant phenotype spectrum compensation method provided by the above-mentioned methods, which includes: receiving a multispectral image of a plant phenotype collected by a multispectral camera and spectral data and light intensity data sent by a sensor array; and performing intensity compensation correction on the multispectral image based on the spectral data and the light intensity data.

[0163] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement the plant phenotype spectrum compensation method provided by the above-mentioned methods, which includes: receiving a multispectral image of a plant phenotype collected by a multispectral camera and spectral data and light intensity data sent by a sensor array; and performing intensity compensation correction on the multispectral image based on the spectral data and the light intensity data.

[0164] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0165] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, and the computer software products can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0166] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A plant phenotyping spectral compensation apparatus, characterized by, The device comprises a sensor array, a multispectral camera, a main control device and a carrying body. The carrying body is used for carrying the sensor array, the multispectral camera and the main control device. The sensor array is used for collecting spectral data and light intensity data and sending the spectral data and the light intensity data to the main control device. The multispectral camera is used for collecting multispectral images of plant phenotypes and sending the multispectral images to the main control device. The main control device is used for performing intensity compensation correction on the multispectral images based on the spectral data and the light intensity data. The sensor array comprises a plurality of spectral sensors and a plurality of light intensity sensors, and the main control device comprises an MCU. The plurality of spectral sensors are specifically used for continuously collecting spectral data of a light environment and sending the spectral data to the MCU. The plurality of light intensity sensors are specifically used for continuously collecting light intensity data of the light environment and sending the light intensity data to the MCU. The MCU is used for determining target spectral data in the spectral data and target light intensity data in the light intensity data in a case where a variation of the spectral data is not less than a spectral threshold or a variation of the light intensity data is not less than a light intensity threshold. Or, the target spectral data in the spectral data and the target light intensity data in the light intensity data are determined according to a preset time interval. The device is further used for generating a trigger signal and a time stamp sequence at a preset frequency, sending the trigger signal to the multispectral camera, and one-to-one corresponding the time stamp sequence to the multispectral images. The main control device further comprises a host, and communication between the MCU and the host is realized based on ESP-NOW wireless communication. The host is used for receiving the target spectral data, the target light intensity data and the time stamp sequence sent by the MCU and the multispectral images sent by the multispectral camera. The time stamp sequence is added to the multispectral images. The multispectral images are subjected to intensity compensation correction according to the target spectral data and the target light intensity data. The host is built-in with a spectral radiation correction model.

2. The plant phenotyping light spectrum compensation device of claim 1, wherein, The host is specifically used for inputting the target spectral data and the target light intensity data into the spectral radiation correction model to obtain radiation correction parameters output by the spectral radiation correction model. The multispectral images are subjected to intensity compensation correction by using the correction parameters. The spectral radiation correction model is obtained by training a preset neural network model based on a sample data set with a radiation correction parameter label. The device further comprises a power supply device.

3. The plant phenotyping light spectrum compensation device according to claim 1 or 2, c h a r a c t e r i z e d i n that The power supply device comprises a battery pack, a step-down module, a communication module and a display screen. The battery pack supplies power to the MCU and the sensor array through the step-down module. The communication module is used for sending power information of the battery pack to the MCU. The display screen is used for displaying the power information of the battery pack. The device comprises 4. A method for compensating plant phenotyping spectrum applied to the plant phenotyping spectrum compensation device according to any one of claims 1-3, characterized in that, ​ receive a multispectral image of a plant phenotype collected by a multispectral camera, and spectral data and light intensity data sent by a sensor array; perform intensity compensation correction on the multispectral image based on the spectral data and the light intensity data; perform intensity compensation correction on the multispectral image based on the spectral data and the light intensity data, comprising: in a case where it is determined that a variation of the spectral data is greater than a spectral threshold, or it is determined that a variation of the light intensity data is greater than a light intensity threshold, determine target spectral data in the spectral data and target light intensity data in the light intensity data; or, determine the target spectral data in the spectral data and the target light intensity data in the light intensity data according to a preset time interval; perform intensity compensation correction on the multispectral image according to the target spectral data and the target light intensity data.

5. The plant phenotype spectral compensation method of claim 4, wherein, the performing intensity compensation correction on the multispectral image according to the target spectral data and the target light intensity data, comprising: input the target spectral data and the target light intensity data into a spectral radiation correction model to obtain a radiation correction parameter output by the spectral radiation correction model; perform intensity compensation correction on the multispectral image by using the correction parameter; the spectral radiation correction model is obtained by training a preset neural network model based on a sample data set with a radiation correction parameter label; the spectral radiation correction model is constructed based on a neural network model.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, the processor implements the plant phenotype spectral compensation method of claim 4 or 5 when executing the program.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, the computer program implements the plant phenotype spectral compensation method of claim 4 or 5 when executed by the processor.

Images