Crop plant multispectral image acquisition method and device, electronic equipment and medium

Abstract

The invention discloses a crop plant multispectral image acquisition method and device, electronic equipment and a medium. The acquisition method comprises the following steps of: hovering and shooting a multispectral image of a crop plant by adopting an unmanned aerial vehicle at a low altitude of 2-5m to obtain the multispectral image of the crop plant; performing multiband image registration on the multispectral image in combination with an image processing and mode recognition algorithm, unmanned aerial vehicle flight parameters and multispectral sensor parameters to obtain a registered multispectral image; constructing a vegetation index model based on the registered multispectral image, wherein the vegetation index model is used for extracting a crop plant image; establishing a plant image mask by adopting the vegetation index model in combination with an image processing and mode recognition algorithm; and performing mask calculation by using the plant image mask and the registered multispectral image to obtain the image and spectral information of the plant. According to the acquisition method, the spectral image registration is more accurate, the extracted plant image is clearer and more complete, and the information loss is small.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural informatics, and more specifically, to a method, device, electronic equipment and medium for acquiring multispectral images of crop plants. Background Art

[0002] The development of modern agriculture, especially smart agriculture, requires fast, non-destructive and reliable crop growth information, such as crop growth, nutritional diagnosis, meteorological disasters, field weeds, pests and diseases, etc. However, traditional methods for obtaining field crop information have disadvantages such as time-consuming and labor-intensive, highly destructive, and poor timeliness, which seriously hinder the acquisition of high-throughput crop growth information. Remote sensing technology is widely used in agricultural production monitoring and crop information acquisition due to its wide spatial coverage, high efficiency and non-destructiveness. However, remote sensing has certain limitations, such as low spatial resolution and temporal resolution, susceptibility to weather, and inability to obtain detailed crop information, especially individual crop information. Summary of the invention

[0003] The purpose of the present invention is to provide a new technical solution of a method, device, electronic equipment and medium for acquiring multispectral images of crop plants, which can solve the technical problem that it is difficult to obtain detailed information of crops by remote sensing in the prior art.

[0004] In a first aspect of the present invention, a method for acquiring images and spectral information of crop plants is provided, comprising the following steps: using an unmanned aerial vehicle (UAV) to hover at a low altitude of 2-5 m to obtain a multispectral image of the crop plants; combining image processing and pattern recognition algorithms, UAV flight parameters and multispectral sensor parameters to perform multi-band image registration on the multispectral image to obtain a registered multispectral image; constructing a vegetation index model based on the registered multispectral image, and the vegetation index model is used to extract crop plant images; using the vegetation index model in combination with image processing and pattern recognition algorithms to establish a plant image mask; and performing mask calculation using the plant image mask and the registered multispectral image to obtain the image and spectral information of the plant.

[0005] Optionally, when the multispectral image of the crop plants is obtained by using the drone, the drone is hovered at a low altitude of 2-5 m to shoot.

[0006] Optionally, the method for acquiring images and spectral information of crop plants also includes the following steps: combining multispectral sensor parameters, drone flight parameters, flight environment information and image processing algorithms, and performing at least one of smoothing denoising, normalization processing, vignetting compensation, distortion calibration, and spectral reflectance calculation on each band spectral image of the multispectral image.

[0007] Optionally, when constructing the vegetation index model, an improved vegetation difference index DVI=NIR-B+GR is constructed, and an image improved vegetation difference index grayscale image is calculated; wherein DVI is the improved vegetation difference index, NIR is the near infrared band image, B is the blue band image, G is the green band image, and R is the red band image.

[0008] Optionally, the plant image mask is constructed by combining a threshold algorithm and a small spot removal algorithm.

[0009] Optionally, when calculating the plant image mask, the vegetation difference index image is multiplied by 255. After integerization, the threshold range for extracting the mask is [0, 254], and the threshold for removing small spots by the small spot removal algorithm is 150 - 200 px.

[0010] Optionally, when performing multi-band image registration on the multispectral image, an improved SIFT algorithm is used, and when selecting the best matching point, the ratio is set to 0.75. The multispectral image registration algorithm.

[0011] In a second aspect of the present invention, a device for acquiring images and spectral information of crop plants is provided, comprising: a multispectral image acquisition module, wherein the multispectral image acquisition module of crop plants adopts a drone to hover at a low altitude of 2-5 m to obtain a multispectral image of the crop plants; a registration module, wherein the registration module combines image processing and pattern recognition algorithms, drone flight parameters and multispectral sensor parameters to perform multi-band image registration on the multispectral image to obtain a registered multispectral image; a plant vegetation index model construction module, wherein the plant vegetation index model construction module constructs a vegetation index model based on the registered multispectral image, and the vegetation index model is used to extract crop plant image information; a plant image mask construction module, wherein the plant image mask construction module adopts the vegetation index model in combination with image processing and pattern recognition algorithms to establish a plant image mask; and a crop plant image and spectral information acquisition module, wherein the crop plant image and spectral information acquisition module uses the plant image mask and the registered multispectral image to perform mask calculation to obtain the plant image and spectral information.

[0012] According to a third aspect of the present invention, an electronic device is provided, comprising: a processor and a memory, wherein computer program instructions are stored in the memory, wherein when the computer program instructions are executed by the processor, the processor executes any one of the above methods.

[0013] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes any of the above-mentioned methods.

[0014] According to the method for acquiring the image and spectral information of the crop plants of the embodiment of the present invention, on the one hand, the multispectral image sensor parameters, the flight parameters of the UAV and the flight environment information can be comprehensively utilized, and the image processing and pattern recognition algorithms can be combined to realize accurate multi-band image registration; on the other hand, based on the accurately registered high-definition multispectral image, a vegetation index model that can effectively extract the crop plant image is constructed, and the background such as soil, roads, weeds, and water pipes is eliminated by combining the image processing and pattern recognition algorithms, so as to facilitate the establishment of a complete and accurate plant image mask, and then facilitate the use of the plant image mask to extract clear and complete plant multispectral image information. The crop plant image and spectral information extraction method based on the multispectral image of the UAV provided by the present invention has more accurate spectral image registration, and the extracted plant image is clearer and more complete, with less information loss, which is conducive to the application of the multispectral image of the UAV in the nutrition diagnosis of crops, the identification and diagnosis of pests and diseases, the detailed investigation of waterlogging disasters, and the acquisition of crop phenotypes.

[0015] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0017] Figure 1 is a flow chart of a method for acquiring images and spectral information of crop plants according to an embodiment of the present invention; FIG. 2 ( a ) is a registered image of a rice plant at the booting stage according to an embodiment of the present invention; FIG2( b ) is an extracted plant image according to an embodiment of the present invention; FIG3 (a) is a registered image of a rice plant at the grain filling stage according to an embodiment of the present invention; FIG3( b ) is an extracted plant image according to an embodiment of the present invention; FIG4( a ) is a registered image of rapeseed plants at the seedling stage according to another embodiment of the present invention; FIG4( b ) is an extracted plant image according to another embodiment of the present invention; FIG5( a ) is a registered image of rapeseed plants in flowering stage according to another embodiment of the present invention; FIG5( b ) is an extracted plant image according to another embodiment of the present invention; FIG. 6 ( a ) is a registered image of rapeseed plants at the angle maturity stage according to another embodiment of the present invention; FIG6( b ) is an extracted plant image according to another embodiment of the present invention; Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present invention.

[0018] Reference numerals: Electronic device 200; Processor 201; Memory 202; operating system 2021; application program 2022; Network interface 203; Input device 204; Hard disk 205; Display device 206. DETAILED DESCRIPTION

[0019] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0020] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0021] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0022] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0023] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0024] The following first describes in detail the method for acquiring the image and spectral information of crop plants according to an embodiment of the present invention with reference to the accompanying drawings.

[0025] The method for acquiring images and spectral information of crop plants disclosed in the embodiments of the present application includes the following steps: The multispectral images of crop plants are obtained by hovering an unmanned aerial vehicle at a low altitude of 2-5 m, that is, the multispectral images of crop plants are taken by an unmanned aerial vehicle equipped with a multispectral image sensor. On the one hand, the multispectral images of crop plants taken by unmanned aerial vehicles are utilized in the embodiments of the present application, so that unmanned aerial vehicle remote sensing shows great potential in the acquisition of crop phenotypic information, meteorological disaster monitoring, crop growth monitoring, etc. It can be seen from the embodiments of the present invention that the widespread application of unmanned aerial vehicles is of great significance to the development of modern agriculture, especially smart agriculture. On the other hand, in this embodiment, the multispectral images taken by unmanned aerial vehicles hovering at 2-5 m can clearly express the detailed information of the plants.

[0026] In combination with image processing and pattern recognition algorithms, UAV flight parameters and multispectral sensor parameters, multi-band image registration is performed on the multispectral image to obtain a registered multispectral image; that is, in combination with image processing and pattern recognition algorithms, UAV flight parameters and multispectral sensor parameters, image registration is performed on the multispectral image to obtain a high-definition multispectral image. It can be seen that in this embodiment, multispectral image sensor parameters, low-altitude UAV flight parameters and flight environment information are comprehensively utilized, and image processing and pattern recognition algorithms are combined to achieve accurate multispectral image registration.

[0027] The vegetation index model is constructed based on the registered multispectral images, and the vegetation index model is used to extract crop plant images. In other words, based on the accurately registered high-definition multispectral images, a vegetation index model that can effectively extract plant information is constructed.

[0028] The vegetation index model is combined with image processing and pattern recognition algorithms to establish a plant image mask. In other words, combining image processing and pattern recognition algorithms to remove background such as soil, weeds, and water pipes is conducive to establishing a complete and accurate plant image mask.

[0029] The plant image mask and the registered multispectral image are used to perform mask calculation to obtain the image and spectral information of the plant, and the plant image mask is used to extract clear and complete multispectral image information of the plant.

[0030] Therefore, in this embodiment, on the one hand, the multispectral image sensor parameters, the UAV flight parameters and the flight environment information can be comprehensively utilized, and the image processing and pattern recognition algorithms can be combined to realize accurate multi-band image registration; on the other hand, based on the accurately registered high-definition multispectral image, a vegetation index model that can effectively extract the crop plant image is constructed, and the background such as soil, roads, weeds, and water pipes is eliminated by combining the image processing and pattern recognition algorithms, so as to facilitate the establishment of a complete and accurate plant image mask, and then facilitate the use of the plant image mask to extract clear and complete plant multispectral image information. The crop plant image and spectral information extraction method based on the multispectral image of the UAV provided by the present invention has more accurate spectral image registration, and the extracted plant image is clearer and more complete, with less information loss, which is conducive to the application of the multispectral image of the UAV in the nutrition diagnosis of crops, the identification and diagnosis of pests and diseases, the detailed investigation of waterlogging disasters, and the acquisition of crop phenotypes.

[0031] According to one embodiment of the present application, when a drone is used to obtain a multispectral image of a crop plant, the drone is used to hover at a low altitude of 2-5 m for shooting. That is to say, in this embodiment, a drone equipped with a multispectral image sensor is used to hover at a low altitude (2-5 m) to shoot the multispectral image of the crop plant. In this embodiment, low-altitude drone remote sensing platform technology is used, which has higher flexibility and operability, can take off and land at any time according to weather conditions, can carry various types of sensors and is easy to replace, and can achieve low cost and high throughput to a certain extent.

[0032] In some specific embodiments of the present application, the method for acquiring images and spectral information of crop plants further includes the following steps: combining multispectral sensor parameters, UAV flight parameters, flight environment information and image processing algorithms, and performing at least one of smoothing and denoising, normalization, vignetting compensation, distortion calibration, and spectral reflectance calculation on each band spectral image of the multispectral image. In other words, by combining multispectral sensor parameters and image processing algorithms, and performing smoothing and denoising, normalization, vignetting compensation, distortion calibration, and spectral reflectance calculation on each band spectral image, the calculation speed and accuracy can be improved.

[0033] According to an embodiment of the present application, when constructing a vegetation index model, an improved vegetation difference index DVI=NIR-B+GR is constructed, and an improved vegetation difference index grayscale image is calculated to obtain an image; wherein, DVI is an improved vegetation difference index, NIR is a near-infrared band image, B is a blue band image, G is a green band image, and R is a red band image. In other words, the constructed improved vegetation difference index is DVI=NIR-B+GR. In this embodiment, by adopting the above-mentioned improved vegetation difference index, plant information can be effectively expressed, the difference between plants and background objects such as soil, roads, weeds, and water pipes can be expanded, and the accuracy and efficiency of crop plant extraction can be effectively improved.

[0034] In some specific embodiments of the present application, a plant image mask is constructed by combining a threshold algorithm and a small spot removal algorithm. In this embodiment, the threshold algorithm can accurately distinguish between plants and background, extract plant information, and remove background information. The small spot removal algorithm can effectively remove sporadic outliers. The two algorithms can make the constructed plant image mask more accurate and complete.

[0035] According to one embodiment of the present application, when calculating the plant image mask, the vegetation difference index image is multiplied by 255. After integerization, the threshold range for extracting the mask is [0, 254], and the threshold for small spots removed by the small spot removal algorithm is 150-200 px. In other words, by integerizing the vegetation difference index image and extracting values ​​in the range of [0, 254], most of the information of the retained plants can be effectively extracted and background information can be removed. Selecting a small spot threshold of 150-200 px can remove most of the abnormal values ​​outside the plants, which can effectively improve the accuracy, completeness and efficiency of crop plant extraction.

[0036] According to one embodiment of the present application, when performing multi-band image registration on a multispectral image, an improved SIFT algorithm is used, and when screening the best matching point, the ratio is set to 0.75. That is to say, in this embodiment, the image registration algorithm is an improved SIFT algorithm, and the improved SIFT algorithm has higher performance in multispectral image registration and more accurate registration. When screening the best matching point, the ratio is set to 0.75, and more and more matching registration point pairs can be searched, so that the accuracy of multi-band image registration is higher, the obtained registered image is clearer, and the accuracy and efficiency of crop plant extraction can be effectively improved.

[0037] Optionally, the main equipment and software used in the method for acquiring images and spectral information of crop plants include (1) DJI Phantom 4 drone; (2) multispectral sensor: 1 RGB imaging sensor, 5 monochrome (blue, green, red, red edge, near infrared) sensors; (3) image processing server; (4) Python, Python-OpenCV.

[0038] The embodiments of the present application are described in detail below in conjunction with specific embodiments.

[0039] Example 1 Example 1 involves the extraction of plant images and spectral information at the booting and filling stages of rice, wherein the multispectral drone images of the tested rice were collected from the experimental field of the Institute of Agricultural Information of Jiangsu Academy of Agricultural Sciences, which is located at 32.03°N and 118.87°E. The experiment is a waterlogging stress regulation experiment, and the rice varieties are Ningxiangjing No. 9 and Nanjing 5718. Ningxiangjing No. 9 and Nanjing 5718 are cultivated in half in each cultivation pool.

[0040] The steps of extracting plant images and spectral information of rice at the booting stage and the grain filling stage in Example 1 include: (1) During the rice booting and grain filling stages, a DJI Phantom 4 multispectral drone was used to hover at an altitude of 2 to 3 m to capture multispectral images of rice plants.

[0041] (2) Combining multispectral sensor parameters, UAV flight parameters, flight environment information and image processing algorithms, the spectral image of each band is smoothed, de-noised, normalized, subjected to vignetting compensation, distortion calibration, and spectral reflectance calculation.

[0042] (3) Image registration of multispectral images is performed by combining image processing algorithms, UAV flight parameters, and multispectral sensor parameters.

[0043] (4) Construct the improved vegetation difference index DVI = NIR-B + GR, and calculate the grayscale image of the improved vegetation difference index. (5) Combine the threshold algorithm and small spot removal algorithm to extract the rice plant image mask.

[0044] (6) The rice plant image mask and multispectral image are used to perform mask calculation to obtain the rice plant image and spectral information. The algorithm flow is as follows: Figure 1 shown.

[0045] The registered images and the images of rice plants at the booting stage and the grain filling stage are shown in Figures 2 and 3. It can be seen from the figures that this method can clearly and completely extract the plant information of rice at the booting stage and the grain filling stage.

[0046] Example 2 Example 2 involves the extraction of plant images and spectral information of rapeseed at different growth stages, wherein the multispectral drone images of the rapeseed for testing were collected from the rapeseed experimental field of the Institute of Agricultural Information of Jiangsu Academy of Agricultural Sciences, which is located at 32.03°N and 118.87°E. The experiment is a waterlogging stress regulation pool planting experiment, and the rapeseed varieties are Yangyou 12 and Ningza 1818. The transplanting density is 1.2×105 plants / hectare, the row spacing is 0.4 m, and the plant spacing is 20 cm. Yangyou 12 and Ningza 1818 are transplanted in half in each cultivation pool.

[0047] The steps of extracting images and spectral information of rapeseed plants at different growth stages in Example 2 include: (1) During the rapeseed seedling, flowering and horn maturity stages, a DJI Phantom 4 multispectral drone was used to capture multispectral images of rapeseed plants at a hovering altitude of 2-3 m.

[0048] (2) Combining multispectral sensor parameters, UAV flight parameters, flight environment information and image processing algorithms, the spectral image of each band is smoothed, de-noised, normalized, subjected to vignetting compensation, distortion calibration, and spectral reflectance calculation.

[0049] (3) Image registration of multispectral images is performed by combining image processing algorithms, UAV flight parameters, and multispectral sensor parameters.

[0050] (4) Construct the improved vegetation difference index DVI = NIR-B + GR, and calculate the grayscale image of the improved vegetation difference index. (5) Combine the threshold algorithm and small spot removal algorithm to extract the rapeseed plant image mask.

[0051] (6) Mask calculation is performed using the rapeseed plant image mask and multispectral image to obtain the rapeseed plant image and spectral information. The algorithm flow is as follows: Figure 1 shown.

[0052] The registered images and the extracted rapeseed plant images at the seedling stage, flowering stage and horn maturity stage are shown in Figures 4 to 6. It can be seen from the figures that this method can clearly and completely extract the rapeseed plant information at the seedling stage, flowering stage and horn maturity stage.

[0053] In combination with Example 1 and Example 2, the robustness and scalability of the algorithm for images of different crops and different growth stages are described below.

[0054] In Example 1 and Example 2, by extracting plant images and spectral information from multispectral images of rice and rapeseed at different growth stages, it is found that the crop plant information extraction method can be well applied to multispectral images of rice and rapeseed at different growth stages. Figures 2 to 6 are respectively the registered images and extracted plant images of rice booting stage, rice filling stage, rapeseed seedling stage, rapeseed flowering stage and rapeseed horn maturity stage. The main interferences in plant image extraction are soil, road surface, weeds, etc.; the growth process of the two varieties of rice in the early stage of booting is different, and one variety has ears and the other variety has no ears in the same image; rice ears, green leaves and yellow leaves coexist in the rice filling stage, and the characteristics of rice plants are complex, which brings great difficulties to the extraction of image and spectral information; in addition to interferences in the flowering stage of rapeseed, the color, shape, texture and other characteristics of the flowers and leaves of the rapeseed plant itself are quite different; in the rapeseed horn maturity stage, the flowers, leaves and siliques of the rapeseed plant coexist, and their color, shape, texture and other characteristics are more different, which requires higher robustness of the algorithm. Although affected by the environment and the stage of plant development, the acquisition method provided by the embodiment of the present invention can achieve good image matching and plant information extraction accuracy based on the image registration and plant information extraction results of rice and rapeseed at different growth stages. The acquisition method provided by the embodiment of the present invention has good robustness and scalability.

[0055] In the early stage of rice heading, the growth process of the two varieties is different, one variety heading earlier and the other later; during the rice filling period, rice ears, green leaves and yellow leaves coexist, and the characteristics of the rice plant itself are complex, which brings great challenges to the accurate, complete and low information loss extraction of rice plant images and spectral information. During the flowering and horn maturity period of rapeseed, the plant has yellow rapeseed flowers, green leaves, and siliques at the same time. The characteristics of the rapeseed plant itself are complex. It is difficult for the general vegetation index model to accurately remove the background such as soil, weeds, and water pipes, and extract the plant image clearly and completely. The crop plant image and spectral information extraction method provided based on the embodiment of the present invention can accurately extract complete rice and rapeseed plant multispectral images and spectral information, with little information loss, high calculation efficiency, and plant integrity of more than 95%. The experimental results show that this method has high anti-interference ability and scalability, and low information loss, which means that the accuracy and precision of the algorithm are more reliable. The extracted crop plant images and spectral information can be used for crop disease and pest identification and diagnosis, nutritional diagnosis, meteorological disaster investigation, growth monitoring, etc.

[0056] The present application also discloses a device for acquiring images and spectral information of crop plants, including: a multispectral image acquisition module, a registration module, a plant vegetation index model construction module, a plant image mask construction module, and a crop plant image and spectral information acquisition module.

[0057] Specifically, the multispectral image acquisition module of the crop plants uses a drone to hover at a low altitude of 2-5 m to obtain the multispectral image of the crop plants. The registration module combines image processing and pattern recognition algorithms, drone flight parameters and multispectral sensor parameters to perform multi-band image registration on the multispectral image to obtain the registered multispectral image. The plant vegetation index model construction module constructs a vegetation index model based on the registered multispectral image. The vegetation index model is used to extract the image information of the crop plants. The plant image mask construction module uses the vegetation index model combined with image processing and pattern recognition algorithms to establish a plant image mask. The image and spectral information acquisition module of the crop plants uses the plant image mask and the registered multispectral image to perform mask calculation to obtain the image and spectral information of the plant. It can be seen that the method for acquiring the image and spectral information of the crop plants in the embodiment of the present application has the advantages of clearer and more complete extracted plant images and less information loss, so the acquisition device of the present application also has the same advantages, which will not be repeated here.

[0058] The present invention further provides an electronic device 200, comprising: a processor 201 and a memory 202, wherein the memory 202 stores computer program instructions, wherein when the computer program instructions are executed by the processor 201, the processor 201 executes the steps of the method in the above embodiment.

[0059] Furthermore, if Figure 7 As shown, the electronic device 200 further includes a network interface 203 , an input device 204 , a hard disk 205 , and a display device 206 .

[0060] The above-mentioned interfaces and devices can be interconnected through a bus architecture. The bus architecture can include any number of interconnected buses and bridges. Specifically, one or more central processing units 201 (CPUs) represented by the processor 201 and various circuits of one or more memories 202 represented by the memory 202 are connected together. The bus architecture can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits together. It can be understood that the bus architecture is used to achieve connection and communication between these components. In addition to the data bus, the bus architecture also includes a power bus, a control bus, and a status signal bus, which are all well known in the art, so they will not be described in detail herein.

[0061] The network interface 203 can be connected to a network (such as the Internet, a local area network, etc.), obtain relevant data from the network, and can be stored in the hard disk 205.

[0062] The input device 204 can receive various instructions input by the operator and send them to the processor 201 for execution. The input device 204 can include a keyboard or a pointing device (for example, a mouse, a trackball, a touch pad or a touch screen, etc.).

[0063] The display device 206 can display the result obtained by the processor 201 executing the instruction.

[0064] The memory 202 is used to store programs and data necessary for the operation of the operating system 2021, as well as data such as intermediate results during the calculation process of the processor 201.

[0065] It is understood that the memory 202 in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. The memory 202 of the apparatus and method described herein is intended to include, but is not limited to, these and any other suitable types of memory 202.

[0066] In some implementations, the memory 202 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof: an operating system 2021 and application programs 2022 .

[0067] The operating system 2021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 2022 includes various application programs 2022, such as a browser, etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present invention can be included in the application 2022.

[0068] The processor 201 executes the steps of the method according to the above embodiment when calling and executing the application 2022 and data stored in the memory 202, specifically, the program or instructions stored in the application 2022.

[0069] The method disclosed in the above embodiment of the present invention can be applied to the processor 201, or implemented by the processor 201. The processor 201 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 201 or the instruction in the form of software. The above processor 201 can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general processor can be a microprocessor or the processor 201 can also be any conventional processor 201, etc. The steps of the method disclosed in conjunction with the embodiment of the present invention can be directly embodied as a hardware decoding processor to be executed, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 202, and the processor 201 reads the information in the memory 202 and completes the steps of the above method in combination with its hardware.

[0070] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions of the present invention or a combination thereof.

[0071] For software implementation, the technology herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions herein. The software code can be stored in the memory 202 and executed by the processor 201. The memory 202 can be implemented in the processor 201 or outside the processor 201.

[0072] Specifically, the processor 201 is also used to read the computer program and execute the following steps: the method predicts and outputs the answer to the question asked by the user.

[0073] The present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by the processor 201, the processor 201 executes the steps of the method in the above embodiment.

[0074] In the several embodiments provided by the present invention, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0075] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0076] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the sending and receiving methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0077] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will be appreciated by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A method for acquiring images and spectral information of crop plants, characterized in that: The steps include: Use drones to hover at a low altitude of 2-5 m to obtain multispectral images of crop plants; Combining image processing and pattern recognition algorithms, UAV flight parameters and multispectral sensor parameters, performing multi-band image registration on the multispectral image to obtain a registered multispectral image; Constructing a vegetation index model based on the registered multispectral image, wherein the vegetation index model is used to extract crop plant images; The vegetation index model is used in combination with image processing and pattern recognition algorithms to establish a plant image mask; The plant image mask and the registered multispectral image are used to perform mask calculation to obtain the image and spectral information of the plant.

2. The method for acquiring images and spectral information of crop plants according to claim 1, characterized in that: When the multispectral image of the crop plants is obtained by using the drone, the drone is hovered at a low altitude of 2-5 m for shooting.

3. The method for acquiring images and spectral information of crop plants according to claim 1, characterized in that: The following steps are also included: In combination with multispectral sensor parameters, UAV flight parameters, flight environment information and image processing algorithms, at least one of smoothing denoising, normalization processing, vignetting compensation, distortion calibration and spectral reflectance calculation is performed on each band spectral image of the multispectral image.

4. The method for acquiring images and spectral information of crop plants according to claim 1, characterized in that: When constructing the vegetation index model, an improved vegetation difference index DVI=NIR-B+GR is constructed, and an image improved vegetation difference index grayscale image is obtained by calculation; wherein DVI is the improved vegetation difference index, NIR is the near infrared band image, B is the blue band image, G is the green band image, and R is the red band image.

5. The method for acquiring images and spectral information of crop plants according to claim 1, characterized in that: The plant image mask is constructed by combining a threshold algorithm and a small spot removal algorithm.

6. The method for acquiring images and spectral information of crop plants according to claim 5, characterized in that: When calculating the plant image mask, the vegetation difference index image is multiplied by 255. After integerization, the threshold range of the extracted mask is [0, 254], and the threshold of small spots removed by the small spot removal algorithm is 150-200 px.

7. The method for acquiring images and spectral information of crop plants according to claim 1, characterized in that: When performing multi-band image registration on the multispectral image, an improved SIFT algorithm is used, and when screening the best matching point, the ratio is set to 0.

75.

8. A device for acquiring images and spectral information of crop plants, characterized in that: include: A multispectral image acquisition module, wherein the multispectral image acquisition module of the crop plants uses a drone to hover at a low altitude of 2-5 m to obtain a multispectral image of the crop plants; A registration module, wherein the registration module combines the image processing mode and the recognition algorithm, the UAV flight parameters and the multispectral sensor parameters to perform multi-band image registration on the multispectral image to obtain a registered multispectral image; A plant vegetation index model construction module, wherein the plant vegetation index model construction module constructs a vegetation index model based on the registered multispectral image, and the vegetation index model is used to extract crop plant image information; A plant image mask construction module, wherein the plant image mask construction module uses the vegetation index model in combination with image processing and pattern recognition algorithms to establish a plant image mask; The module for acquiring the image and spectral information of crop plants uses the plant image mask and the registered multi-spectral image to perform mask calculation to obtain the image and spectral information of the plant.

9. An electronic device, characterized in that: include: A processor and a memory, wherein computer program instructions are stored in the memory, wherein when the computer program instructions are executed by the processor, the processor is caused to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

Images