UAV image transmission method and system for forestry natural enemy delivery

Abstract

The present invention relates to the field of image communication technology, and specifically to a method and system for transmitting drone images for forestry natural enemy deployment. The present invention analyzes the changes between adjacent drone images, and reduces the image scale impact caused by drone cruising based on the hierarchical clustering results of pixel optical flow features, thereby accurately analyzing and obtaining background pixels and moving pixels suspected to be pests in each frame of drone images; and then accurately evaluating the pest coefficient of each moving pixel in combination with the consistent motion characteristics of background pixels such as trees, thereby accurately evaluating the pest area in each frame of drone images, thereby reducing the information transmission pressure of a large number of non-pest areas while accurately compressing the pest area without reducing the clarity, and better balancing the resource consumption of image transmission and the risk of image compression distortion.

Description

Technical Field

[0001] The present invention relates to the field of image communication technology, and in particular to a method and system for transmitting images of unmanned aerial vehicles (UAVs) used for the release of natural enemies in forestry. Background Art

[0002] The application of drone technology in forestry management has become increasingly mature and widespread, especially in the field of forest pest control. The use of drone technology to accurately release natural enemy insects is an innovative biological control method to reduce dependence on chemical pesticides and achieve sustainable development of forestry. This method uses drones to collect forestry images, which are then transmitted to a data management platform to analyze pest types and their densities, thereby accurately releasing forest natural enemies for biological control. Therefore, the quality of the transmitted drone images is crucial.

[0003] Due to the complex cruising environment of drones and limited image transmission resources, directly transmitting a large number of images will take up too many resources, and simply using high compression rates for compressed transmission may cause distortion of key information, thereby affecting the analysis of pest types and densities and the accuracy of natural enemy deployment methods. Therefore, how to balance the resource consumption of image transmission and the risk of image compression distortion becomes a key issue. Summary of the invention

[0004] In order to solve the technical problem of the difficulty in balancing the compression and transmission consumption resources of UAV image transmission, the purpose of the present invention is to provide a UAV image transmission method and system for forestry natural enemy delivery. The technical solution adopted is as follows:

[0005] The present invention proposes a method for transmitting unmanned aerial vehicle images for forest natural enemy delivery, the method comprising:

[0006] Acquire several continuous frames of drone images to be transmitted when the forest natural enemy releases the drone in the target forest area;

[0007] According to the change of each frame of drone image relative to the previous frame of drone image, the optical flow vector of each pixel in each frame of drone image and all the changed pixels in each frame of drone image are obtained; hierarchical clustering is performed on all the pixels in each frame of drone image according to the optical flow vector;

[0008] In each frame of drone image, all background pixels and all moving pixels are screened out from all the changing pixels according to the position distribution of the changing pixels and the similarity of the position distribution of the pixels in each cluster at each level in the corresponding hierarchical clustering results; in each frame of drone image, a background optical flow vector is obtained according to the optical flow vectors of all the background pixels, and a self-motion optical flow vector of each moving pixel is obtained according to the difference between the optical flow vector of each moving pixel and the background optical flow vector;

[0009] In each frame of drone image, the pest coefficient of each moving pixel is obtained according to the self-motion optical flow vector of each moving pixel and the fluctuation of the background optical flow vector and the optical flow vector of all the background pixels; the pest area in each frame of drone image is obtained according to the pest coefficient; the pest area in each frame of drone image is losslessly compressed and the non-pest area is losslessly compressed to obtain and transmit the compressed image of each frame of drone image.

[0010] Furthermore, the method for acquiring the changed pixel points includes:

[0011] Take any frame of drone image as the target image, obtain the differential image between the target image and the previous adjacent frame of drone image, take the pixel points corresponding to all non-zero pixel values ​​in the differential image as differential change pixel points, map the differential change pixel points to the target image, and take the pixel points at the corresponding position after mapping as all changed pixel points in the target image.

[0012] Furthermore, the method for obtaining background pixels includes:

[0013] An image coordinate system is constructed with the lower left corner of the main view of each frame of the drone image as the coordinate origin to obtain the position coordinates of each pixel point; in each frame of the drone image, the position coordinate set of all the changed pixel points is taken as a first set, and the position coordinate set of all the pixel points in each cluster at each level in the hierarchical clustering result is taken as a second set;

[0014] The total number of position coordinates in the intersection of the first set and each second set is obtained, and the intersection corresponding to the maximum total number is used as the target intersection; all pixel points in the cluster to which the second set corresponding to the target intersection belongs are used as background pixel points.

[0015] Furthermore, the method for acquiring moving pixels includes:

[0016] The pixel points corresponding to the remaining position coordinates in the first set that do not belong to the target intersection are taken as moving pixel points.

[0017] Furthermore, the method for obtaining the background optical flow vector includes:

[0018] In each frame of drone imagery, the optical flow modulus and optical flow direction of each pixel are obtained based on the optical flow vector, the average of the optical flow modulus of all the background pixels is taken as the background optical flow modulus, and the average of the optical flow directions of all the background pixels is taken as the background optical flow direction; the background optical flow vector is obtained based on the background optical flow modulus and the background optical flow direction.

[0019] Furthermore, the method for obtaining the self-motion optical flow vector includes:

[0020] The background optical flow vector is subtracted from the optical flow vector of each moving pixel to obtain a self-motion optical flow vector of each moving pixel.

[0021] Furthermore, the method for obtaining the pest coefficient includes:

[0022] In each frame of drone image, the background optical flow floating parameter is obtained according to the deviation of the modulus of the optical flow vector of each background pixel point relative to the average level; in each frame of drone image, the difference between the modulus of the self-motion optical flow vector of each moving pixel point and the background optical flow floating parameter is normalized, and the normalized result is used as the first pest parameter of each moving pixel point;

[0023] In each frame of the drone image, a second pest parameter of each moving pixel is obtained according to the direction difference between the direction of the self-motion optical flow vector of each moving pixel and the direction of the background optical flow vector; the direction difference is positively correlated with the second pest parameter;

[0024] The pest coefficient of each of the moving pixel points is obtained according to the first pest parameter and the second pest parameter; the first pest parameter and the second pest parameter are both positively correlated with the pest coefficient.

[0025] Furthermore, the method for obtaining the pest area includes:

[0026] In each frame of drone image, the pixel points whose pest coefficient is greater than a preset threshold are taken as pest pixel points; and the area formed by all the pest pixel points in each frame of drone image is taken as the pest area.

[0027] Furthermore, the preset threshold is 0.75.

[0028] The present invention also proposes a drone image transmission system for the release of forest natural enemies. The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the drone image transmission method for the release of forest natural enemies are implemented.

[0029] The present invention has the following beneficial effects:

[0030] The present invention first obtains several consecutive frames of drone images to be transmitted when a forest natural enemy releases a drone in a cruise target forestry area; according to the change of each frame of the drone image relative to the adjacent previous frame of the drone image, obtains the optical flow vector of each pixel point in each frame of the drone image and all the changed pixel points in each frame of the drone image, so as to prepare for the subsequent motion analysis of the pixel points; hierarchical clustering is performed on all the pixel points in each frame of the drone image according to the optical flow vector, so as to subsequently evaluate the scale impact brought by the drone cruise based on the hierarchical clustering results, and then facilitates in each frame of the drone image, according to the position distribution of the changed pixel points, and the similarity with the position distribution of the pixel points in each cluster under each level in the corresponding hierarchical clustering results, to screen out all background pixel points and all moving pixel points from all the changed pixel points, wherein the moving pixel points are the adjacent frames. The pixels that actually move and change between them are selected, which narrows the analysis scope of the subsequent pest areas to improve the analysis efficiency; in each frame of drone image, the background optical flow vector is obtained according to the optical flow vectors of all background pixels, and the self-motion optical flow vector of each moving pixel is obtained according to the difference between the optical flow vector of each moving pixel and the background optical flow vector; in each frame of drone image, the pest coefficient of each moving pixel is obtained according to the self-motion optical flow vector of each moving pixel, combined with the fluctuation of the background optical flow vector and the optical flow vectors of all background pixels. The pest coefficient is the possibility of the pest corresponding to the pixel, and then the pest area in each frame of drone image can be obtained according to the pest coefficient; the pest area in each frame of drone image is losslessly compressed, and the non-pest area is losslessly compressed to obtain and transmit the compressed image of each frame of drone image. The present invention analyzes the changes between adjacent drone images and reduces the image scale impact caused by drone cruising based on the hierarchical clustering results of pixel optical flow features, thereby accurately analyzing and obtaining background pixels and moving pixels suspected to be pests in each frame of drone image; and then accurately evaluating the pest coefficient of each moving pixel in combination with the consistent motion features of background pixels such as trees, thereby accurately evaluating the pest area in each frame of drone image, thereby reducing the information transmission pressure of a large number of non-pest areas while accurately compressing the pest area without reducing the clarity, and better balancing the resource consumption of image transmission and the risk of image compression distortion. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1A flowchart of a method for transmitting unmanned aerial vehicle images for forestry natural enemy delivery provided by one embodiment of the present invention;

[0033] Figure 2 A schematic structural diagram of a UAV image transmission system for forest natural enemy delivery provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a drone image transmission method and system for forest natural enemy delivery proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0035] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0036] The following is a detailed description of a specific solution of a drone image transmission method and system for forestry natural enemy delivery provided by the present invention in conjunction with the accompanying drawings.

[0037] See also Figure 1 , which shows a flow chart of a method for transmitting drone images for forestry natural enemy delivery provided by an embodiment of the present invention, specifically comprising:

[0038] Step S1, obtaining a plurality of continuous frames of drone images to be transmitted when a forest natural enemy releases a drone in a cruise target forestry area.

[0039] It should be noted that the cruising scenario of the drone in the embodiment of the present invention is that the forestry pests should be in a non-dormant static state so that their activities can be observed, and the activity range of the forestry pests is within the visible area of ​​the drone, such as the tree surface.

[0040] In one embodiment of the present invention, a forest natural enemy deployment drone is first used to cruise a target forestry area to be analyzed and evaluated for whether to deploy and what type of forest natural enemies to deploy, and the drone is specifically controlled to perform a circular cruise in the target forestry area so that the acquisition angle of the image acquisition module integrated on the drone can cover the entire target forestry area as much as possible, wherein the cruising speed is 5m / s, and the cruising video of the forestry area is collected in real time during the cruising process; then the cruising video is frame-processed with a frame rate of 0.5FPS to obtain all continuous frame images, and these continuous frame images are pre-processed such as grayscale as the drone images to be transmitted, so that the drone images can be subsequently compressed and transmitted to the forestry data management platform with high quality for identification and evaluation of the types and densities of pests.

[0041] It should be noted that the total number of frames of drone images is affected by factors such as the size of the target forestry area or the cruising mode, and the number of frames obtained in different situations is different; the implementer can also set the cruising mode and frame processing method by himself, and can also perform other preprocessing on the grayscale image, which are all existing technologies well known to technicians in this field and will not be elaborated here.

[0042] Step S2, according to the changes of each frame of drone image relative to the previous frame of drone image, obtain the optical flow vector of each pixel in each frame of drone image and all the changed pixels in each frame of drone image; hierarchically cluster all the pixels in each frame of drone image according to the optical flow vector.

[0043] It should be noted that, among the several continuous frames of drone images in the embodiment of the present invention, if the earliest frame collected, i.e., the first frame of drone image, has no adjacent previous frame, subsequent compression transmission will not be performed, and analysis and processing will start from the second frame of drone image.

[0044] Considering that pests are in motion under normal circumstances, their corresponding pixel points should change between adjacent frames of drone images, and a high definition needs to be maintained during the subsequent transmission process in order to evaluate the type and density of pests; while areas such as trees or land are usually in a static state, their corresponding pixel points are usually considered to be pixels in the background area, and the definition can be appropriately reduced and the compression rate can be increased during the subsequent transmission process in order to improve the transmission efficiency; therefore, the embodiment of the present invention first analyzes the changes in pixel points between adjacent frames of drone images, and can evaluate the optical flow vector of each pixel point, and can also preliminarily screen out the changed pixel points that have changed in each frame of drone images.

[0045] Considering that the corresponding acquisition perspectives of different drone images are constantly changing during the drone's cruising process, and trees may also change due to interference from external environmental factors such as wind; therefore, these changing pixels may not only be caused by the movement of pests, but may also be caused by factors such as changes in the acquisition perspective due to cruising or changes in the background area due to wind blowing through trees; and the unchanged pixels can be considered to be pixels in some background areas such as trees or land.

[0046] Preferably, in one embodiment of the present invention, taking into account that the optical flow method and the frame difference method can be used to evaluate the changes in images between adjacent frames, any frame of drone image is taken as the target image, and the differential image between the target image and the adjacent previous frame of drone image is obtained, and the pixel points corresponding to all non-zero pixel values ​​in the differential image are used as differential change pixel points, and the differential change pixel points are mapped to the target image, and the pixel points at the corresponding positions after mapping are used as all changed pixel points in the target image; then, for the target image and the adjacent previous frame of drone image, the LK optical flow method is specifically used to evaluate the changes in the pixel points at the same positions in the adjacent frames of drone images, and the optical flow vector of each pixel point in the target image is obtained; and then the optical flow vector of each pixel point in each frame of drone image and all changed pixel points in each frame of drone image can be obtained.

[0047] The optical flow vector reflects the movement change of each pixel in each frame of drone image relative to the pixel in the previous adjacent frame of drone image, providing data support for subsequent further analysis and acquisition of pest areas; the changed pixel points only indicate the pixels at the same position in the grayscale images of adjacent frames of drones that are different, which are not only the movement changes caused by pest movement, but may also be pixel changes caused by other non-pest movements; however, the acquisition of changed pixel points can preliminarily exclude some background areas such as stationary trees or land, thereby improving the efficiency of subsequent analysis.

[0048] Considering that the existence of some changed pixels may be due to different acquisition angles or different scales of drone images, which may cause the pixels in the background area that should be stationary to change; however, the change in the movement direction of the drone cruise has a relatively consistent effect on the movement changes of all pixels between adjacent frames. Specifically, between two adjacent drone images, the optical flow between the corresponding pixels in the stationary background area should have a high consistency; considering that clustering can divide data with similar features into a cluster, and hierarchical clustering can intuitively view the hierarchical relationship and similarity of the data, it does not need to pre-set the number of clusters;

[0049] Therefore, one embodiment of the present invention further hierarchically clusters all pixels in each frame of drone imagery according to the optical flow vector; by hierarchically clustering the pixels in each frame of drone imagery, pixels with similar motion features can be identified, thereby preparing for subsequent screening of background pixels and motion pixels.

[0050] It should be noted that the hierarchical clustering algorithm is an existing technology well known to those skilled in the art. An embodiment of the present invention specifically adopts a bottom-up hierarchical clustering method for clustering. The implementer may also adopt a top-up method, which will not be described in detail here.

[0051] Step S3, in each frame of drone image, according to the position distribution of changed pixels and the similarity with the position distribution of pixels in each cluster at each level in the corresponding hierarchical clustering results, all background pixels and all moving pixels are screened out from all changed pixels; in each frame of drone image, the background optical flow vector is obtained according to the optical flow vectors of all background pixels, and the self-motion optical flow vector of each moving pixel is obtained according to the difference between the optical flow vector of each moving pixel and the background optical flow vector.

[0052] Considering that the hierarchical clustering result can be regarded as a tree structure, as the level changes or the number of clusters increases, the image information contained in each cluster is gradually refined, similar to the gradual increase in image details seen by an observer as he gradually approaches an object from a long distance; therefore, in the tree structure corresponding to the hierarchical clustering result, the path from the root node to the leaf node can be regarded as a change process from a coarse scale to a fine scale;

[0053] Considering that drone cruising may cause changes in the scale of drone images, if the position distribution of pixels in the clustering results at a certain level has a higher degree of overlap with the position distribution of changed pixels in a frame of drone images, it can be approximately considered that the scale of the drone image corresponding to the clustering results at the current level is consistent with the scale of the drone image of the frame, and the motion feature details presented are relatively similar; then, based on the clustering results at this level, the changed pixels in the frame of drone images can be analyzed and evaluated. The changed pixels corresponding to the changes in the pixels that should have been stationary in the background area due to different acquisition perspectives or different drone image scales are usually background pixels; further, moving pixels can be screened out.

[0054] Therefore, the embodiment of the present invention will screen out all background pixels and all moving pixels from all changed pixels in each frame of drone image according to the position distribution of changed pixels and the similarity of the position distribution of pixels in each cluster at each level in the corresponding hierarchical clustering result; the background pixels are pixels that change due to different viewing angles or scales, and they should belong to the stationary background area pixels; and the moving pixels are pixels that actually move and change under the premise of excluding the influence of scale, such pixels may be caused by the movement of pests, or by the swaying of trees caused by external environmental factors such as wind; this step further narrows the analysis scope of the corresponding pixels in the pest area and improves the efficiency of subsequent analysis.

[0055] Preferably, in one embodiment of the present invention, the method for acquiring background pixels includes:

[0056] The image coordinate system is constructed with the lower left corner of the main view of each frame of the drone image as the coordinate origin to obtain the position coordinates of each pixel point; in each frame of the drone image, the position coordinate set of all changed pixels is taken as the first set, and the position coordinate set of all pixels in each cluster at each level in the hierarchical clustering result is taken as the second set;

[0057] The total number of position coordinates in the intersection of the first set and each second set is obtained, and the intersection corresponding to the maximum total number is used as the target intersection; all pixel points in the cluster to which the second set corresponding to the target intersection belongs are used as background pixel points.

[0058] It should be noted that when constructing the image coordinate system of the drone image, the construction method of all image coordinate systems needs to be consistent, and the main viewing direction and image size of the collected drone images must be consistent.

[0059] Preferably, in one embodiment of the present invention, considering that in each frame of drone image, after filtering out all background pixels in the changed pixels, the remaining moving pixels can be filtered out; therefore, the method for obtaining the moving pixels includes:

[0060] The pixel points corresponding to the remaining position coordinates in the first set that do not belong to the target intersection are taken as moving pixel points.

[0061] At this point, all background pixels and all moving pixels in the changing pixels in each frame of drone imagery can be obtained, preparing for further analysis of the pest area.

[0062] Considering that the pixels in the background area are usually stationary or change due to the factors of drone cruising, the possibility that such pixels are the corresponding pixels in the pest area is extremely low, but it provides a certain analysis basis for analyzing the optical flow changes between adjacent frames of drone images when the drone is cruising. It can evaluate the impact of the optical flow changes of drone images during normal cruising, that is, the background optical flow vector, that is, the motion change information that may be generated by all pixels including moving pixels;

[0063] Then, by analyzing the optical flow vector of the moving pixels based on the background optical flow vector, we can evaluate the motion change characteristics of each moving pixel caused by its own motion, that is, the self-motion optical flow vector. Such changes are usually actual motion changes such as tree swaying caused by pest movement or environmental factors such as wind.

[0064] Therefore, in each frame of drone image, the embodiment of the present invention first obtains the background optical flow vector based on the optical flow vectors of all background pixels, and further obtains the self-motion optical flow vector of each moving pixel based on the difference between the optical flow vector of each moving pixel and the background optical flow vector, so as to subsequently further accurately screen out the pixel points corresponding to the pest area from the moving pixels.

[0065] Preferably, in one embodiment of the present invention, the average motion characteristics corresponding to the optical flow vectors of all background pixels in each frame of the drone image are taken into consideration, which can reflect the optical flow change effect of the drone on the drone image when the drone cruises between adjacent frames of the drone image, that is, the generated background optical flow vector; and the optical flow vector is usually in vector form, which is not convenient for directly analyzing the average characteristics, and because the modulus and direction of the optical flow vector can also reflect the motion change characteristics corresponding to the optical flow vector, the optical flow modulus reflects the change or change size of the pixel point, and the optical flow direction reflects the change direction of the pixel point; then the method for obtaining the background optical flow vector includes:

[0066] In each frame of drone imagery, the optical flow modulus and direction of each pixel are obtained based on the optical flow vector, the mean of the optical flow modulus of all background pixels is taken as the background optical flow modulus, and the mean of the optical flow directions of all background pixels is taken as the background optical flow direction; the background optical flow vector is obtained based on the background optical flow modulus and direction.

[0067] It should be noted that obtaining the optical flow modulus and the optical flow direction based on the optical flow vector, and obtaining the optical flow vector based on the optical flow modulus and the optical flow direction are both existing technologies well known to those skilled in the art and will not be described in detail here.

[0068] It should be noted that, in order to facilitate the evaluation of the direction of the optical flow vector, the x-axis in the image coordinate system is uniformly used as a reference in the embodiments of the present invention. For example, the direction coinciding with the positive direction of the x-axis is represented as 0°, the direction coinciding with the negative direction of the x-axis is represented as 180°, and so on. No further details will be given.

[0069] Preferably, in one embodiment of the present invention, the method for obtaining the self-motion optical flow vector includes:

[0070] The optical flow vector of each moving pixel point is subtracted from the background optical flow vector to obtain the self-motion optical flow vector of each moving pixel point; it should be noted that if the optical flow vector of each pixel point has been obtained in step S2, the optical flow vector of the moving pixel point can be directly obtained.

[0071] So far, the background optical flow vector in each frame of the drone image and the self-motion optical flow vector of each moving pixel in it have been obtained.

[0072] Step S4, in each frame of drone image, according to the self-motion optical flow vector of each moving pixel point, combined with the background optical flow vector and the fluctuation of the optical flow vectors of all background pixels, obtain the pest coefficient of each moving pixel point; according to the pest coefficient, obtain the pest area in each frame of drone image; perform lossless compression on the pest area in each frame of drone image, perform lossy compression on the non-pest area, obtain the compressed image of each frame of drone image and transmit it.

[0073] Considering that in each frame of drone image, the optical flow change of each background pixel may fluctuate to a certain extent, that is, the optical flow changes of not all background pixels are completely consistent; and the self-motion optical flow vector of each moving pixel is obtained by analyzing the optical flow vector of the background pixel, so the analysis accuracy of the self-motion optical flow vector may also be affected by certain fluctuations;

[0074] Considering that the smaller the optical flow modulus of the self-motion optical flow vector of the moving pixel is and the closer it is to the optical flow direction of the background optical flow vector, it means that the moving pixel is more likely to be a tree pixel caused by environmental changes, and the less likely it is to be a pest pixel.

[0075] Therefore, in each frame of drone image, the embodiment of the present invention obtains the pest coefficient of each moving pixel point based on the self-motion optical flow vector of each moving pixel point, combined with the background optical flow vector and the fluctuation of the optical flow vectors of all background pixels; the pest coefficient reflects the possibility that each moving pixel point is the pixel point corresponding to the pest, preparing for the subsequent acquisition of the pest area.

[0076] Preferably, in one embodiment of the present invention, considering that the optical flow directions of the moving pixels corresponding to the trees caused by external environmental factors such as wind are usually similar, while the optical flow directions of the moving pixels corresponding to the pests are usually random due to the randomness of the pests' movements; therefore, the vector modulus of the self-motion optical flow vector of each moving pixel is analyzed on the basis of the fluctuation effect, and the possibility of it being a pest pixel can be preliminarily evaluated. The shorter the vector modulus, the closer the change speed of the changing pixel is to the change speed of the background pixel, and the lower the possibility of it being a pest pixel, and vice versa. At the same time, if the direction of the self-motion optical flow vector is closer to the direction of the background optical flow vector, the lower the possibility of it being a pest pixel, and vice versa. The pest coefficient of each moving pixel can be accurately evaluated by combining the two angles of modulus and direction; therefore, the method for obtaining the pest coefficient includes:

[0077] In each frame of drone image, the background optical flow floating parameter is obtained according to the deviation of the modulus of the optical flow vector of each background pixel relative to the average level; in each frame of drone image, the difference between the modulus of the self-motion optical flow vector of each moving pixel and the background optical flow floating parameter is normalized, and the normalized result is used as the first pest parameter of each moving pixel;

[0078] In each frame of drone image, the second pest parameter of each moving pixel is obtained according to the direction difference between the direction of the self-motion optical flow vector of each moving pixel and the direction of the background optical flow vector; the direction difference is positively correlated with the second pest parameter;

[0079] The pest coefficient of each moving pixel is obtained according to the first pest parameter and the second pest parameter; the first pest parameter and the second pest parameter are both positively correlated with the pest coefficient.

[0080] As an example, the pest coefficient is calculated as: ;in, is the pest coefficient of each i-th moving pixel in the A-th frame of the drone image; is an S-type function, used for normalization; is the first pest parameter of each i-th moving pixel in the A-th frame of the drone image; ; is the self-motion optical flow vector of each i-th moving pixel in the A-th frame of the drone image; is the symbol of the vector’s modulus; is the background optical flow floating parameter; is the second pest parameter of each i-th moving pixel in the A-th frame of the drone image; ; is the absolute value symbol; is the direction of the self-motion optical flow vector of each i-th moving pixel in the A-th frame of the drone image; The direction of the background optical flow vector of the drone image in frame A.

[0081] It should be noted that the method for obtaining the background optical flow floating parameter b is: in each frame of drone imagery, the mean modulus of the optical flow vector of each background pixel is used as a floating reference benchmark, and the floating reference benchmark is also the background optical flow modulus of the background optical flow vector; then the absolute value of the difference between the modulus of the optical flow vector of each background pixel and the floating reference benchmark is used as a floating value, and the mean of the floating values ​​of all background pixels is used as the background optical flow floating parameter.

[0082] In the calculation formula of the pest coefficient, the difference between the modulus of the self-motion optical flow vector of the moving pixel point in each frame of the drone image and the background optical flow floating parameter is mapped to the S-shaped function for normalization. The smaller the normalized value, the closer the change speed of each moving pixel point is to the change speed of the background pixel point based on the fluctuation influence, the lower the possibility of it being a pest pixel point, and the lower the first pest coefficient, and vice versa. The smaller the directional difference between the self-motion optical flow vector of the moving pixel point in each frame of the drone image and the background optical flow vector, the lower the possibility of it being a pest pixel point, and the lower the second pest coefficient, and vice versa. Then the first pest coefficient and the second pest coefficient are multiplied and combined and mapped to the S-shaped function for normalization. The smaller the normalized value, the smaller the pest coefficient.

[0083] In other embodiments of the present invention, implementers may also use other normalization functions, or other basic mathematical primitives or related mapping methods to combine the first pest coefficient and the second pest coefficient, which will not be described in detail here.

[0084] After obtaining the pest coefficient of each pixel in each frame of drone image, the pest area in each frame of drone image can be obtained; then the pest area in each frame of drone image is further losslessly compressed, and the non-pest area is losslessly compressed, and the compressed image of each frame of drone image is obtained and transmitted; thereby maintaining a high clarity of the pest area during the subsequent compression and transmission process in order to evaluate the pest type and density, and to increase the compression rate of the non-pest area to alleviate the transmission pressure.

[0085] Preferably, in one embodiment of the present invention, in each frame of drone image, the pixel points whose pest coefficient is greater than a preset threshold are taken as pest pixel points; the area formed by all pest pixel points in each frame of drone image is taken as the pest area; wherein the preset threshold is 0.75, and the implementer may also set the preset threshold by himself.

[0086] After obtaining the pest area in each frame of drone image, each frame of drone image can be compressed and transmitted; in one embodiment of the present invention, a lossless compression algorithm such as PNG or FLAC can be selected for compression of the pest area, and a lossy compression algorithm such as JPEG can be selected for compression of the non-pest area, and then all drone compressed images obtained after mixed compression are transmitted to the forestry data management platform, so that relevant staff can analyze and evaluate the pest types and densities, and thus formulate corresponding biological control strategies, and then forestry natural enemy release drones can be used to release natural enemies.

[0087] To summarize, the present invention first obtains several consecutive frames of drone images to be transmitted, and obtains the optical flow vector of each pixel point and all changed pixel points in each frame of the drone image; then, all the pixel points in each frame of the drone image are hierarchically clustered according to the optical flow vector, so as to filter out all background pixel points and all moving pixel points from all changed pixel points; then, in each frame of the drone image, the background optical flow vector and the self-motion optical flow vector of each moving pixel point are obtained, and then the pest coefficient of each moving pixel point is obtained to obtain the pest area; finally, the pest area is losslessly compressed, and the non-pest area is losslessly compressed, and transmitted after compression. The present invention analyzes the changes between adjacent drone images and reduces the image scale impact caused by drone cruising based on the hierarchical clustering results of pixel optical flow features, thereby accurately analyzing and obtaining background pixels and moving pixels suspected to be pests in each frame of drone image; and then accurately evaluating the pest coefficient of each moving pixel in combination with the consistent motion features of background pixels such as trees, thereby accurately evaluating the pest area in each frame of drone image, thereby reducing the information transmission pressure of a large number of non-pest areas while accurately compressing the pest area without reducing the clarity, and better balancing the resource consumption of image transmission and the risk of image compression distortion.

[0088] The present invention also proposes a UAV image transmission system for forestry natural enemy delivery, please refer to Figure 2 , which shows a structural schematic diagram of a drone image transmission system for forest natural enemy delivery provided by an embodiment of the present invention. The system includes a memory 201, a processor 202, and a computer program 203 stored in the memory 201 and executable on the processor 202. When the processor 202 executes the computer program 203, steps S1-S4 of a drone image transmission method for forest natural enemy delivery are implemented.

[0089] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0090] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A method for transmitting unmanned aerial vehicle images for forest natural enemy deployment, characterized in that: The method comprises: Acquire several continuous frames of drone images to be transmitted when the forest natural enemy releases the drone in the target forest area; According to the change of each frame of drone image relative to the previous frame of drone image, the optical flow vector of each pixel in each frame of drone image and all the changed pixels in each frame of drone image are obtained; hierarchical clustering is performed on all the pixels in each frame of drone image according to the optical flow vector; In each frame of drone image, all background pixels and all moving pixels are screened out from all the changing pixels according to the position distribution of the changing pixels and the similarity of the position distribution of the pixels in each cluster at each level in the corresponding hierarchical clustering results; in each frame of drone image, a background optical flow vector is obtained according to the optical flow vectors of all the background pixels, and a self-motion optical flow vector of each moving pixel is obtained according to the difference between the optical flow vector of each moving pixel and the background optical flow vector; In each frame of drone image, the pest coefficient of each moving pixel is obtained according to the self-motion optical flow vector of each moving pixel, combined with the background optical flow vector and the fluctuation of the optical flow vectors of all the background pixels; the pest area in each frame of drone image is obtained according to the pest coefficient; the pest area in each frame of drone image is losslessly compressed, and the non-pest area is losslessly compressed, and the compressed image of each frame of drone image is obtained and transmitted; The method for obtaining background pixels comprises: An image coordinate system is constructed with the lower left corner of the main view of each frame of the drone image as the coordinate origin to obtain the position coordinates of each pixel point; in each frame of the drone image, the position coordinate set of all the changed pixel points is taken as a first set, and the position coordinate set of all the pixel points in each cluster at each level in the hierarchical clustering result is taken as a second set; Obtain the total number of position coordinates in the intersection of the first set and each second set, and take the intersection corresponding to the maximum total number as the target intersection; take all pixel points in the cluster to which the second set corresponding to the target intersection belongs as background pixel points; The method for acquiring moving pixels comprises: The pixel points corresponding to the remaining position coordinates in the first set that do not belong to the target intersection are taken as moving pixel points; The method for obtaining the self-motion optical flow vector includes: Subtract the background optical flow vector from the optical flow vector of each moving pixel point to obtain a self-motion optical flow vector of each moving pixel point; The method for obtaining the pest coefficient includes: In each frame of drone image, the background optical flow floating parameter is obtained according to the deviation of the modulus of the optical flow vector of each background pixel point relative to the average level; in each frame of drone image, the difference between the modulus of the self-motion optical flow vector of each moving pixel point and the background optical flow floating parameter is normalized, and the normalized result is used as the first pest parameter of each moving pixel point; In each frame of the drone image, a second pest parameter of each moving pixel is obtained according to the direction difference between the direction of the self-motion optical flow vector of each moving pixel and the direction of the background optical flow vector; the direction difference is positively correlated with the second pest parameter; The pest coefficient of each of the moving pixel points is obtained according to the first pest parameter and the second pest parameter; the first pest parameter and the second pest parameter are both positively correlated with the pest coefficient.

2. The method for transmitting unmanned aerial vehicle images for forest natural enemy delivery according to claim 1, characterized in that: The method for acquiring the changed pixel points comprises: Take any frame of drone image as the target image, obtain the differential image between the target image and the previous adjacent frame of drone image, take the pixel points corresponding to all non-zero pixel values ​​in the differential image as differential change pixel points, map the differential change pixel points to the target image, and take the pixel points at the corresponding position after mapping as all changed pixel points in the target image.

3. The method for transmitting unmanned aerial vehicle images for forest natural enemy delivery according to claim 1, characterized in that: The method for obtaining the background optical flow vector includes: In each frame of drone imagery, the optical flow modulus and optical flow direction of each pixel are obtained based on the optical flow vector, the average of the optical flow modulus of all the background pixels is taken as the background optical flow modulus, and the average of the optical flow directions of all the background pixels is taken as the background optical flow direction; the background optical flow vector is obtained based on the background optical flow modulus and the background optical flow direction.

4. The method for transmitting unmanned aerial vehicle images for forest natural enemy delivery according to claim 1, characterized in that: The method for obtaining the pest area comprises: In each frame of drone image, the pixel points whose pest coefficient is greater than a preset threshold are taken as pest pixel points; and the area formed by all the pest pixel points in each frame of drone image is taken as the pest area.

5. The method for transmitting unmanned aerial vehicle images for forest natural enemy delivery according to claim 4, characterized in that: The preset threshold is 0.

75.

6. A drone image transmission system for forestry natural enemy delivery, characterized in that: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of a method for transmitting images of unmanned aerial vehicles for the release of natural enemies in forestry as described in any one of claims 1 to 5 are implemented.

Images