Method for positioning dead trees of pine wilt disease based on cloud platform camera detection

Abstract

The application discloses a kind of based on cloud platform camera detection's positioning method of pine wood nematode disease dead tree, including cloud platform and N high-altitude camera, N The high-altitude camera is respectively connected with the cloud platform, reasonably utilize high-altitude camera, through the connection of cloud platform and camera, make the dual combination of software algorithm and image processing, the difficulty of using high-altitude cloud platform video to recognize positioning is solved.Utilize the repeated shooting of camera, while positioning the camera, in combination with its shooting angle and zoom value, the picture of shooting is positioned, then in combination with local content, local accurate positioning is carried out, realizes single, or group, or cluster positioning.The beneficial effect is: improve forest protection auxiliary equipment technical means, reduce the workload of forest protection staff.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of forest pest tree positioning, and particularly relates to a pine wood nematode disease dead tree positioning method based on cloud platform camera detection. BACKGROUND

[0002] The pine wood nematode disease is also called the "cancer" of pine trees, and is a devastating pine disease caused by pine wood nematodes, which can harm more than 50 species of pine and a few non-pine species under natural conditions, causing huge economic losses.

[0003] The high-altitude cloud platform video or vehicle-mounted platform deployed in the forest area is helpful for pine wood nematode disease monitoring and identification. Due to its real-time, traceability and repeatability, it becomes the optimal means for the latest monitoring and prevention. However, the traditional high-altitude cloud platform video algorithm can only identify the dead trees in the photos taken from the high altitude, identify the areas where the dead trees exist, for example, the patent number 202110040607.3, the invention name is a pine wood nematode disease dead tree detection method based on two-stage high-altitude cloud platform video, which discloses the principle of identifying the dead trees in the pictures taken from the high altitude, confirming whether the corresponding shooting area exists the dead trees by identifying whether the pictures exist the dead trees, and realizing the detection. However, for the needs of forest protection work, the pine wood nematode disease needs to be accurately positioned and timely removed. When the staff obtains the existence of dead trees in the area, it cannot solve the need for positioning single trees, clusters and clusters in the existing national standards, and cannot better provide accurate geographical location information to the number of trees. If you want to find the corresponding trees in the jungle, you need to spend a lot of time and physical effort, and the forest protection is difficult, so it is urgent to provide an address location accurate positioning algorithm for single dead pine trees. SUMMARY

[0004] In view of the above technical problems, the application provides a pine wood nematode disease dead tree positioning method based on cloud platform camera detection, which realizes accurate positioning of single dead pine trees and provides more accurate technical support for forest protection detection, greatly reducing the workload of forest protection workers.

[0005] In order to solve the above technical problems, the application adopts the following technical scheme:

[0006] A pine wood nematode disease dead tree positioning method based on cloud platform camera detection, comprising a cloud platform and N high-altitude cameras, wherein N is a positive integer; the key technical point is that the following steps are included:

[0007] S1: arbitrarily select a high-altitude camera C above a pine forest, and set a four-tuple set A collected by the high-altitude camera C and a positioning output set S of pine wood nematode disease trees output by the cloud platform, the four-tuple set includes high-altitude camera number and PTZ orientation data, and initialize the four-tuple set A and the positioning output set S as empty sets;

[0008] S2: the cloud platform controls the high-altitude camera C to arbitrarily take a positioning image T1, and obtains a corresponding four-tuple according to the positioning image T1;

[0009] S3: the cloud platform performs backtracking positioning on the high-altitude camera C according to the four-tuple obtained in step S2;

[0010] S4: the cloud platform controls the high-altitude camera C to arbitrarily take a pine forest image T2, and obtains a panoramic image TQ centered on the pine forest image T2 after detecting, expanding and fusing the edges of the pine forest image T2;

[0011] S5: set a frame selection threshold, and the cloud platform identifies and removes the disease tree frame in the panoramic image TQ according to the frame selection threshold to obtain a detection frame set K;

[0012] S6: generate the geographic location of the withered tree according to the detection frame set K, and output the geographic location corresponding to the detection frame to form a positioning output set S; determine whether the detection is complete, if yes, end the positioning; if not, return to step S4.

[0013] Through the above design, the high-altitude camera is reasonably utilized, the connection of the cloud platform and the camera is made, the double combination of software algorithm and image processing is realized, and the difficulty of recognizing and positioning using high-altitude cloud platform video is solved. In the application, the camera is repeatedly photographed, the camera is positioned, the shooting angle and the zoom value are combined, the region of the photographed picture is positioned, the local content is combined, the local accurate positioning is performed, the single or group or cluster positioning is realized, the forest protection auxiliary equipment technology means is improved, and the workload of the forest protection personnel is reduced.

[0014] When reading the PTZ coordinates, the geotools tool is used to convert them to latitude and longitude with the latitude and longitude of the camera as the coordinate origin.

[0015] When the geographic location of the detection frame is obtained, a third-party navigation platform is combined, a starting point is set, the geographic location of the detection frame is taken as a navigation terminal point, and a multi-species (person / car) forest cutting route is navigated.

[0016] More suggestions are given to the forest protection personnel for forest protection, and the life safety of the forest protection personnel is ensured. Even if the forest protection personnel loses direction in the forest, the rescue personnel can still obtain the work site and route of the forest protection personnel on the same day through the cloud platform, and can quickly lock the area where the lost forest protection personnel may be located.

[0017] Further technical solutions are: in step S3, the ONVIF protocol is used for positioning according to the four-tuple obtained from the positioning image T1.

[0018] In the above manner, the position of the selected high-altitude camera C is confirmed. The ONVIF protocol is prior art and will not be described again.

[0019] Further technical solutions are: the specific steps of step S4 are:

[0020] The preset values are: set image set Pic = []; disease tree detection set C = []; boundary box set B = []; edge movement distance = L, L is less than or equal to the height or width of the camera screen;

[0021] S41: The cloud platform controls the high-altitude camera C to randomly take a pine forest image T2 and places it in the image set Pic;

[0022] S42: The cloud platform detects the disease tree according to the characteristics of the pine wood nematode disease tree and performs frame selection marking to obtain the disease tree detection set C;

[0023] S43: Obtain the coordinates (x1, y1, x2, y2) of each frame in the disease tree detection set C, and determine whether there are coordinate values equal to 0 or equal to the width or height of the image; if so, place the corresponding frame in the boundary box set B, select a boundary box Bi from the boundary box set B, and enter step S44; otherwise, enter step S47;

[0024] S44: The cloud platform controls the high-altitude camera to rotate so that its screen moves L to the edge of the selected boundary box Bi and then takes and recognizes it to obtain the disease tree detection set C1, and places the corresponding moving photograph in the image set Pic;

[0025] S45: Determine whether all frames in the boundary box Bi appear in the disease tree detection set C1; if so, enter step S46; otherwise, return to step S44;

[0026] The judgment standard is whether the coordinates of the boundary box Bi coincide with the image T Bi Edge coordinates, if there is a coincidence, the judgment result is "no"; if there is no coincidence, the judgment result is "yes";

[0027] S46: Determine whether all boundary boxes in the boundary box set B have been moved; if so, enter step S47; otherwise, let i = i + 1 and return to step S44;

[0028] S47: Align and fuse all images in the image set Pic, cover the repeated parts, and generate a panoramic image TQ centered on the pine forest image T2.

[0029] The high-altitude camera cannot predict the shooting picture at each shooting time, and the dead pine trees may appear at the edge of the shooting image, and the dead pine trees may not be shot completely. By using the above scheme, the image frame can be extended in combination with the image boundary box, and a larger panoramic image is obtained, and the panoramic image contains all the pine wilt dead trees in the pine image T2. The panoramic image clearly and completely obtains the picture of the dead pine trees, which is convenient for subsequent frame selection and positioning to avoid omission.

[0030] Preferably, in step S47, the key points are generated using OpenCV, and the RANSAC algorithm is used for alignment and fusion.

[0031] Further technical solutions are: in step S5, the dead tree frame within the frame selection threshold is obtained, and the dead tree frame not in the pine image T2 is removed, and finally the remaining dead tree frame forms a detection frame set K.

[0032] By using the above scheme, the panoramic image is detected for the third time, the appropriate detection frame is generated, and the relative positioning is performed with the detection frame as the center. The center of positioning is the center of the single tree, or the cluster, or the cluster of dead pine trees. The frame selection threshold is set according to the experience value of the frame size of the single tree, or the cluster, or the cluster of pine wilt dead trees.

[0033] Further technical solutions are: the specific steps of step S6 are:

[0034] Let the width and height of the panoramic image be Width and Height respectively;

[0035] S61: The cloud platform obtains the center (K jx , K jy ) and size K area of the detection frame K j in the detection frame set K;

[0036] S62: The cloud platform controls the zoom of the high-altitude camera to perform local shooting of the detection frame, and obtains a local enlarged image T Kj ;

[0037] S63: The cloud platform uses the relative positioning of Onvif to position the new center of the detection frame K j , wherein the PTZ values of the detection frame K j are (K jx / Width), (K jy / Height) and Z1 respectively;

[0038] S64: The cloud platform obtains the current four-tuple of the high-altitude camera, and obtains the current positioning PTZ Kj value;

[0039] S65: the cloud platform combines the storage PTZ in the positioning output set S with the PTZ of the current detection frame obtained in step S64 Kj The position is compared with the value, and if the position is within the set proximity threshold, it is determined that the geographic position of the current pine tree has been obtained, and the next detection frame is detected, and the step S61 is returned until all the detection frames are detected; otherwise, it is considered that it is a new pine tree, and the step S66 is entered;

[0040] S66: the cloud platform generates corresponding geographic position information according to the PTZ of the current detection frame Kj , and saves the geographic position information in the positioning output set S; the next detection frame is detected, and the step S61 is returned until all the detection frames are detected.

[0041] According to the above scheme, according to the position of the detection frame in the panoramic image, the high-altitude camera is used for zooming and shooting again, and the PTZ of the detection frame and the current camera PTZ are obtained by combining the zooming magnification and the center position of the detection frame, and after de-duplication and saving, the position of the detection frame is positioned, and the final output result is obtained. After the above scheme, the single or group or cluster precise position can be obtained, which is beneficial for the forest guard to quickly find the position of the dead tree, greatly saves the search time, and thus the dead tree can be quickly removed to prevent the spread to a larger area.

[0042] A further technical solution is that in step S62, when the detection frame is locally photographed, the calculation formula of the zooming scaling value Z1 of the high-altitude camera is:

[0043] Z1=d / K area =(1 / r)*Width*Height / K area ;

[0044] r is the minimum number of times of the high-altitude camera after zooming to photograph all the scenes of the panoramic image; the value range of r is 2-8 times.

[0045] According to the above scheme, the PTZ value of the detection frame is obtained by using the relationship between the state of the camera in the single shooting of the detection frame and the shooting of the complete panoramic image, and the precise positioning is realized.

[0046] The beneficial effects of the present application are:

[0047] Through three times of detection and three times of positioning, the geographic position information of the single or group or cluster pine wood nematode dead tree is finally realized, so that the AI method is used to guarantee the achievement of the national standard. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1A main flowchart for the present application to precisely locate the dead trees of pine wood nematode disease;

[0049] Figure 2 A positioning schematic diagram for pine forest image T2;

[0050] Figure 3 A flowchart for obtaining a panoramic image according to the pine forest image T2;

[0051] Figure 4 A flowchart for obtaining a panoramic image according to the pine forest image T2; Figure 2 A schematic diagram of the panoramic image obtained according to the pine forest image T2;

[0052] Figure 5 A flowchart for generating the geographic location of the dead tree according to the detection frame;

[0053] Figure 6 A schematic diagram for the aerial camera to shoot the detection frame. DETAILED DESCRIPTION

[0054] The present application will be further described in detail below with reference to the accompanying drawings.

[0055] In specific implementation, as shown in the embodiment, referring to Figs. Figures 1 to 6 , Figure 2 , 4 , 6, a single dead pine tree is located.

[0056] A positioning method for dead trees of pine wood nematode disease based on cloud platform camera detection, comprising a cloud platform and N aerial cameras and M intelligent terminals, wherein M and N are positive integers;

[0057] The N aerial cameras are respectively connected with the cloud platform, for the cloud platform to control the aerial cameras to shoot and obtain aerial camera shooting parameters, wherein the shooting parameters at least include a four-tuple set;

[0058] The cloud platform is connected with a third-party navigation platform, for navigation according to the dead tree position by multiple types (people, vehicles, etc.), to obtain a forest protection navigation route;

[0059] The M intelligent terminals are connected with the cloud platform, for the staff to obtain the dead tree positioning result and the forest protection navigation route from the intelligent terminal;

[0060] The positioning method is performed according to the following steps:

[0061] S1: arbitrarily selecting an aerial camera C above a pine forest, and setting a four-tuple set A collected by the aerial camera C and a positioning output set S of pine wood nematode disease trees output by the cloud platform, the four-tuple set including aerial camera number and PTZ azimuth data, and initializing the four-tuple set A and the positioning output set S as empty sets;

[0062] In the embodiment, the set of quadruples A is exemplified as: A[{“c_id”:”a001”, p:20, t:-10, z:1}, {“c_id”:”a002”, p:30, t:-20, z:2}, {}...];

[0063] Wherein {“c_id”:”a001”, p:20, t:-10, z:1} represents that the dead pine tree is found at the PTZ values of 20 degrees, 20 degrees and 1 times focal length respectively, and the number is a001.

[0064] S2: The cloud platform controls the aerial camera C to randomly shoot a positioning image T1, and obtains a corresponding quadruple according to the positioning image T1;

[0065] S3: The cloud platform performs backtracking positioning on the aerial camera C according to the quadruple obtained in step S2.

[0066] In step S3, the quadruple obtained according to the positioning image T1 is positioned by using the ONVIF protocol.

[0067] S4: The cloud platform controls the aerial camera C to randomly shoot a pine forest image T2, and obtains a panoramic image TQ centered on the pine forest image T2 after detecting, expanding and fusing the edges of the pine forest image T2.

[0068] The specific steps of step S4 are as follows:

[0069] The preset values are: set image set Pic = []; dead tree detection set C = []; boundary box set B = []; edge moving distance = L, L is less than or equal to the height or width of the camera shooting screen;

[0070] S41: The cloud platform controls the aerial camera C to randomly shoot a pine forest image T2 and put it into the image set Pic.

[0071] S42: The cloud platform detects the pine forest image T2 according to the characteristics of the pine wood nematode disease tree and marks the frame selection to obtain the dead tree detection set C.

[0072] S43: Obtain the coordinates (x1, y1, x2, y2) of each frame in the dead tree detection set C, and judge whether there is a coordinate value equal to 0 or equal to the width or height of the image; if yes, put the corresponding frame into the boundary box set B, select a boundary box Bi from the boundary box set B, and enter step S44; otherwise, enter step S47.

[0073] S44: The cloud platform controls the aerial camera to rotate so that its shooting screen moves to the edge of the selected boundary box Bi by L and then shoots and recognizes to obtain a dead tree detection set C1, and the corresponding moving shot is put into the image set Pic.

[0074] S45: judging whether all the boxes in the boundary box Bi appear in the pine tree detection set C1; if yes, going to step S46; otherwise, returning to step S44;

[0075] wherein the judging criterion is whether the coordinates of the boundary box Bi coincide with the image T Bi whether the edge coordinates coincide, if yes, the judging result is “no”; if no, the judging result is “yes”;

[0076] S46: judging whether all the boundary boxes in the boundary box set B have been moved, if yes, going to step S47; otherwise, setting i=i+1 and returning to step S44;

[0077] S47: aligning and fusing all the images of the image set Pic, covering the repeated parts, and generating a panorama TQ centered on the pine image T2.

[0078] In this embodiment, in step S47, the image alignment and fusion use OpenCV to generate key points and use RANSAC algorithm for alignment. The RANSAC algorithm is described in RANSAC, https: / / baike.baidu.com / item / ransac / 10993469?fr=aladdin.

[0079] S5: setting a box selection threshold, and the cloud platform identifies and removes the pine tree boxes in the panorama image TQ according to the box selection threshold, to obtain a detection box set K;

[0080] In step S5, the pine tree boxes within the box selection threshold are obtained, and the pine tree boxes not in the pine image T2 are removed, and finally the remaining pine tree boxes form the detection box set K.

[0081] In this embodiment, the box selection threshold is set according to the experience value of the aerial camera shooting of the single pine nematode disease dead tree. In this embodiment, it is set to 50mmx50mm.

[0082] The target here is to remove excessively small boxes, which generally represent distant targets and cannot really provide accurate geographic location information. The box not in the pine image T2 refers to that the pine tree detection set C1 and the detection box set K are bisection matched to find the boxes of the pine tree detection set C1 in the detection box set K, and all the boxes not in the pine tree detection set C1 are removed. The bisection matching is performed by using the Hungarian algorithm, which is described in detail as follows:

[0083] https: / / baike.baidu.com / item / %E5%8C%88%E7%89%99%E5%88%A9%E7%AE%97%E6%B3%95 / 9089246?fr=aladdin.

[0084] S6: generating the dry tree geographical position according to the detection frame set K, and outputting the geographical position corresponding to the detection frame to form a positioning output set S; determining whether the detection is completed, if yes, ending the positioning; if not, returning to step S4.

[0085] The specific steps of step S6 are as follows:

[0086] Suppose the width and height of the panoramic picture are Width and Height respectively;

[0087] S61: the cloud platform acquires the center (K j , K jx ) and size K jy of the detection frame K area in the detection frame set K;

[0088] S62: the cloud platform controls the zoom of the high-altitude camera to take a local picture of the detection frame, and obtains a local enlarged image T Kj ;

[0089] In step S62, when the detection frame is locally photographed, the calculation formula of the zoom value Z1 of the high-altitude camera is:

[0090] Z1 = d / K area = [(1 / r)*Width*Height] / K area ;

[0091] r is the minimum number of times of zooming after the high-altitude camera takes a panoramic picture of all the scenes; the value range of r is 2-8 times.

[0092] In this embodiment, r = 4.

[0093] S63: the cloud platform uses the relative positioning of Onvif to position the new center of the detection frame K j , wherein the PTZ values of the detection frame K j are (K jx / Width), (K jy / Height) and Z1 respectively;

[0094] S64: the cloud platform acquires the current four-tuple of the high-altitude camera, and obtains the current positioning PTZ Kj value;

[0095] S65: the cloud platform combines the storage positioning PTZ in the positioning output set S and the positioning PTZ of the current detection frame obtained in step S64 Kj The position is compared with the value, and if the position is within the set proximity threshold, it is determined that the geographical position of the current pine tree has been obtained, and the current pine tree is discarded directly. The next detection frame is detected, and step S61 is returned until all the detection frames are detected. Otherwise, it is considered that it is a new pine tree, and step S66 is entered.

[0096] S66: the cloud platform generates corresponding geographical position information according to the positioning PTZ of the current detection frame Kj and saves the geographical position information in the positioning output set S. The next detection frame is detected, and step S61 is returned until all the detection frames are detected.

[0097] The above is only a preferred embodiment of the present application. It should be noted that, for those skilled in the art, several modified and improved technical solutions made without departing from the technical solutions should also be considered to fall within the scope of the present application.

Claims

1. A method for locating dead trees caused by pine wilt disease based on cloud platform camera detection, comprising a cloud platform and N aerial cameras, wherein the N aerial cameras are respectively connected to the cloud platform, and the method is characterized in that According to the following steps: S1: randomly select a high-altitude camera C above the pine forest, and set the four-tuple set A collected by the high-altitude camera C and the positioning output set S output by the cloud platform, the four-tuple set including the high-altitude camera number and PTZ orientation data, and initialize the four-tuple set A and the positioning output set S as empty sets; S2: the cloud platform controls the high-altitude camera C to randomly take a positioning image T1, and obtains a corresponding four-tuple according to the positioning image T1; S3: the cloud platform performs backtracking positioning on the high-altitude camera C according to the four-tuple obtained in step S2; S4: the cloud platform controls the high-altitude camera C to randomly take a pine forest image T2, and detects, extends and fuses the edges of the pine forest image T2 to obtain a panoramic image TQ centered on the pine forest image T2; S5: set a frame selection threshold, and the cloud platform identifies and removes the disease tree frame in the panoramic image TQ according to the frame selection threshold to obtain a detection frame set K; S6: generate the geographic location of the dead tree according to the detection frame set K, and output the geographic location corresponding to the detection frame to form the positioning output set S; determine whether the detection is complete, if yes, end the positioning; if no, return to step S4; The specific steps of step S4 are: preset value: set image set Pic=[]; disease tree detection set C=[]; bounding box set B=[]; edge movement distance=L, L is less than or equal to the height or width of the camera shooting screen; S41: the cloud platform controls the high-altitude camera C to randomly take a pine forest image T2 and put it into the image set Pic; S42: the cloud platform performs disease tree detection on the pine forest image T2 according to the characteristics of the pine wood nematode disease tree and performs frame selection and marking to obtain a disease tree detection set C; S43: obtain the coordinates (x1, y1, x2, y2) of each frame in the disease tree detection set C, and determine whether there is a coordinate value equal to 0 or equal to the width or height of the image; if yes, put the corresponding frame into the bounding box set B, select a bounding box Bi from the bounding box set B, and enter step S44; otherwise, enter step S47; S44: the cloud platform controls the high-altitude camera to rotate so that its shooting screen moves L to the edge of the selected bounding box Bi after shooting and recognition, and obtains a disease tree detection set C1, and the corresponding moving photograph is put into the image set Pic; S45: determine whether all frames in the bounding box Bi appear in the disease tree detection set C1; if yes, enter step S46; otherwise, return to step S44; Wherein, the judgment standard is whether the coordinates of the bounding box Bi and the image T Bi The edge coordinates are coincident, if there is coincidence, the judgment result is "no"; if there is no coincidence, the judgment result is "yes"; S46: determine whether all bounding boxes in the bounding box set B have been moved; if yes, enter step S47; otherwise, let i=i+1 and return to step S44; S47: align and fuse all images in the image set Pic to cover the repeated parts to generate a panoramic image TQ centered on the pine forest image T2.

2. The method for locating pine wilt disease-infected dead trees based on cloud platform camera detection according to claim 1, characterized in that... In step S3, the four-tuple obtained according to the positioning image T1 is positioned using the ONVIF protocol.

3. The method for locating pine wilt disease-infected dead trees based on cloud platform camera detection according to claim 1, characterized in that... In step S47, OpenCV is used to generate key points, and RANSAC algorithm is used for alignment and fusion. 4.The method of claim 3, wherein the method further comprises: determining the location of the dead pine tree based on the cloud platform video detection. In step S5, the sick tree frame within the frame selection threshold is obtained, and the sick tree frame not in the pine forest image T2 is removed, and finally the remaining sick tree frame forms the detection frame set K. 5.The method of claim 1, wherein the method further comprises: determining a location of the dead pine tree based on the cloud platform camera detection. The specific steps of step S6 are as follows: Suppose the width and height of the panoramic image are Width and Height respectively; S61: Cloud platform obtains detection boxes K from the detection box set K. j The center (K) jx K jy ) and size K area ; S62: The cloud platform controls the zoom of the high-altitude camera to take a local photo of the detection frame, and obtains a local enlarged image T of the detection frame Kj ; S63: The cloud platform uses the relative positioning of Onvif to locate the detection frame K j New center, where the detection frame K j PTZ values are (K jx / Width), (K jy / Height), Z1; S64: The cloud platform acquires the current quaternion of the high-altitude camera and obtains the current positioning PTZ Kj value; S65: The cloud platform combines the storage position PTZ in the positioning output set S and the position PTZ of the current detection frame obtained in step S64 Kj The position of the current detection frame is compared with the position of the previous detection frame, and if the two positions are within a set proximity threshold, it is determined that the geographic position of the current pine tree has been obtained, and the current pine tree is discarded. The next detection frame is detected, and the process returns to step S61 until all detection frames are detected. Otherwise, it is considered that a new pine tree is detected, and the process proceeds to step S66. S66: The cloud platform generates the corresponding geographical position information according to the positioning of the current detection frame PTZ Kj The value generates the corresponding geographical position information and saves it in the positioning output set S. The next detection frame is detected, and step S61 is returned until all detection frames are detected.

6. The positioning method of pine wilt disease dead trees based on cloud platform camera detection according to claim 5, characterized in that: In step S62, when the detection frame is locally photographed, the calculation formula of the zoom value Z1 of the aerial camera is: Z1 = d / K area = [(1 / r) * Width * Height] / K area ; r is the minimum number of times of the aerial camera after zooming to photograph all the scenes of the panoramic image; the value range of r is 2-8 times.

Images