A stolen tree matching method and system based on perspective transformation and deep learning

Abstract

The application provides a stolen tree matching method and system based on perspective transformation and deep learning, which sets an angle point detection auxiliary device on a tree section, and carries out distortion correction, checkerboard detection, angle point detection, angle point sorting, perspective transformation, contour extraction, diameter measurement, diameter comparison, contour comparison after an image is acquired by a smart phone, and can provide clear technical support for whether two tree sections belong to the same tree. The angle point detection speed is improved by 30 times through neural network assistance, and there is no requirement for a shooting posture, angle and position. The angle point sorting method of the application can still accurately obtain a perspective transformation matrix in the case of a large area of missing angle points, edge missing, and missing angle points at vertex positions. The neural network is used to extract a felling stake contour, measure a large diameter and a small diameter, and after image mirroring and rotation transformation of the diameter matching image, whether the two sections belong to the same tree can be judged by naked eyes.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method and system for matching stolen trees based on perspective transformation and deep learning. Background Technology

[0002] When tracking stolen trees, forest police often use methods such as roadside surveillance and stakeouts at timber processing plants. However, for suspicious timber found, they lack the technical means to confirm whether it is indeed stolen timber. This poses a certain obstacle to the recovery of stolen trees and deterring criminals. To assist in determining whether suspicious trees are stolen, measuring and comparing them is an important technically feasible method. Measuring the size and outline of the tree trunk requires expensive instruments such as 3D scanners and laser measuring instruments. These instruments are inconvenient to carry, expensive, and when used outdoors, their ability to handle ambient light interference is not ideal. Some instruments also require the application of fluorescent agents, which not only pollutes the environment but also damages the trees.

[0003] Therefore, there is an urgent need for a portable, simple, and low-cost method and system for matching stolen tree cross-sections. Summary of the Invention

[0004] To address the aforementioned technical problems, one objective of this invention is to provide a method for matching stolen trees based on perspective transformation and deep learning. This method involves setting up a corner detection auxiliary device on the tree, photographing the stump of the stolen tree and the cross-section of the target tree, and then sequentially performing distortion correction, detection by the corner detection auxiliary device, corner detection, corner sorting, perspective transformation, contour extraction, diameter measurement, diameter comparison, and contour comparison. This provides a simple and effective way to clearly determine whether two tree cross-sections belong to the same tree.

[0005] This invention sets a corner detection auxiliary device on the cross-section of a tree as a reference object, finds the corner detection auxiliary device through a neural network, and then performs corner detection. This enables faster corner detection and perspective transformation, thus improving detection efficiency.

[0006] One objective of this invention is to provide a corner sorting method that allows for large-area missing corners, missing edge corners, and missing corners at vertex positions. This sorting method has strong robustness and application value.

[0007] One objective of this invention is to provide a stolen tree matching system based on perspective transformation and deep learning. This system does not require expensive instruments such as 3D scanners or laser measuring instruments, has strong resistance to light interference, does not damage trees, and can simply and effectively provide clear technical support for determining whether two tree cross-sections belong to the same tree.

[0008] Note that the description of these objectives does not preclude the existence of other objectives. One aspect of the invention does not require achieving all of the above objectives. Objectives other than those described above can be extracted from the description, drawings, and claims.

[0009] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0010] A method for matching stolen trees based on perspective transformation and deep learning includes the following steps:

[0011] Step S1, stump image acquisition: Place the corner detection auxiliary device on the cross section of the stump and take a picture to acquire a full image of the stump including the corner detection auxiliary device;

[0012] Step S2, Target Image Acquisition: Place the corner detection auxiliary device on the cross-section of the target tree and capture a full cross-section image of the tree including the corner detection auxiliary device;

[0013] Step S3, Preliminary Image Processing: Perform distortion correction on the images obtained in Steps S1 and S2;

[0014] Step S4, Neural Network-Assisted Corner Detection: A deep learning semantic segmentation neural network is used to classify the stump image and target image processed in step S3 into a corner detection auxiliary device area and a background area, and these areas are labeled separately. The corners in the corner detection auxiliary device area are detected and sorted.

[0015] Step S5, Perspective Transformation: Using the sorted corner points, perform least squares perspective transformation to obtain the perspective transformation matrix. Use this matrix to perform perspective transformation on the stump image and the target image respectively.

[0016] Step S6, Contour Extraction: A deep learning semantic segmentation neural network is used to classify the perspective-transformed stump image and target image into corner detection auxiliary device region, tree cross-section region and background region. Holes in the tree cross-section region are filled and the edges are smoothed. The contours of the stump image and target image are extracted respectively.

[0017] Step S7, Diameter Measurement: Find the two contour points with the maximum distance between the stump image and the target image respectively. This distance is defined as the major axis. Then, draw a perpendicular line from the center point of the diameter and define the line connecting the two intersection points of the perpendicular line and the contour as the minor axis.

[0018] Step S8, Diameter Comparison: Compare the major and minor axes of the stump image and the target image respectively. If the error is within the preset threshold, proceed to the next step. If the error is outside the preset threshold, it is considered that the target tree and the stump do not belong to the same tree, and the matching is stopped.

[0019] Step S9, Contour Comparison: Mirror and rotate the perspective-transformed stump image and the target image, and compare the contours. If the contours overlap during mirroring and rotation, the target tree and the stump are considered to be the same tree. If the contours do not overlap, they are not the same tree.

[0020] In the above scheme, the corner detection auxiliary device is a checkerboard pattern.

[0021] In the above scheme, the processing accuracy of the chessboard pattern is less than 0.1mm.

[0022] In the above scheme, the back of the chessboard grid is provided with a sharp part, and when the chessboard grid is placed on the cross-section of a tree, the sharp part is inserted into the tree to fix the chessboard grid.

[0023] In the above scheme, the neural network in step S4 adopts deeplabV3+ deep learning semantic segmentation neural network, the backbone network is inceptionresnetv2, the network category is set to two categories, one is corner detection auxiliary device, and the other is background. The annotation tool is MATLAB's ImageLabeler, and the target chessboard is labeled with polygon boxes, and the rest is labeled as background.

[0024] In the above scheme, the neural network input format in step S4 is an RGB three-layer image. When training the neural network, random rotation, scaling, random flipping, and random translation strategies are used for the image samples. The training computing power is CPU, and the gradient descent method is ADAM.

[0025] In the above scheme, the corner sorting algorithm in step S4 specifically includes the following steps:

[0026] Step S4.1: Represent the corner point set as T, find the four nearest neighbors of the point set T, and match them with the four vertices of the square with side length Q. According to the perspective transformation rules, find the perspective transformation matrix P between these four pairs of vertices. Use the transformation matrix P to perform perspective transformation on all the corner points in T to obtain a new point set, represented by S.

[0027] The method for finding the four nearest neighbors in a point set T first calculates a histogram of distances between points in T. Then, it selects the maximum value V1 and the second largest value V2 from the histogram. Next, it finds the top-left vertex of point set T. Then, it finds two points near V1 and V2 respectively. Finally, it finds the point other than these two points that is closest to the top-left vertex. These four points constitute the four nearest neighbors of point set T.

[0028] Step S4.2: Label the first point in the upper left corner of S as 0. Starting from this point, search to the right and down with a search step size of Q. Within the δ neighborhood of the target position at one interval, search for the existence of a corner point. If it exists, select the point closest to the target position as the target point and increment its label by 1. If it does not exist, set the coordinates of the point to NULL and increment its label by 1. Continuously execute the above search process and add a label to each found or unfound point until the last corner point.

[0029] Step S4.3: Sort the corner points of the N*2N chessboard grid with uniform spacing. The sorting start point is the top left corner point, and the sorting direction is right and down. Since the corner points are evenly spaced, no special sorting method is required; in this invention, N is 6.

[0030] Step S4.4: Match the sorted points obtained in step S4.2 with the sorted points obtained in step S4.3 one by one, with each pair having the same index; if the coordinates of a point in a pair contain a NULL value, then that pair of points will not participate in the subsequent least squares perspective transformation; set all points participating in the least squares perspective transformation.

[0031] In the above scheme, step S6 contour extraction uses a deeplabV3+ deep learning semantic segmentation neural network, with the backbone network being InceptionResNetV2. The network is configured with three categories: tree cross-sections, checkerboard patterns, and background. The annotation tool is MATLAB's ImageLabeler, using pixel labels to annotate tree cross-sections, polygonal bounding boxes to annotate the checkerboard patterns, and pixel labels for the remaining areas as background. The network input format is a three-layer RGB image. To enrich the training samples, random rotation, scaling, flipping, and translation strategies are employed. The training computation is performed using a CPU, and the gradient descent method is ADAM.

[0032] In the above scheme, the preset error threshold for diameter comparison in step S8 is 20%.

[0033] A stolen tree matching system based on perspective transformation and deep learning includes a mobile terminal and a corner detection auxiliary device. The mobile terminal includes an Android APP software, which includes an image acquisition module, an image correction module, a corner detection module, a perspective transformation module, a contour extraction module, a diameter measurement module, a diameter comparison module, and a contour comparison module. The corner detection auxiliary device is a checkerboard pattern.

[0034] The image acquisition module is used to place the corner detection auxiliary device on the cross-section of the stump and the cross-section of the target tree, and capture a full image of the stump and a full image of the tree cross-section, including the corner detection auxiliary device.

[0035] The image correction module is used to correct distortion in the images acquired by the image acquisition module;

[0036] The corner detection module is used to classify the stump image and target image processed by the image correction module into a corner detection auxiliary device area and a background area using a deep learning semantic segmentation neural network, and label them respectively. It detects the corners in the corner detection auxiliary device area and sorts the corners.

[0037] The perspective transformation module is used to perform least-squares perspective transformation using the sorted corner points to obtain a perspective transformation matrix. This matrix is ​​then used to perform perspective transformation on the stump image and the target image, respectively.

[0038] The contour extraction module is used to classify the perspective-transformed stump image and target image using a deep learning semantic segmentation neural network, dividing them into a corner detection auxiliary device region, a tree cross-section region, and a background region. After filling the holes in the tree cross-section region and smoothing the edges, the contours of the stump image and target image are extracted respectively.

[0039] The diameter measurement module is used to find the two contour points with the maximum distance between the stump image and the target image, and define them as the major axis. A perpendicular line is drawn from the center point of the diameter, and the line connecting the two intersection points of the perpendicular line and the contour is defined as the minor axis.

[0040] The diameter comparison module is used to compare the major and minor axes of the stump image and the target image respectively. If the error is within a preset threshold, contour comparison is performed. If the error is outside the preset threshold, the target tree and the stump are considered not to belong to the same tree, and matching is stopped.

[0041] The contour comparison module is used to mirror and rotate the perspective-transformed stump image and the target image, and compare the contours. If the contours overlap during mirroring and rotation, the target tree and the stump are considered to be the same tree. If the contours do not overlap, they are not the same tree.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] According to one aspect of the present invention, a corner detection auxiliary device is set on a tree as a reference, and the corner detection auxiliary device is detected by a neural network. Then, the corners on the corner detection auxiliary device are detected. This method can perform corner detection more quickly and improves detection efficiency.

[0044] According to one aspect of the present invention, the detected corner points are sorted. This sorting method allows for the existence of large-area missing corner points, allows for the absence of edge corner points, and allows for the absence of corner points at vertex positions. Missing corner points are not included in the calculation of least squares perspective transformation. This method has stronger application value and robustness.

[0045] According to one aspect of the present invention, images are taken of the stumps of stolen trees and the cross-section of the target tree. After obtaining the images, distortion correction, corner detection auxiliary device detection, corner detection, perspective transformation, contour extraction, diameter measurement, diameter comparison, and contour comparison are performed sequentially. This provides clear technical support for determining whether the cross-sections of two trees belong to the same tree in a simple and effective manner.

[0046] According to one aspect of the present invention, tree matching can be performed using an Android APP without the need for expensive instruments such as 3D scanners and laser measuring instruments. It has strong resistance to light interference and will not damage the trees.

[0047] Note that the description of these effects does not preclude the existence of other effects. One aspect of the invention does not necessarily have all the aforementioned effects. Effects other than those described above can be readily observed and extracted from the description, drawings, claims, etc. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating one embodiment of the present invention.

[0049] Figure 2 These are cross-sectional images of stumps and trees obtained according to one embodiment of the present invention.

[0050] Figure 3 This is the perspective transformation result of a cross-sectional image of a stump and a tree according to one embodiment of the present invention.

[0051] Figure 4 This is a comparison diagram of tree cross-sections according to one embodiment of the present invention. Detailed Implementation

[0052] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0053] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "front," "rear," "left," "right," "upper," "lower," "axial," "radial," "vertical," "horizontal," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0054] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0055] Example 1

[0056] Figure 1 The image shows a preferred embodiment of the stolen tree matching method based on perspective transformation and deep learning.

[0057] A method for matching stolen trees based on perspective transformation and deep learning includes the following steps:

[0058] Step S1, stump image acquisition: Place the corner detection auxiliary device on the cross section of the stump and take a picture to acquire a full image of the stump including the corner detection auxiliary device;

[0059] Step S2, Target Image Acquisition: Place the corner detection auxiliary device on the cross-section of the target tree and capture a full image of the tree cross-section including the corner detection auxiliary device. According to this embodiment, preferably, the corner detection auxiliary device is a checkerboard pattern, which is rectangular with a size of 2N*N, where N can be 3, 4, ..., 10, or other natural numbers. In this invention, N is 6. The physical dimensions of the checkerboard pattern must be precise, with an accuracy controlled within 0.1mm.

[0060] The checkerboard pattern has sharp points on its back. When placed on a tree cross-section, these sharp points insert into the tree to secure the checkerboard. During photography, the checkerboard is placed in the center of the tree cross-section, and images are captured on-site using a smartphone. There is no need to focus on the shooting angle or posture, nor to manually adjust the focus. In one embodiment of the invention, the captured image is as follows: Figure 2 As shown;

[0061] Step S3, Preliminary Image Processing: Distortion correction is performed on the images obtained in steps S1 and S2; according to this embodiment, preferably, the distortion correction algorithm adopts Zhang Zhengyou's checkerboard method;

[0062] Step S4, Neural Network-Assisted Corner Detection: A deep learning semantic segmentation neural network is used to classify the stump image and target image processed in step S3 into a corner detection auxiliary device area and a background area, and these areas are labeled separately. The corners in the corner detection auxiliary device area are detected and sorted.

[0063] According to this embodiment, preferably, the neural network adopts deeplabV3+ deep learning semantic segmentation neural network, the backbone network is inceptionresnetv2, the network category is set to two categories, one is corner detection auxiliary device, and the other is background. The annotation tool is MATLAB's ImageLabeler, and the target chessboard is labeled with polygon boxes, and the rest is labeled as background.

[0064] In step S4, the neural network input format is an RGB three-layer image. When training the neural network, random rotation, scaling, random flipping, and random translation strategies are used for the image samples. The training computing power is CPU, without the need for GPU acceleration. The gradient descent method is ADAM.

[0065] When performing corner detection, the training model is loaded, the image of the stump is input, the checkerboard pattern is detected, and the algorithm gives the region where the checkerboard pattern is located. The region where the checkerboard pattern is located is marked as 1, and the background region is marked as 0.

[0066] Since the area of ​​the checkerboard pattern only accounts for a small portion of the entire stump image area, directly applying corner detection methods to detect the corners of the checkerboard pattern in the stump image is a very time-consuming method. This invention trains a deep neural network to find the checkerboard pattern in the stump image, which can improve the speed of the corner detection method by more than 30 times.

[0067] Step S5, Perspective Transformation: Using the sorted corner points, perform least-squares perspective transformation to obtain a perspective transformation matrix. Use this matrix to perform perspective transformation on both the stump image and the target image; transform images taken from any angle and in any posture to the main (vertical) view. In one embodiment of this invention, the perspective-transformed image is as follows: Figure 3As shown.

[0068] Step S6, Contour Extraction: A deep learning semantic segmentation neural network is used to classify the perspective-transformed stump image and target image into corner detection auxiliary device region, tree cross-section region and background region. Holes in the tree cross-section region are filled and the edges are smoothed. The contours of the stump image and target image are extracted respectively.

[0069] Step S7, Diameter Measurement: Find the two contour points with the maximum distance between the stump image and the target image respectively. This distance is defined as the major axis. Draw a perpendicular line from the center point of this diameter. The line connecting the two intersection points of the perpendicular line and the contour is defined as the minor axis.

[0070] Step S8, Diameter Comparison: Compare the major and minor axes of the stump image and the target image respectively. If the error is within the preset threshold, proceed to the next step. If the error is outside the preset threshold, it is considered that the target tree and the stump do not belong to the same tree, and the matching is stopped.

[0071] According to this embodiment, preferably, the automatic diameter measurement results can be manually checked and some errors can be manually corrected.

[0072] Step S9, Contour Comparison: Mirror and rotate the perspective-transformed stump image and the target image, and compare the contours. If the contours overlap during mirroring and rotation, the target tree and the stump are considered to be the same tree. If the contours do not overlap, they are not the same tree.

[0073] The corner sorting algorithm in step S4 specifically includes the following steps:

[0074] Step S4.1: Represent the corner point set as T, find the four nearest neighbors of the point set T, and match them with the four vertices of the square with side length Q. According to the perspective transformation rules, find the perspective transformation matrix P between these four pairs of vertices. Use the transformation matrix P to perform perspective transformation on all the corner points in T to obtain a new point set, represented by S.

[0075] According to this embodiment, preferably, the method for finding the four nearest neighbors in the point set T first calculates a histogram of the distances between points in the point set T. Then, the maximum value V1 and the second largest value V2 of the histogram are selected. The top-left vertex of the point set T is found. Then, two points are found near the vertex V1 and V2 respectively. Finally, the point closest to the top-left vertex, excluding these two points, is found. These four points constitute the four nearest neighbors of the point set T.

[0076] Step S4.2: Label the first point in the upper left corner of S as 0. Starting from this point, search to the right and down with a search step size of Q. Within the δ neighborhood of the target position at one interval, search for the existence of a corner point. If it exists, select the point closest to the target position as the target point and increment its label by 1; if it does not exist, set the coordinates of the point to NULL and increment its label by 1. Continuously execute the above search process and add a label to each found or unfound point until the last corner point. According to this embodiment, preferably, the search step size Q is a fixed value, and in this embodiment, Q is 80 pixels. The size of the neighborhood δ is generally 20% of Q. Even if this parameter is set to 10% or 30% of Q, it will not affect the effect of the algorithm, so it is easy to determine. The advantage of this sorting method of the present invention is that even in the case of large-area missing corner points, missing edges, and missing corner points at vertex positions, the sorting method can run stably, and the sorting method has extremely strong robustness.

[0077] Step S4.3: Sort the corner points in the standard corner point detection auxiliary device, starting from the top left corner point and sorting in the right and down direction.

[0078] Step S4.4: Match the sorted points obtained in step S4.2 with the sorted points obtained in step S4.3 one by one, with each pair having the same index; if the coordinates of a point in a pair contain a NULL value, then that pair of points will not participate in the subsequent least squares perspective transformation; set all points participating in the least squares perspective transformation.

[0079] In one embodiment of the present invention, the chessboard size is 6*12, therefore, the maximum number of detectable corner points is 55. The least squares perspective transformation in step S5 uses the following formula:

[0080]

[0081] In the formula: a 11 ,a 12 …a 31 ,a 32 Let u1, v1, ... u1 be variables, and pinv denote the pseudo-inverse; 55 ,v 55 The coordinates of the corner points detected by the corner detection algorithm are sorted according to step 4.2. In perspective transformation, these points are called moving points, x1, y1, ... x 55 ,y 55 The coordinates of the fixed points are determined by step 4.3. If there are points with NULL coordinates among these 55 pairs, then in the specific calculation of the perspective transformation matrix, the rows and columns with NULL values ​​on the right side of the above formula can be deleted.

[0082] The contour extraction in step S6 uses DeepLabV3+ deep learning semantic segmentation neural network with InceptionResNetV2 as the backbone. The network is divided into three categories: tree cross-section, checkerboard, and background. The annotation tool is MATLAB's ImageLabeler. Tree cross-section is labeled with pixel labels, checkerboard is labeled with polygon boxes, and the remaining parts are labeled with pixels as background. According to this embodiment, preferably, the labels corresponding to tree cross-section and checkerboard are set to 1, and the background label is set to 0. Holes are filled in the area with the largest area of ​​label 1, the edges are smoothed, and the contour is extracted.

[0083] The neural network input format is a three-layer RGB image. To enrich the training samples, random rotation, scaling, flipping, and translation strategies were used. Training computation was performed using the CPU, without the need for GPU acceleration, and the ADAM method was used for gradient descent.

[0084] According to this embodiment, preferably, the preset error threshold for diameter comparison in step S8 is 20%.

[0085] According to this embodiment, preferably, when performing contour comparison, the stump image and the target image are perspective-transformed, placed together, and separated left and right. The second image is then mirrored and rotated. While rotating, the two images are visually observed to see if they match. If the observation process is difficult, the rotation angle is adjusted, and the observation is repeated until a conclusion is reached.

[0086] like Figure 4 As shown, Figure 4 The position of the slider is adjustable between 0 and 360 degrees, corresponding to the rotation angle of the right image. By adjusting the position of the slider, you can observe the rotation of the right image. By continuously adjusting the position of the slider, you can make the rotation angle of the right image and the left image keep the same. As long as the rotation angle is appropriate, you can use the naked eye to determine whether the images on the left and right sides belong to the same tree.

[0087] During tree felling, workers often cut a double-line notch between two sections, additionally sawing off one or more V-shaped thin blades to control the direction of the tree's fall. Figure 2 The location and outline of the missing V-shaped sheet can be seen. Due to factors such as occlusion and the absence of the V-shaped sheet, the outline of some locations cannot be observed, but after rotation, it becomes visible. Figure 2It can be seen that the outlines of the two images show a high degree of matching in terms of V-shaped thin-film defects around 120 degrees clockwise, and a high degree of matching in terms of outline shape around 180 degrees clockwise. Considering that the diameters of the two images are similar and that some outlines are highly matched, it is technically possible to determine that they belong to the same tree. The left image is of the base stump, and the right image is of the cross-section of the stolen timber, thus providing technical support for the diagnosis of the case.

[0088] This invention, based on perspective transformation and deep learning technology, involves installing a corner detection auxiliary device on a tree. It photographs the stump of a stolen tree and the cross-section of the target tree. After acquiring the images, it sequentially performs distortion correction, corner detection auxiliary device detection, corner detection, perspective transformation, contour extraction, diameter measurement, diameter comparison, and contour comparison. This provides a simple and effective technical basis for determining whether two tree cross-sections belong to the same tree. Furthermore, by using the corner detection auxiliary device as a reference point on the tree and employing a neural network to assist in corner detection, corner detection and perspective transformation can be performed more quickly, improving detection efficiency.

[0089] Example 2

[0090] A stolen tree matching system based on perspective transformation and deep learning includes a mobile terminal and a corner detection auxiliary device. Preferably, the mobile terminal is a smartphone or smart tablet, and the mobile terminal includes Android APP software. The Android APP software includes an image acquisition module, an image correction module, a corner detection module, a perspective transformation module, a contour extraction module, a diameter measurement module, a diameter comparison module, and a contour comparison module. The corner detection auxiliary device is a checkerboard pattern.

[0091] The image acquisition module is used to place the corner detection auxiliary device on the cross-section of the stump and the cross-section of the target tree, and capture a full image of the stump and the full image of the tree cross-section, including the corner detection auxiliary device.

[0092] The image correction module is used to correct distortion in the images acquired by the image acquisition module;

[0093] The corner detection module is used to classify the stump image and target image processed by the image correction module into a corner detection auxiliary device area and a background area using a deep learning semantic segmentation neural network, and label them respectively. It detects the corners in the corner detection auxiliary device area and sorts the corners.

[0094] The perspective transformation module is used to perform least-squares perspective transformation using the sorted corner points to obtain a perspective transformation matrix. This matrix is ​​then used to perform perspective transformation on the stump image and the target image, respectively.

[0095] The contour extraction module is used to classify the perspective-transformed stump image and target image using a deep learning semantic segmentation neural network, dividing them into a corner detection auxiliary device region, a tree cross-section region, and a background region. After filling the holes in the tree cross-section region and smoothing the edges, the contours of the stump image and target image are extracted respectively.

[0096] The diameter measurement module is used to find the two contour points with the maximum distance between the stump image and the target image, respectively. This distance is defined as the major axis. A perpendicular line is drawn from the center point of the diameter, and the line connecting the two intersection points of the perpendicular line and the contour is defined as the minor axis.

[0097] The diameter comparison module is used to compare the major and minor axes of the stump image and the target image respectively. If the error is within a preset threshold, contour comparison is performed. If the error is outside the preset threshold, the target tree and the stump are considered not to belong to the same tree, and matching is stopped.

[0098] The contour comparison module is used to mirror and rotate the perspective-transformed stump image and the target image, and compare the contours. If the contours overlap during mirroring and rotation, the target tree and the stump are considered to be the same tree. If the contours do not overlap, they are not the same tree.

[0099] This invention does not require expensive instruments such as 3D scanners or laser measuring instruments, has strong resistance to light interference, will not damage trees, and can provide clear technical support for whether two tree cross sections belong to the same tree in a simple and effective way.

[0100] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0101] The detailed descriptions listed above are merely specific illustrations of feasible embodiments of the present invention and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for matching stolen trees based on perspective transformation and deep learning, characterized in that, Includes the following steps: Step S1, stump image acquisition: Place the corner detection auxiliary device on the cross section of the stump and take a picture to acquire a full image of the stump including the corner detection auxiliary device; Step S2, Target Image Acquisition: Place the corner detection auxiliary device on the cross-section of the target tree and capture a full cross-section image of the tree including the corner detection auxiliary device; Step S3, Preliminary Image Processing: Perform distortion correction on the images obtained in Steps S1 and S2; Step S4, Neural Network-Assisted Corner Detection: A deep learning semantic segmentation neural network is used to classify the stump image and target image processed in step S3 into a corner detection auxiliary device area and a background area, and these areas are labeled separately. The corners in the corner detection auxiliary device area are detected and sorted. Step S5, Perspective Transformation: Using the sorted corner points, perform least squares perspective transformation to obtain the perspective transformation matrix. Use this matrix to perform perspective transformation on the stump image and the target image respectively. Step S6, Contour Extraction: A deep learning semantic segmentation neural network is used to classify the perspective-transformed stump image and target image into corner detection auxiliary device region, tree cross-section region and background region. Holes in the tree cross-section region are filled and the edges are smoothed. The contours of the stump image and target image are extracted respectively. Step S7, Diameter Measurement: Find the two contour points with the maximum distance between the stump image and the target image respectively. The distance between the two points is defined as the major axis. Draw a perpendicular line from the center point of the diameter. The line connecting the two intersection points of the perpendicular line and the contour is defined as the minor axis. Step S8, Diameter Comparison: Compare the major and minor axes of the stump image and the target image respectively. If the error is within the preset threshold, proceed to the next step. If the error is outside the preset threshold, it is considered that the target tree and the stump do not belong to the same tree, and the matching is stopped. Step S9, Contour Comparison: Mirror and rotate the perspective-transformed stump image and the target image, and compare the contours. If the contours overlap during mirroring and rotation, the target tree and the stump are considered to be the same tree. If the contours do not overlap, they are not the same tree.

2. The stolen tree matching method based on perspective transformation and deep learning according to claim 1, characterized in that, The corner detection auxiliary device is a checkerboard pattern.

3. The stolen tree matching method based on perspective transformation and deep learning according to claim 2, characterized in that, The processing precision of the chessboard pattern is less than 0.1 mm.

4. The stolen tree matching method based on perspective transformation and deep learning according to claim 2, characterized in that, The back of the chessboard grid has a sharp point, which is inserted into the tree to fix the chessboard grid when it is placed on the cross-section of a tree.

5. The stolen tree matching method based on perspective transformation and deep learning according to claim 1, characterized in that, In step S4, the neural network adopts deeplabV3+ deep learning semantic segmentation neural network, the backbone network is inceptionresnetv2, and the network is set to two categories: one is a corner detection auxiliary device and the other is the background. The annotation tool is MATLAB's ImageLabeler, and the target chessboard is labeled with polygon boxes, and the rest is labeled as the background.

6. The stolen tree matching method based on perspective transformation and deep learning according to claim 1, characterized in that, In step S4, the neural network input format is an RGB three-layer image. When training the neural network, random rotation, scaling, random flipping, and random translation strategies are used for the image samples. The training computing power is CPU, and the gradient descent method is ADAM.

7. The stolen tree matching method based on perspective transformation and deep learning according to claim 1, characterized in that, The corner sorting algorithm used in step S4 specifically includes the following steps: Step S4.1: Represent the corner point set as T, find the four nearest neighbors of the point set T, and match them with the four vertices of the square with side length Q. According to the perspective transformation rules, find the perspective transformation matrix P between these four pairs of vertices. Use the transformation matrix P to perform perspective transformation on all the corner points in T to obtain a new point set, represented by S. Step S4.2: Label the first point in the upper left corner of S as 0. Starting from this point, search to the right and down with a search step size of Q. Within the δ neighborhood of the target position at one interval, search for the existence of a corner point. If it exists, select the point closest to the target position as the target point and increment its label by 1. If it does not exist, set the coordinates of the point to NULL and increment its label by 1. Continuously execute the above search process and add a label to each found or unfound point until the last corner point is reached. Step S4.3: Sort the corner points in the corner point detection auxiliary device, starting from the top left corner point and sorting in the right and down direction; Step S4.4: Match the sorted points obtained in step S4.2 with the sorted points obtained in step S4.3 one by one, with each pair having the same index; if the coordinates of a point in a pair contain a NULL value, then that pair of points will not participate in the subsequent least squares perspective transformation; set all points participating in the least squares perspective transformation.

8. The stolen tree matching method based on perspective transformation and deep learning according to claim 1, characterized in that, Step S6, contour extraction, employs DeepLabV3+ deep learning semantic segmentation neural network with InceptionResNetV2 as the backbone. The network is categorized into three classes: tree cross-sections, checkerboard patterns, and background. The annotation tool is MATLAB's ImageLabeler, using pixel labels to annotate tree cross-sections, polygonal bounding boxes to annotate the checkerboard patterns, and pixel labels for the remaining areas as background. The network input format is a three-layer RGB image. Random rotation, scaling, flipping, and translation strategies are used for the samples. Training computation is performed using a CPU, and the gradient descent method is ADAM.

9. The stolen tree matching method based on perspective transformation and deep learning according to claim 1, characterized in that, The preset error threshold for diameter comparison in step S8 is 20%.

10. A system for matching stolen trees based on perspective transformation and deep learning according to any one of claims 1-9, characterized in that, The device includes a mobile terminal and a corner detection auxiliary device. The mobile terminal includes an Android APP software, which includes an image acquisition module, an image correction module, a corner detection module, a perspective transformation module, a contour extraction module, a diameter measurement module, a diameter comparison module, and a contour comparison module. The corner detection auxiliary device is a checkerboard pattern. The image acquisition module is used to place the corner detection auxiliary device on the cross-section of the stump and the cross-section of the target tree, and capture a full image of the stump and a full image of the tree cross-section, including the corner detection auxiliary device. The image correction module is used to correct distortion in the images acquired by the image acquisition module; The corner detection module is used to classify the stump image and target image processed by the image correction module into a corner detection auxiliary device area and a background area using a deep learning semantic segmentation neural network, and label them respectively. It detects the corners in the corner detection auxiliary device area and sorts the corners. The perspective transformation module is used to perform least-squares perspective transformation using the sorted corner points to obtain a perspective transformation matrix. This matrix is ​​then used to perform perspective transformation on the stump image and the target image, respectively. The contour extraction module is used to classify the perspective-transformed stump image and target image using a deep learning semantic segmentation neural network, dividing them into a corner detection auxiliary device region, a tree cross-section region, and a background region. After filling the holes in the tree cross-section region and smoothing the edges, the contours of the stump image and target image are extracted respectively. The diameter measurement module is used to find the two contour points with the maximum distance between the stump image and the target image, respectively. This distance is defined as the major axis. A perpendicular line is drawn from the center point of the diameter, and the line connecting the two intersection points of the perpendicular line and the contour is defined as the minor axis. The diameter comparison module is used to compare the major and minor axes of the stump image and the target image respectively. If the error is within a preset threshold, contour comparison is performed. If the error is outside the preset threshold, the target tree and the stump are considered not to belong to the same tree, and matching is stopped. The contour comparison module is used to mirror and rotate the perspective-transformed stump image and the target image, and compare the contours. If the contours overlap during mirroring and rotation, the target tree and the stump are considered to be the same tree. If the contours do not overlap, they are not the same tree.

Images