A tree multi-parameter image processing method and device, a terminal and a medium

Abstract

The application discloses a tree multi-parameter image processing method and device, a terminal and a medium. The method comprises the following steps: acquiring an image of a target tree, inputting a preset tree target point three-dimensional coordinate calculation model, obtaining three-dimensional coordinates and basic parameters of the target tree, analyzing the basic parameters and the three-dimensional coordinates based on a tree crown and tree extraction model of instance segmentation, acquiring image coordinates of a measurement target point of the target tree, obtaining world coordinates of the measurement target point by using a mapping relationship from the obtained image coordinates of the measurement target point to world coordinates, obtaining an absolute position of the tree, and synchronously acquiring tree height, diameter at breast height and crown width information of the target tree by calculating differences between world coordinates of different target points of the tree. Therefore, the application overcomes the dependence on a reached point in a mainstream tree measurement method, and realizes synchronous acquisition of absolute position, diameter at breast height, crown width and tree height information of a tree to be measured.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, terminal, and medium for multi-parameter image processing of trees. Background Technology

[0002] Tree surveys have become an essential part of resource investigations, project construction control planning adjustments and revisions, and development and approval processes such as project initiation and design schemes. Tree surveys mainly include multi-parameter information such as tree location, diameter at breast height (DBH), tree height, and crown width. Currently, existing tree survey image measurement methods can be broadly divided into two categories:

[0003] (1) Single-image measurement algorithm based on known distance values: This algorithm mainly measures distance based on known length and collinear / coplanar constraints. For example, when acquiring images, a shooting pole, ruler, or template can be placed close to the tree to be measured, or a target of known length can be selected for simultaneous shooting. Alternatively, during the measurement process, the distance from the measuring station to the tree can be obtained through a laser rangefinder or steel ruler. The shooting angle and shooting height can also be limited or obtained. By obtaining distance reference values ​​through the above three methods, the tree's diameter at breast height (DBH), tree height, and crown width can be calculated.

[0004] (2) Image Pair Measurement Algorithm Based on Forward Intersection: This algorithm mainly calculates the 3D coordinates or depth values ​​of the measurement points by forward intersection of two image pairs taken at different angles, and then infers the distance. It is divided into two methods: single acquisition and two acquisitions. Single acquisition, as in the paper "Research on Tree Height Measurement Method Based on Binocular Vision", obtains camera lens parameters through binocular camera calibration, and combines the SGBM algorithm with the BM algorithm to obtain the depth image of the tree to be measured. Then, the spatial coordinates of the key points of the tree are extracted, and the tree height is calculated. At the same time, deep learning and binocular vision are combined to realize the identification and detection of tree species. Two acquisitions are mainly achieved by limiting the shooting position to the same horizontal straight line, determining the baseline distance according to the simple scale of the gimbal plane, or obtaining the two acquisition images by measuring the angle information of the theodolite. Summary of the Invention

[0005] This invention provides a method, device, terminal, and medium for multi-parameter image processing of trees. Addressing the challenges of high difficulty, low efficiency, and limited applicability in image measurement operations using monocular and binocular images acquired with a monocular camera, this patent employs a GNSS / IMU-based binocular camera to acquire two image pairs at different positions and angles simultaneously. Simultaneously, it develops a multi-scale crown and trunk fine segmentation method based on the SOLO framework, enabling simultaneous measurement of multiple tree parameters in a single image acquisition.

[0006] To achieve the above objectives, in a first aspect, embodiments of the present invention provide a method for processing multi-parameter images of trees, comprising:

[0007] The image of the target tree is acquired, and a preset three-dimensional coordinate calculation model of the tree target point is input to obtain the three-dimensional coordinates and basic parameters of the target tree; the basic parameters include tree height, crown width, absolute position of the tree and diameter at breast height.

[0008] Based on the instance segmentation canopy and tree extraction model, the basic parameters and three-dimensional coordinates are analyzed to obtain the image coordinates of the measurement target points of the target tree;

[0009] By utilizing the mapping relationship between the image coordinates and world coordinates of the measured target point, the world coordinates of the measured target point are obtained, and the absolute position of the tree is obtained. By subtracting the world coordinates of different target points of the tree, the tree height, diameter at breast height, and crown width information of the target tree are obtained simultaneously.

[0010] Furthermore, the tree multi-parameter image processing method further includes:

[0011] The tree height, diameter at breast height (DBH), and crown width of the target trees are entered into a database for tree resource information management.

[0012] Secondly, embodiments of the present invention provide a tree multi-parameter image processing apparatus, comprising:

[0013] The model calculation module is used to acquire images of the target tree, input a preset three-dimensional coordinate calculation model of the tree target point, and obtain the three-dimensional coordinates and basic parameters of the target tree; the basic parameters include tree height parameters, crown width parameters, tree absolute position parameters, and diameter at breast height parameters.

[0014] The model extraction module is used to analyze the basic parameters and three-dimensional coordinates of the canopy and tree extraction model based on instance segmentation, and obtain the image coordinates of the measurement target points of the target tree;

[0015] The coordinate mapping module is used to obtain the world coordinates of the target point by utilizing the obtained mapping relationship between the image coordinates and world coordinates of the target point, thereby obtaining the absolute position of the tree. By subtracting the world coordinates of different target points of the tree, the tree height, diameter at breast height (DBH), and crown width information of the target tree are obtained simultaneously.

[0016] Furthermore, the aforementioned tree multi-parameter image processing device also includes:

[0017] The tree management module is used to process the data of the target tree's height, diameter at breast height (DBH), and crown width into a database for tree resource information management.

[0018] Thirdly, embodiments of the present invention provide a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the above-described tree multi-parameter image processing method.

[0019] Furthermore, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the above-described tree multi-parameter image processing method.

[0020] Compared with existing technologies, the present invention discloses a method, apparatus, terminal, and medium for multi-parameter image processing of trees. This method acquires an image of a target tree, inputs a preset three-dimensional coordinate calculation model for the tree target point, and obtains the three-dimensional coordinates and basic parameters of the target tree. It then analyzes the basic parameters and three-dimensional coordinates based on a canopy and tree extraction model derived from instance segmentation. The method acquires the image-side coordinates of the measurement target point of the target tree. Utilizing the obtained mapping relationship between the image-side coordinates of the measurement target point and world coordinates, the method obtains the world coordinates of the measurement target point, thereby obtaining the absolute position of the tree. Finally, by calculating the difference between the world coordinates of different target points of the tree, the method simultaneously acquires the tree height, diameter at breast height (DBH), and crown width information of the target tree. Therefore, the embodiments of the present invention can acquire tree species and height parameters by acquiring binocular images in one go, processing them to obtain disparity maps, performing target detection and three-dimensional coordinate calculation, accurately locating and identifying tree objects, and selecting them by bounding boxes; it overcomes the dependence of existing mainstream tree measurement methods on the arrival point, and realizes the simultaneous acquisition of the absolute position, diameter at breast height, crown width, and tree height information of the tree to be measured. On the other hand, it combines tree measurement and tree species assessment, reduces the workload of field work, simplifies and merges them into tree image acquisition, facilitates indoor verification, and improves quality. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a multi-parameter image processing method for trees provided in an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of the structure of a tree multi-parameter image processing device provided in an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of a three-dimensional coordinate calculation model for tree target points provided in an embodiment of the present invention;

[0024] Figure 4 This is a schematic diagram of a multi-scale residual UNet network structure provided in an embodiment of the present invention;

[0025] Figure 5This is a schematic diagram of a multi-scale residual block structure provided in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] It should be noted that the terms "comprising" and "specific" in this invention, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0028] Please see Figure 1 , Figure 1 This is a flowchart illustrating a multi-parameter image processing method for trees provided in an embodiment of the present invention. The method includes steps S11 to S13:

[0029] S11: Acquire an image of the target tree, input a preset three-dimensional coordinate calculation model for the tree target point, and obtain the three-dimensional coordinates and basic parameters of the target tree; the basic parameters include tree height parameters, crown width parameters, absolute position parameters of the tree, and diameter at breast height parameters;

[0030] S2: Based on the instance segmentation canopy and tree extraction model, analyze the basic parameters and three-dimensional coordinates to obtain the image coordinates of the measurement target points of the target tree;

[0031] S13: Using the obtained mapping relationship between the image coordinates of the measurement target point and the world coordinates, the world coordinates of the measurement target point are obtained, and the absolute position of the tree is obtained. By calculating the difference between the world coordinates of different target points of the tree, the tree height, diameter at breast height, and crown width information of the target tree are obtained simultaneously.

[0032] In practice, a GNSS / IMU-based binocular camera is first used to select and locate points in an open area. The selected locations must not only be easily identifiable by RTK fixed solutions but also meet the line-of-sight requirement for the trees being measured. After selection, the instrument position is determined. attitude The measurement and calculation, combined with the known calibration values ​​of the instrument. Using the following formula (1), the mapping relationship between the image-side coordinates of the target point and the world coordinates is obtained; next, tree images are captured, and the left and right images should simultaneously contain the target point to obtain binocular images; then, instance segmentation is performed on the acquired binocular image data to obtain tree crown and trunk instances, and the absolute position, tree height, diameter at breast height, and crown width measurement target points are extracted to obtain image coordinates x1, y1, x2, y2. Combined with the known calibration values ​​K1, K2, R, T of the instrument and the following formulas (2) and (6), the image-side coordinates of the target point can be obtained; then, using the obtained mapping relationship between the image-side coordinates of the target point and the world coordinates, the world coordinates of the target point can be obtained, and the absolute position of the tree can be directly obtained. On this basis, the difference between the world coordinates of different target points of the tree is calculated to simultaneously obtain the tree height, diameter at breast height, and crown width information.

[0033] Furthermore, the tree multi-parameter image processing method further includes:

[0034] S14: The tree height, diameter at breast height, and crown width of the target tree are processed into a database for tree resource information management.

[0035] Specifically, acquiring an image of the target tree, inputting a preset 3D coordinate calculation model for the tree target point, and obtaining the 3D coordinates and basic parameters of the target tree specifically includes:

[0036] Acquire an image of the target tree, calculate the highest vertex, leftmost point, and rightmost point of the tree crown in the image, and obtain and store the three-dimensional coordinates of the tree crown based on a preset three-dimensional coordinate calculation model of the tree target point.

[0037] Calculate the distance between the leftmost and rightmost points to obtain the crown width parameters of the tree canopy;

[0038] Calculate the lowest point of the tree trunk in the image, and obtain and store the three-dimensional coordinates of the tree trunk based on the three-dimensional coordinate calculation model of the tree target point;

[0039] Calculate the distance between the highest and lowest points to obtain the tree height parameter of the tree trunk;

[0040] Using the lowest point as the search starting point, calculate the Hough transform of the tree trunk in the image, extract the horizontal straight line segment, take the midpoint of the straight line segment, and calculate and store the three-dimensional coordinates of the midpoint based on the three-dimensional coordinate calculation model of the tree target point to obtain the absolute position parameters of the target tree.

[0041] Using the straight line segment as a reference, search vertically upwards to calculate the spatial distance between the midpoint of the leftmost and rightmost points where the straight line segment or its extension intersects with the tree trunk entity and the midpoint of the straight line segment. Find a reference line at a preset height from the midpoint of the straight line segment. The endpoints of the reference line are the leftmost and rightmost points where the straight line segment intersects with the tree trunk entity.

[0042] Calculate the distance between the leftmost and rightmost points where the tree trunks intersect, and obtain the diameter at breast height (DBH) parameter of the target tree.

[0043] In the specific implementation, the top vertex Pt, leftmost point Pl, and rightmost point Pr of the tree crown in the image are calculated respectively, and their three-dimensional coordinates are obtained and stored according to the three-dimensional coordinate calculation model; the distance between points Pl and Pr is calculated to obtain the crown width parameter; the bottom point Pd of the tree trunk in the image is calculated, and its three-dimensional coordinates are obtained and stored according to the three-dimensional coordinate calculation model; the distance between points Pt and Pd is calculated to obtain the tree height parameter; using Pd as the search starting point, the Hough transform of the tree trunk in the image is calculated to extract the horizontal straight line segment L0, and the midpoint P0 is taken. Its three-dimensional coordinates are obtained and stored according to the three-dimensional coordinate calculation model to obtain the absolute position parameter of the tree; using the straight line segment L0 as the reference, the search is carried out vertically upward to calculate the spatial distance between the midpoint p0 of the leftmost and rightmost points where the straight line segment or its extension intersects with the tree trunk entity and P0, and to find the reference line 10 1.3m away from point P0. Its endpoints are the leftmost point pl and the rightmost point pr where the straight line intersects with the tree trunk entity; the distance between points pl and pr is calculated to obtain the diameter at breast height (DBH).

[0044] Specifically, the calculation formula for the preset three-dimensional coordinate calculation model of the tree target point is as follows:

[0045]

[0046] in, The three-dimensional coordinates of the measurement target point W in the IMU inertial navigation coordinate system b, in meters: Let O be the three-dimensional coordinates of the measurement target point W in the world coordinate system m. m The starting point is in meters (m). Let O be the three-dimensional coordinates of the measurement target point W in the camera coordinate system c. c The starting point is in meters (m). The transformation matrix from the IMU inertial navigation coordinate system b to the world coordinate system m; For GNSS receiver center O n To IMU Center O b vector O n O b Three-dimensional coordinates in the world coordinate system m, with units of meters; For the GNSS receiver center O n Three-dimensional coordinates in the world coordinate system m, with units of meters; The transformation matrix from the camera coordinate system c to the IMU inertial navigation coordinate system b; Left optical center O of the binocular camera c The three-dimensional coordinates in the IMU inertial navigation coordinate system b, with the starting point being O. b Points, in meters (m).

[0047] Further, please see Figure 3 O c For the left optical center of the binocular camera, O c ′ represents the right optical center of the binocular camera, O c O c The target point W intersects the left image acquired by the stereo camera at point e and the right image at point e'. The projections of the target point W onto the stereo camera are W1 and W2. Its three-dimensional coordinates in the camera coordinate system c are... Satisfying equation (2)

[0048]

[0049] The meanings of the symbols and variables in the above formula are as follows:

[0050] x1 and y1 are the horizontal and vertical coordinates of the target point's projection W1 on the left camera, in meters; x2 and y2 are the horizontal and vertical coordinates of the target point's projection W2 on the right camera, in meters; K1 and K2 are the 3×3 intrinsic parameter matrices of the left and right cameras; X1, Y1, and Z1 are the three-dimensional coordinates of the target point's projection W1 in the camera coordinate system c, in meters; X2, Y2, and Z2 are the three-dimensional coordinates of the target point's projection W2 in the camera coordinate system c, in meters; R is the 3×3 rotation matrix of the right camera relative to the left camera, obtained during calibration; t is the baseline distance of the right camera relative to the left camera, obtained during calibration.

[0051] Let the three-dimensional coordinates of the target point W in the camera coordinate system c be... For [XYZ] T Its projection W1 of the left camera and the left optical center O of the binocular camera c Collinear, satisfying equation (3), and with the projection W2 of the right camera and the right optical center O of the binocular camera. c 'Collinear, satisfying equation (4)'

[0052]

[0053]

[0054] Equations (3) and (4) can be rearranged to obtain equation (5).

[0055]

[0056] Therefore, the three-dimensional coordinates of the target point W in the camera coordinate system c can be obtained. It can be expressed as equation (6).

[0057]

[0058] Among them, X1, Y1, Z1, X2, Y2, Z2 are calculated by equation (2) based on the known values ​​of the image coordinates (x1, y1) and (x2, y2) of the projection W1 and W2, the intrinsic parameter matrices K1 and K2 of the left and right cameras, the rotation matrix R of the right camera relative to the left camera, and the baseline distance t of the right camera relative to the left camera.

[0059] The three-dimensional coordinates of the target point W in the camera coordinate system c are calculated. Then, the three-dimensional coordinates of the target point W in the world coordinate system m can be calculated according to equation (1). The transformation matrix from the camera coordinate system c to the IMU inertial navigation coordinate system b is... Binocular camera left optical center O c Three-dimensional coordinates in the IMU inertial navigation coordinate system b GNSS Receiver Center O n To IMU Center O b vector O n O b Three-dimensional coordinates in the world coordinate system m Given the known calibration values, the GNSS receiver center O n Three-dimensional coordinates in the world coordinate system m The transformation matrix from the IMU inertial navigation coordinate system b to the world coordinate system m, calculated from RTK observations. Calculated from IMU observations.

[0060] Specifically, the data augmentation method for the instance segmentation canopy and tree extraction model includes:

[0061] The sample image is flipped and rotated to obtain similar sample data.

[0062] The trees in the sample image are transferred and rotated to obtain similarity data of the trees in the sample image;

[0063] The sample image is subjected to color transformation processing to generate a virtual sample image;

[0064] The sample image is subjected to noise enhancement processing to obtain virtual sample enhancement data of the sample image;

[0065] The formula for the color transformation process is as follows:

[0066] P r =α m P m +λn st0.9<α m <1.1,

[0067] In the formula, P r For the virtual sample image, α m P is a light intensity parameter. m The sample image is λn, which is random Gaussian noise.

[0068] The formula for the noise enhancement process is:

[0069] P d =f(P m )+λn,

[0070] In the formula, P d For the virtual sample augmentation data, f(·) represents wavelet transform.

[0071] It should be noted that flipping and rotation processing do not change the shape information of the image; they randomly rotate the captured image vertically and horizontally. Migration and rotation processing assume that most trees in the field are similar, limited to areas with significant background differences. The labeled tree pixel positions remain unchanged, but different background areas are used, while the trees are tilted and rotated to a certain degree (e.g., ≤5°) to generate data with significant background differences and similar tree targets. It is assumed that during data acquisition, the same type of image at different locations may be affected by different lighting conditions, leading to color differences. Therefore, color transformation processing generates virtual samples under varying radiation conditions by multiplying the light intensity parameter by the actual sample, and then adds random Gaussian noise to the generated virtual samples to further simulate weather variation errors. Due to the influence of atmospheric conditions and sensor signal-to-noise ratio, the captured photos will have varying degrees of noise. Based on this, wavelet transform can be used to denoise the acquired data to eliminate the influence of other factors on the signal during imaging; to increase the robustness of the data, random Gaussian noise is added to the denoised data.

[0072] Specifically, the model architecture of the instance segmentation canopy and tree extraction model includes:

[0073] The multi-scale residual UNet network is used to extract deep features from input tree images, including feature information at different scales.

[0074] The semantic category branch is used to determine and output the type of the tree trunk and crown in the tree image based on the feature information;

[0075] The mask branch is used to standardize the input feature information to obtain standardized pixel coordinates, generate an instance mask, perform instance segmentation, and output the image coordinates of the measured target points in the tree image.

[0076] For more details, please see Figure 4 The network encoder of the multi-scale residual UNet network contains 5 multi-scale residual blocks, and its decoder contains 4 multi-scale residual blocks. The output of each multi-scale residual block of the encoder and the features of the decoder are concatenated to preserve the details.

[0077] It should be noted that, please refer to Figure 5 The multi-scale residual block consists of three 3×3 convolutions and one 1×1 convolution. It first sequentially passes the input features through the three 3×3 convolutions, with the receptive field of each convolution increasing progressively. By cascading the output features of these three convolutions, it can capture information from different input features at different scales. Compared to methods like spatial pyramid pooling with parallel connections, this approach effectively achieves feature reuse and reduces computational memory requirements. Furthermore, to preserve the detailed features of trees and canopies, the output features of the three convolutions are further cascaded with the features resulting from the 1×1 convolution of the input features, thus capturing additional spatial and detailed information.

[0078] More specifically, the semantic category branch predicts the trunk and crown types after passing through the multi-scale residual UNet network. The input tree image outputs features X∈R after passing through the multi-scale residual UNet network. H×W×B X is divided into s×s grids, with each grid cell predicting the target's class probability. Taking the first grid as an example, if a grid cell overlaps with the central region by more than a threshold, it is considered a tree; otherwise, it is considered background. Each positive example corresponds to a binary mask. This means that after marking the grid cell containing the positive example in the semantic category branch, a channel in the corresponding mask prediction branch is labeled. This involves aligning the feature map to obtain its size, then processing it through semantic category to output the class. Therefore, a corresponding relationship is established between the semantic category and the generated mask.

[0079] More specifically, the mask branch employs a fully convolutional operation. During tree instance segmentation, each cell grid must be separated by different feature channels, requiring a spatial transformation model. Therefore, the input features are first processed using the CoordConv operator to obtain more accurate feature information. The standardized pixel coordinates are then directly fed back to the network to generate an instance mask for instance segmentation. For this purpose, the feature map X∈R... H×W×B The input is fed into the mask prediction branch, and after upsampling, the image coordinates H×W×S are obtained. 2 Output.

[0080] It should be noted that, firstly, the input image is divided into s×s grid cells and corresponding one-to-one with the center positions of s×s objects. The center positions of the objects are then modeled using geometric information along the channel dimension of the feature map to generate the corresponding instance mask.

[0081] Furthermore, the training loss of the instance segmentation canopy and tree extraction model, including semantic category loss and mask prediction loss, is expressed as follows:

[0082] L = L class +λL mask ,

[0083] In the formula, L is the total loss function of the canopy and tree extraction model for the instance segmentation, L class Let L be the semantic category loss function. mask Let λ be the mask prediction loss function, and let λ be a hyperparameter, which can be set to 2 based on experience.

[0084] Specifically, the semantic category loss function is the focus loss function, which can be expressed as follows:

[0085]

[0086] In the formula, pred is the set of predicted values, and true is the set of true values.

[0087] The expression for the mask prediction loss function is:

[0088]

[0089] In the formula, the index j is k mod S, and the index order is from left to right and from top to bottom, N pos p represents the number of positive samples. * and m * Represents the predicted values ​​for classification and masking, where Π is the index function, when... If the value is 1, then the value is 0; otherwise, the value is 0.

[0090] It should be noted that the numerator of the semantic category loss function is the intersection of pred and true. It is multiplied by 2 because the denominator has the common elements between pred and true that are repeatedly calculated. The denominator of the semantic category loss function is the union of pred and true.

[0091] Figure 2 This is a schematic diagram of a tree multi-parameter image processing device provided in an embodiment of the present invention. The tree multi-parameter image processing device includes:

[0092] The model calculation module 21 is used to acquire an image of the target tree, input a preset three-dimensional coordinate calculation model of the tree target point, and obtain the three-dimensional coordinates and basic parameters of the target tree; the basic parameters include tree height parameters, crown width parameters, tree absolute position parameters, and diameter at breast height parameters.

[0093] The model extraction module 22 is used to analyze the basic parameters and three-dimensional coordinates of the canopy and tree extraction model based on instance segmentation, and obtain the image coordinates of the measurement target point of the target tree;

[0094] The coordinate mapping module 23 is used to obtain the world coordinates of the measurement target point by utilizing the obtained mapping relationship between the image coordinates of the measurement target point and the world coordinates, thereby obtaining the absolute position of the tree. By calculating the difference between the world coordinates of different target points of the tree, the tree height, diameter at breast height, and crown width information of the target tree are obtained simultaneously.

[0095] Furthermore, the aforementioned tree multi-parameter image processing device also includes:

[0096] The tree management module 24 is used to process the tree height, diameter at breast height (DBH), and crown width information of the target tree into a database for tree resource information management.

[0097] The tree multi-parameter image processing device provided in this embodiment of the invention can realize all the processes of the tree multi-parameter image processing method of the above embodiments. The functions and technical effects of each module in the device are the same as the functions and technical effects of the tree multi-parameter image processing method of the above embodiments, and will not be repeated here.

[0098] This invention provides a terminal device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps described in the above-described tree multi-parameter image processing method embodiment. Alternatively, when the processor executes the computer program, it implements the functions of each module described in the above-described tree multi-parameter image processing device embodiment.

[0099] The terminal device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the schematic diagram is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the terminal device may also include input / output devices, network access devices, buses, etc.

[0100] The processor can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting various parts of the terminal device via various interfaces and lines.

[0101] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart memory card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0102] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0103] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to perform the tree multi-parameter image processing method as described above.

[0104] In summary, the tree multi-parameter image processing method, apparatus, terminal, and medium disclosed in this embodiment of the invention acquires an image of a target tree, inputs a preset three-dimensional coordinate calculation model of the tree target point, and obtains the three-dimensional coordinates and basic parameters of the target tree; analyzes the basic parameters and three-dimensional coordinates based on the canopy and tree extraction model of instance segmentation; obtains the image-side coordinates of the measurement target point of the target tree; uses the obtained mapping relationship between the image-side coordinates of the measurement target point and the world coordinates to obtain the world coordinates of the measurement target point, obtains the absolute position of the tree, and simultaneously obtains the tree height, diameter at breast height (DBH), and crown width information of the target tree by calculating the difference between the world coordinates of different target points of the tree. Therefore, the embodiments of the present invention can acquire tree species and height parameters by acquiring binocular images in one go, processing them to obtain disparity maps, performing target detection and three-dimensional coordinate calculation, accurately locating and identifying tree objects, and selecting them by bounding boxes; it overcomes the dependence of existing mainstream tree measurement methods on the arrival point, and realizes the simultaneous acquisition of the absolute position, diameter at breast height, crown width, and tree height information of the tree to be measured. On the other hand, it combines tree measurement and tree species assessment, reduces the workload of field work, simplifies and merges them into tree image acquisition, facilitates indoor verification, and improves quality.

[0105] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for processing multi-parameter images of trees, characterized in that, include: A GNSS / IMU-based binocular camera is used to select and locate points in an open area. The mapping relationship between the image coordinates of the target point and the world coordinates is obtained by combining the known calibration values ​​of the instrument, so as to obtain the preset three-dimensional coordinate calculation model of the tree target point. Obtain a binocular image of the target tree, and based on the instance segmentation-based canopy and tree extraction model of the binocular image, obtain the image-side coordinates of the measurement target point of the target tree; Based on the image-side coordinates of the target point, the highest vertex, leftmost point, and rightmost point of the tree canopy in the binocular image are calculated respectively. The world coordinates of the tree canopy of the target tree are obtained and stored according to the preset three-dimensional coordinate calculation model of the tree target point. Calculate the distance between the leftmost and rightmost points to obtain the crown width parameters of the tree canopy; Calculate the lowest point of the tree trunk in the image, and obtain and store the world coordinates of the tree trunk based on the three-dimensional coordinate calculation model of the tree target point; Calculate the distance between the highest and lowest points to obtain the tree height parameter of the tree trunk; Using the lowest point as the search starting point, calculate the Hough transform of the tree trunk in the image, extract the horizontal straight line segment, take the midpoint of the straight line segment, and calculate and store the world coordinates of the midpoint based on the three-dimensional coordinate calculation model of the tree target point to obtain the absolute position parameters of the target tree. Using the straight line segment as a reference, search vertically upwards to calculate the spatial distance between the midpoint of the leftmost and rightmost points where the straight line segment or its extension intersects with the tree trunk entity and the midpoint of the straight line segment. Find a reference line at a preset height from the midpoint of the straight line segment. The endpoints of the reference line are the leftmost and rightmost points where the straight line segment intersects with the tree trunk entity. Calculate the distance between the leftmost and rightmost points where the tree trunks intersect, and obtain the diameter at breast height (DBH) parameter of the target tree.

2. The tree multi-parameter image processing method as described in claim 1, characterized in that, Also includes: The tree height, diameter at breast height (DBH), and crown width of the target trees are entered into a database for tree resource information management.

3. The tree multi-parameter image processing method as described in claim 1, characterized in that, The calculation formula for the preset three-dimensional coordinate calculation model of the tree target point is as follows: in, Let W be the three-dimensional coordinates of the measurement target point W in the IMU inertial navigation coordinate system b. Starting point; Let W be the three-dimensional coordinates of the measurement target point W in the world coordinate system m. Starting point; Let W be the three-dimensional coordinates of the measurement target point W in the camera coordinate system c. Starting point; The transformation matrix from the IMU inertial navigation coordinate system b to the world coordinate system m; GNSS receiver center To IMU Center vector Three-dimensional coordinates in the world coordinate system m; For the GNSS receiver center Three-dimensional coordinates in the world coordinate system m; The transformation matrix from the camera coordinate system c to the IMU inertial navigation coordinate system b; Left optical center of binocular camera The three-dimensional coordinates in the IMU inertial navigation coordinate system b, with the starting point being... point.

4. The tree multi-parameter image processing method as described in claim 1, characterized in that, The data augmentation method for the instance segmentation canopy and tree extraction model specifically includes: The sample image is flipped and rotated to obtain similar sample data. The trees in the sample image are transferred and rotated to obtain similarity data of the trees in the sample image; The sample image is subjected to color transformation processing to generate a virtual sample image; The sample image is subjected to noise enhancement processing to obtain virtual sample enhancement data of the sample image; The formula for the color transformation process is as follows: ， In the formula, The virtual sample image, For light intensity parameters, The sample image, It is random Gaussian noise; The formula for the noise enhancement process is: ， In the formula, Augmentation data for the virtual samples, This represents wavelet transform.

5. The tree multi-parameter image processing method as described in claim 1, characterized in that, The model architecture for the instance segmentation canopy and tree extraction model specifically includes: The multi-scale residual UNet network is used to extract deep features from input tree images, including feature information at different scales. The semantic category branch is used to determine and output the type of the tree trunk and crown in the tree image based on the feature information; The mask branch is used to standardize the input feature information to obtain standardized pixel coordinates, generate an instance mask, perform instance segmentation, and output the image coordinates of the measured target points in the tree image.

6. A multi-parameter image processing device for trees, characterized in that, include: The model calculation module is used to select and locate points in open areas using a GNSS / IMU-based binocular camera, and obtain the mapping relationship from the image coordinates of the target point to the world coordinates by combining the known calibration values ​​of the instrument, so as to obtain the preset three-dimensional coordinate calculation model of the tree target point. The model extraction module is used to acquire binocular images of the target tree, and to obtain the image-side coordinates of the measurement target points of the target tree based on the canopy and tree extraction model of the binocular images based on instance segmentation. The coordinate mapping module is specifically used for: Based on the image-side coordinates of the target point, the highest vertex, leftmost point, and rightmost point of the tree canopy in the binocular image are calculated respectively. The world coordinates of the tree canopy of the target tree are obtained and stored according to the preset three-dimensional coordinate calculation model of the tree target point. Calculate the distance between the leftmost and rightmost points to obtain the crown width parameters of the tree canopy; Calculate the lowest point of the tree trunk in the image, and obtain and store the world coordinates of the tree trunk based on the three-dimensional coordinate calculation model of the tree target point; Calculate the distance between the highest and lowest points to obtain the tree height parameter of the tree trunk; Using the lowest point as the search starting point, calculate the Hough transform of the tree trunk in the image, extract the horizontal straight line segment, take the midpoint of the straight line segment, and calculate and store the world coordinates of the midpoint based on the three-dimensional coordinate calculation model of the tree target point to obtain the absolute position parameters of the target tree. Using the straight line segment as a reference, search vertically upwards to calculate the spatial distance between the midpoint of the leftmost and rightmost points where the straight line segment or its extension intersects with the tree trunk entity and the midpoint of the straight line segment. Find a reference line at a preset height from the midpoint of the straight line segment. The endpoints of the reference line are the leftmost and rightmost points where the straight line segment intersects with the tree trunk entity. Calculate the distance between the leftmost and rightmost points where the tree trunks intersect, and obtain the diameter at breast height (DBH) parameter of the target tree.

7. A terminal device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the tree multi-parameter image processing method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the tree multi-parameter image processing method as described in any one of claims 1-5.

Images