A method, system, and product for tree trunk feature recognition

Abstract

This invention relates to a tree trunk feature recognition method, system, and product, belonging to the field of image processing. The method includes: establishing a dense point cloud map using LiDAR; constructing a convolutional neural network; determining a first fine-grid feature map based on the convolutional neural network; feeding the fine-grid feature map into the convolutional neural network for training to obtain a second fine-grid feature map; determining a loss function; using the loss function to perform classification regression detection on the second fine-grid feature map to obtain the tree trunk center point of each grid; and determining tree trunk features based on the tree trunk center point of each grid. The above method in this invention can extract semantic features and corresponding three-dimensional coordinates from the dense map, which can be used for subsequent rubber tapping robots to accurately locate rubber trees, complete autonomous planning and control, and realize a fully automated rubber tapping system.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a method, system and product for tree trunk feature recognition. Background Technology

[0002] Most existing automated rubber tapping robots operate on a "one tree, one machine" model, meaning one tapping machine per rubber tree. This approach is costly and difficult to maintain, and is not the direction for future technological development. The current "one machine, multiple trees" solution—an automated rubber tapping robot with a mobile chassis and tapping device—is the most suitable solution for this task. However, current mobile autonomous rubber tapping robots, due to the complex environment of rubber plantations, cannot identify and extract the characteristics of each individual rubber tree for subsequent tasks such as path planning. Therefore, this invention utilizes an autonomous mobile tracked chassis, uses LiDAR to create a dense point cloud map, and then employs a convolutional neural network to extract trunk information, obtaining accurate trunk features and 3D coordinates for subsequent navigation, planning, and rubber plantation management. Summary of the Invention

[0003] The purpose of this invention is to provide a tree trunk feature recognition method, system, and product. It uses a convolutional neural network to extract semantic features and corresponding three-dimensional coordinates from a dense map, which are then used by a rubber tapping robot to accurately locate rubber trees, complete autonomous planning and control, and realize a fully automated rubber tapping system.

[0004] To achieve the above objectives, the present invention provides the following solution:

[0005] In a first aspect, the present invention provides a tree trunk feature recognition method, the recognition method comprising:

[0006] Establish a dense point cloud map using lidar;

[0007] Constructing a convolutional neural network;

[0008] The first fine-grid feature map is determined based on the convolutional neural network;

[0009] The fine mesh feature map is fed into the convolutional neural network for training to obtain a second fine mesh feature map;

[0010] Determine the loss function;

[0011] The loss function is used to perform classification and regression detection on the second fine grid feature map to obtain the trunk center point of each grid.

[0012] The trunk features are determined based on the trunk center point of each grid.

[0013] Optionally, the establishment of the dense point cloud map by lidar specifically includes the following steps:

[0014] Obtain a LiDAR point cloud map;

[0015] The LiDAR point cloud map is subjected to distortion correction processing to obtain an initial dense point cloud map; the initial dense point cloud map contains one frame of point cloud.

[0016] Geometric line and surface features are extracted based on the initial dense point cloud map; the geometric line and surface features include three-dimensional straight line features and three-dimensional planar features.

[0017] The optimal robot pose is solved using a nonlinear optimization method based on its geometric line and surface characteristics.

[0018] Obtain the pose estimated by the IMU;

[0019] The optimal odometry is obtained by fusing the robot's optimal pose with the pose estimated by the IMU.

[0020] The dense point cloud map of the lidar is determined based on the optimal odometer.

[0021] Optionally, distortion correction processing is performed on the lidar point cloud map, specifically by using cubic spline interpolation to transform the discretely estimated pose of the IMU into a continuously estimated pose.

[0022] Optionally, determining the first fine-grid feature map based on the convolutional neural network specifically includes the following steps:

[0023] The dense point cloud map of the lidar is projected onto a plane using a top-down view and divided into H*W grids, each grid containing K points, resulting in H*W*K points;

[0024] The point cloud features of the H*W*K points are upscaled to obtain features of K*2m dimensions;

[0025] The point cloud features are adjusted using a self-attention mechanism to obtain features of a fixed size that are independent of K.

[0026] The fixed-size features independent of K are processed through fully connected layers and convolutional layers to obtain a 48-dimensional vector;

[0027] Repeat the above steps until all H*W grids have been processed, resulting in a fine grid feature map of size H*W*48, which is the first fine grid feature map.

[0028] Optionally, the expression for the loss function is as follows:

[0029]

[0030] Among them, L cls L represents the cross-entropy loss function for binary classification;cls (x seg y seg ) in x seg y seg This indicates the first output and first label predicted by the backbone network, i.e., foreground or background; L cls (x cls y cls ) in x cls y cls This represents the second output and second label of the backbone network prediction, indicating whether it is a tree; L s L represents the smoothing L1 loss function, L s (x center y center ) in x center y center This represents the output and label of the predicted center point of the backbone network; L s (x z y z ) in x z y z Z represents the height Z and label of the backbone network prediction, α1, α2, and α3 represent the weights; x represents the pixel points involved in the calculation, I represents the set of all points, and I' represents the set of positive sample (foreground) points.

[0031] Optionally, determining the trunk features based on the trunk center point of each grid specifically includes the following steps:

[0032] Determine whether the center point of two adjacent grids is less than a preset threshold;

[0033] If it is less than 1, then the two adjacent networks are merged into a single trunk to obtain the trunk feature.

[0034] Optionally, the preset threshold is 0.5m.

[0035] Secondly, based on the above-described method of this invention, this invention provides a tree trunk feature recognition system, the recognition system comprising:

[0036] The LiDAR dense point cloud map construction module is used to build LiDAR dense point cloud maps.

[0037] The Convolutional Neural Network Building Block is used to build convolutional neural networks;

[0038] The first fine-scale mesh feature map determination module is used to determine the first fine-scale mesh feature map based on the convolutional neural network;

[0039] The second fine mesh feature map determination module is used to feed the fine mesh feature map into the convolutional neural network for training to obtain the second fine mesh feature map;

[0040] The loss function determination module is used to determine the loss function;

[0041] The trunk center point determination module is used to perform classification and regression detection on the second fine grid feature map using the loss function to obtain the trunk center point of each grid.

[0042] The trunk feature determination module is used to determine the trunk features based on the trunk center point of each grid.

[0043] Thirdly, the present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the above-described tree trunk feature recognition method.

[0044] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described tree trunk feature recognition method.

[0045] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0046] The method described in this invention constructs a lidar odometry based on the geometric features of the environment (i.e., three-dimensional line features and three-dimensional plane features). It uses an extended Kalman filter to fuse the lidar odometry and IMU pre-integration pose to obtain optimized odometry data. It uses hash voxels to efficiently maintain a local sub-map for lidar point cloud matching. At the same time, the innovative local sub-map maintenance scheme provides a point cloud with a suitable size and density for the convolutional neural network to extract semantic features of rubber trees, which greatly improves the efficiency and robustness of semantic feature extraction. The convolutional neural network is used to extract semantic features and corresponding three-dimensional coordinates from the dense map, which can be used for subsequent rubber tapping robots to accurately locate rubber trees, complete autonomous planning and control, and realize a fully automatic rubber tapping system. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of the tree trunk feature recognition method of the present invention;

[0049] Figure 2 This is a schematic diagram of a dense point cloud map for the lidar system of this invention.

[0050] Figure 3This is a schematic diagram of the K*2m dimension feature color generation process of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] The purpose of this invention is to provide a tree trunk feature recognition method, system, and product. It uses a convolutional neural network to extract semantic features and corresponding three-dimensional coordinates from a dense map, which are then used by a rubber tapping robot to accurately locate rubber trees, complete autonomous planning and control, and realize a fully automated rubber tapping system.

[0053] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] Figure 1 This is a flowchart of the tree trunk feature recognition method of the present invention, as shown below. Figure 1 As shown, the method in this invention includes:

[0055] Step 1: Create a dense point cloud map using lidar.

[0056] Step 1 specifically includes:

[0057] Step 1.1: Obtain the LiDAR point cloud map.

[0058] Step 1.2: Perform distortion correction processing on the lidar point cloud map to obtain an initial dense point cloud map; the initial dense point cloud map contains one frame of point cloud.

[0059] The point cloud collected by mechanical lidar during robot movement will be distorted, so distortion compensation is needed to build an accurate point cloud map.

[0060] When the robot moves forward, the points collected by the lidar are not collected instantaneously, so a frame of point cloud will experience motion distortion (i.e., the position of the points changes due to the robot's forward movement). By compensating for motion distortion, a distortion-free lidar point cloud is finally obtained, making subsequent dense point cloud mapping more accurate.

[0061] This invention uses a MEMS inertial measurement unit (IMU) to measure the robot's linear acceleration and angular velocity. IMU pre-integration is used to calculate the robot's position and attitude, thereby compensating for motion distortion in all point clouds within a LiDAR frame. This ensures that all points in a LiDAR frame are compensated up to the last moment of that frame's data. However, because the IMU's measurement frequency is 200Hz (1 measurement every 5ms), its estimated pose is still discrete and cannot correspond to every single LiDAR point.

[0062] Therefore, this invention proposes a method based on cubic B-spline interpolation to interpolate and compensate the poses of two IMU frames (i.e., 5ms apart) to obtain a time-continuous trajectory. This time-continuous trajectory is then used for distortion compensation of LiDAR points.

[0063] The robot's position, orientation, and velocity are p, v, and R, respectively. The motion compensation process is shown below:

[0064] p j-1 =p j -v j Δt

[0065]

[0066]

[0067] Where p, v, and R represent the robot's position, velocity, and attitude, respectively; Δt is the time difference between two IMU frames; a and ω are the acceleration and angular velocity acquired by the IMU; and b... a and b g Let a and w be the deviations. We use the above formula for motion distortion compensation, but it is a discrete process. The time interval is based on the detection frequency of the IMU sensor, which is generally around 200Hz, but the frequency of each point of the LiDAR is around 30kHz. Therefore, discrete motion compensation cannot perform motion distortion correction for every point of the LiDAR. This invention uses cubic spline interpolation to interpolate the discrete process into a continuous time process.

[0068] The cubic B-spline interpolation process is as follows:

[0069] P(t) = P0F 0,3 (t)+P1F 1,3 (t)+P2F 2,3 (t)+P3F 3,3 (t)

[0070] in:

[0071]

[0072]

[0073]

[0074]

[0075] P1, P2, and P3 are three discrete points for interpolation, representing the robot poses estimated by the three IMUs. t represents time, and the final continuous trajectory output P(t) is a function of time t.

[0076] Between two IMU frames (assuming the IMU is 200Hz), i.e., within 5ms, cubic spline interpolation is used to interpolate the poses at the two time points to obtain a continuous trajectory. Thus, the discrete IMU poses can be transformed into a smooth, continuous trajectory. Motion compensation is then performed based on the scanning time of each point on the LiDAR. Finally, a distortion-free point cloud frame is obtained, making the subsequent mapping process more accurate.

[0077] The distortion correction process is characterized by the innovative use of cubic spline interpolation to transform the discrete pose estimated by the IMU into a continuous pose estimate, making the obtained point cloud more accurate and providing a foundation for the subsequent creation of an accurate point cloud map.

[0078] Step 1.3: Extract geometric line and surface features based on the initial dense point cloud map; the geometric line and surface features include three-dimensional straight line features and three-dimensional planar features.

[0079] This algorithm makes full use of the geometric features in the environment and uses surface features (i.e., the three-dimensional plane extracted from the point cloud) to perform point-to-surface matching, thereby optimizing the robot pose and obtaining the corresponding odometry.

[0080] This invention employs Principal Component Analysis (PCA) to extract geometric line and surface features from the dense point cloud map composed of the aforementioned distortion-free point cloud (this dense point cloud map is the initial dense point cloud map, which initially consists of only one frame of point cloud, and is subsequently accumulated based on the estimated odometry). Based on these geometric line and surface features, a nonlinear optimization method is used to solve for the robot's optimal pose, and the front-end odometry is calculated based on this optimal pose.

[0081] The specific steps are as follows:

[0082] Step 1.3.1: First, use the RANSAC algorithm to separate the ground. Its core idea is random sampling consistency. Assuming the ground is a plane, since nursing robots mostly work indoors and the ground is mostly horizontal, the algorithm is used to fit the plane at the bottom of the point cloud, which is the ground point.

[0083] Step 1.3.2: After removing ground points, the DBSCAN density clustering algorithm is used to cluster the denser points among the non-ground points to determine whether they are 3D lines or 3D planes. Principal Component Analysis (PCA) is then used to fit the 3D lines and planes to obtain 3D planes and 3D lines based on the dense point cloud. Specifically, the covariance matrix of the selected point cloud coordinates is first calculated, then eigenvalues ​​and corresponding eigenvectors are extracted, and the eigenvalues ​​and eigenvectors are used to determine whether it is a plane or a line. In subsequent robot pose estimation, the corresponding planar points are paired with the 3D plane to calculate the residuals, and finally, the robot's pose estimate is obtained to construct the dense point cloud map.

[0084] Specifically, we first calculate the covariance matrix C, where the formula for the covariance of the x and y coordinates of the point cloud is as follows:

[0085]

[0086] Similarly, the covariance matrix C of the point cloud is finally obtained:

[0087]

[0088] Solving for the eigenvalues ​​and decomposition of the covariance matrix, the eigenvalues ​​are arranged from largest to smallest as ρ∈{ρ1,ρ2,ρ3}. When the largest eigenvalue ρ1 > 3ρ2 and ρ1 > 3ρ3, the cluster is considered a 3D linear feature, and the eigenvector v1 corresponding to the largest eigenvalue ρ1 is taken as the principal direction of the 3D line. The 3D plane discrimination method is as follows: if the largest eigenvalue is the smallest eigenvalue ρ3 < s (where s is a user-defined small value) and does not satisfy the condition that the largest eigenvalue ρ1 > 3ρ2 and ρ1 > 3ρ3, then the cluster is considered a plane. Specifically, 3D planar point clouds are divided into ground point clouds and non-ground point clouds. Ground point clouds can effectively constrain the pitch and roll angles of rotation and the pose in the z-direction, while non-ground point clouds can constrain the pose in the yaw and xy directions.

[0089] The covariance matrix calculated above is used to determine whether the point cloud belongs to a 3D planar feature or a 3D straight line feature. We will use these features to calculate the point-to-line and point-to-surface matching to obtain the robot's final pose transformation, i.e., to obtain accurate odometry data for subsequent dense point cloud mapping.

[0090] Step 1.4: Solve the optimal pose of the robot using a nonlinear optimization method based on its geometric line and surface characteristics.

[0091] Step 1.5: Obtain the pose estimated by the IMU.

[0092] Step 1.6: Fuse the optimal pose of the robot with the pose estimated by the IMU to obtain the optimal odometry.

[0093] Step 1.7: Determine the dense point cloud map of the lidar based on the optimal odometer.

[0094] Based on the 3D plane and 3D line information obtained in step 1.3 above, the residual from the corresponding plane point to the plane is calculated, and this residual is optimized to obtain the final optimal pose estimate. For the lidar point p... j And the center point q of the surface element that matches this point. j The residual constraints established from the point to the plane are as follows:

[0095]

[0096] Where n j It is the normal vector of the surface element.

[0097] Based on this residual, a nonlinear optimization method is used to obtain the optimal robot pose, which serves as the optimal radar odometry estimate.

[0098] Based on the residuals obtained above, a first-order approximation is performed to obtain the Jacobian matrix H, which is used to subsequently fuse the predicted IMU-estimated pose with the observed radar residuals to obtain the optimal pose estimate. The first-order approximation equation is as follows:

[0099]

[0100] Then the matrix can be calculated. For subsequent fusion.

[0101] The pose estimated by IMU and the pose estimated by radar odometry are fused to obtain the final optimal odometry, which is then used for subsequent dense point cloud mapping.

[0102] Finally, the extended Kalman filter algorithm with error state is used to fully integrate the pose estimated by IMU and the pose estimated by lidar odometry to achieve the best state estimation effect.

[0103] The merging and updating process is as follows:

[0104]

[0105]

[0106]

[0107] in P is the error of the robot's state variables to be estimated, H is the Jacobian matrix obtained by linearizing the radar point cloud matching equation, and P is the error of the state variables to be estimated. k and These are the optimal estimate and the predicted value of the covariance matrix of the robot's state variables, respectively.

[0108] The Kalman filter algorithm described above is used to fuse the pose estimates from the IMU and radar odometry to obtain the optimal odometry. Finally, all the point clouds are superimposed to obtain a dense point cloud map. Because a mechanical lidar is used, its measurement angle is 360°, which can fully establish a dense point cloud map of the surrounding environment. The detected objects all have true scale (i.e., true size) and accurate depth information.

[0109] A dense rubber forest point cloud map obtained by stitching together single-frame point clouds, such as... Figure 2 As shown, it is used for subsequent segmentation of the rubber tree body.

[0110] Step 2: Construct a convolutional neural network.

[0111] Step 3: Determine the first fine mesh feature map based on the convolutional neural network.

[0112] Specifically, the following steps are included:

[0113] Step 3.1: KNN (k-nearest neighbor) encoding. Using a top-down view, the dense 3D point cloud map is projected onto a plane and divided into H*W grids. Each grid has K points. If there are not enough points, zeros are added. If there are too many points, the K-nearest neighbor algorithm is used to compensate. Finally, H*W*K points are obtained.

[0114] Specifically, the top-down view of the point cloud map is divided into a fine H*W grid. K points are retained in each grid. For grids with no or few points, the K-nearest neighbor algorithm is used to add points from surrounding grids. Points in each grid are represented by P... i Description, P i ={(g x ,g y ),(O x O y ),(z i ,r i )}, where g x ,g y Represents the coordinates of the top-left corner point, (O x O y The coordinates z ∈ [0,1] represent the offset of the point relative to the top-left corner. i ,r i These represent the altitude and radar reception strength of the point, respectively. For each grid cell, {P0, P1, ..., P...} are defined. k-1 There are K such points in total, resulting in H*W*K points.

[0115] Step 3.2: Upscale the point cloud features using a fully connected FC layer and a stacked Voxel feature encoding layer (VFE) to obtain K*2m dimensional features.

[0116] Specifically, select H*W*K points obtained in step 3.1, and represent each point as P. i ={(g x ,g y ),(O x O y ),(z i ,r i Each grid cell has K points, corresponding to a K*6 dimensional vector. A fully connected layer is used to increase the dimensionality of this vector to k*m dimensions. A VFE layer is then applied to the k*m vectors, and the vectors are concatenated to obtain a K*2m dimensional feature. The process is as follows: Figure 3 As shown.

[0117] For each grid cell, points of size K*6 are increased to size K*m by a fully connected layer, and then passed through a VFE layer. For each vector of size K*1, the maximum value in the 1*m dimension is concatenated to obtain a feature of size K*2m.

[0118] This approach strengthens the representation of each point among the K points in the current cell, providing rich information for attention and feature extraction. Let n = 2m.

[0119] Step 3.3: Adjust the point cloud features using the Transformer self-attention mechanism. Perform self-attention processing on the K*n dimension (where 2m = n) features obtained in Step 3.2 above to obtain features of a fixed size that are independent of K.

[0120] For the K*n dimension feature F obtained by feature upscaling, the points need to be reorganized into a regular and ordered representation through the self-attention mechanism, which automatically determines which point should play a key role and attract our more attention in order to extract stronger features.

[0121] Let f be the mapping after the Transformer, then we have

[0122]

[0123] Therefore, the characteristics of the self-attention mechanism are: The influence of K on the features predicted by the input network.

[0124] Step 3.4: Vectorize cell-level features and reorganize fine map features.

[0125] For each cell n*n obtained above, the features (i.e. We obtained n*n dimensional features, where f is k*n dimensional, its transpose is n*k dimensional, and F is k*n dimensional. Multiplying the two together gives us the n*n dimensional features. These features are then processed through fully connected layers and convolutional layers to obtain a 48-dimensional vector.

[0126] Point cloud features enhanced by self-attention mechanism After being transformed into a 48-dimensional vector by fully connected layers and convolutional layers, and after the above operations are performed on H*W grids, a fine grid feature map of size H*W*48 can be obtained, which is the first fine grid feature map. It contains the feature information that needs to be focused on for each point in each grid, and is used for subsequent feature extraction networks.

[0127] Step 4: Feed the fine mesh feature map into the convolutional neural network for training to obtain the second fine mesh feature map.

[0128] Considering the speed and accuracy requirements of this application scenario, the Backbone (i.e., the backbone network, as shown in Table 1) adopts a stacked hourglass network structure design. The network outputs an H*W*5 result, which is the second fine-grained grid feature map. Here, H*W represents each fine cell. In the 5-dimensional array, the first value represents the foreground score, with values ​​close to 1 indicating foreground and values ​​close to 0 indicating background. The second and third values ​​represent the center point of the object that may exist in the fine-grained grid, and the fourth and fifth values ​​represent the upper and lower limits of the height, respectively.

[0129] Table 1 Backbone Network Structure Parameters

[0130]

[0131]

[0132] Step 5: Determine the loss function.

[0133] Step 6: Use the loss function to perform classification and regression detection on the second fine grid feature map to obtain the trunk center point of each grid.

[0134] The loss for determining the foreground and background and segmentation is BCE loss, while the loss for regressing the object center point and upper and lower limits is SmoothL1 loss. The overall loss is the result of weighting the two losses with different proportions, as shown below.

[0135] The H*W*5 feature map obtained in step 4 above is used for classification and regression using the weighted overall loss, which determines whether it is a tree trunk and the specific length and size of the trunk. For classification and regression, only the loss of positive samples is calculated, that is, only the loss of features that are tree trunks is calculated, and the loss of features that are not tree trunks is not calculated.

[0136] The loss function is as follows:

[0137] BCELoss:

[0138] l cls =y i ·logσ(x i )+(1-y i )·log(1-σ(x i ))

[0139] SmoothL1 loss

[0140]

[0141] Overall Loss:

[0142]

[0143] Step 7: Determine the trunk features based on the trunk center point of each grid.

[0144] Because the grid is fine enough and the distance between rubber trees in the actual rubber plantation is statistically analyzed, targets whose center point distance between each grid prediction is less than 0.5m are considered as the same target. The instance segmentation results are then merged to achieve accurate trunk segmentation results.

[0145] That is, H*W grids are divided, and the center point of the tree trunk is obtained by detecting the tree trunk in each grid. If the center point of the tree trunk obtained by each grid is less than 0.5m, the targets of the two grids are merged into the same tree trunk.

[0146] Based on the above-described method of this invention, this invention further provides a tree trunk feature recognition system, the recognition system comprising:

[0147] The LiDAR dense point cloud map construction module is used to create LiDAR dense point cloud maps.

[0148] The Convolutional Neural Network Building Module is used to build convolutional neural networks.

[0149] The first fine mesh feature map determination module is used to determine the first fine mesh feature map based on the convolutional neural network.

[0150] The second fine mesh feature map determination module is used to feed the fine mesh feature map into the convolutional neural network for training to obtain the second fine mesh feature map.

[0151] The loss function determination module is used to determine the loss function.

[0152] The trunk center point determination module is used to perform classification and regression detection on the second fine grid feature map using the loss function to obtain the trunk center point of each grid.

[0153] The trunk feature determination module is used to determine the trunk features based on the trunk center point of each grid.

[0154] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the above-described tree trunk feature recognition method.

[0155] In practical applications, the aforementioned electronic devices can be servers.

[0156] In practical applications, electronic devices include: at least one processor, memory, bus, and communication interface.

[0157] The processor, communication interface, and memory communicate with each other via a communication bus.

[0158] A communication interface is used to communicate with other devices.

[0159] The processor is used to execute programs, specifically the methods described in the above embodiments.

[0160] Specifically, the program may include program code, which includes computer operation instructions.

[0161] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The electronic device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0162] Memory is used to store programs. Memory may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device.

[0163] Based on the description of the above embodiments, this application provides a storage medium storing computer program instructions thereon, which can be executed by a processor to implement the methods described in any embodiment.

[0164] The tree trunk feature recognition system provided in this application exists in various forms, including but not limited to:

[0165] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0166] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access capabilities. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0167] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes: audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0168] (4) Other electronic devices with data interaction functions.

[0169] Specific embodiments of the subject matter have now been described. Other embodiments are within the scope of the appended claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing can be advantageous.

[0170] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0171] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components. Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0172] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0173] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0174] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0175] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0176] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0177] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, and CD-ROM.

[0178] Digital multifunction optical disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape, disk storage or other magnetic storage devices

[0179] Or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0180] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0181] This application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific transactions or implement specific abstract data types. This application can also be practiced in distributed computing environments where transactions are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0182] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0183] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for identifying tree trunk features, characterized in that, The identification method includes: Establish a dense point cloud map using lidar; Constructing a convolutional neural network; The first fine-grid feature map is determined based on the convolutional neural network; The fine mesh feature map is fed into the convolutional neural network for training to obtain a second fine mesh feature map; Determine the loss function; The loss function is used to perform classification and regression detection on the second fine grid feature map to obtain the trunk center point of each grid. The trunk features are determined based on the trunk center point of each grid. The establishment of a dense point cloud map using lidar specifically includes the following steps: Obtain a LiDAR point cloud map; The LiDAR point cloud map is subjected to distortion correction processing to obtain an initial dense point cloud map; the initial dense point cloud map contains one frame of point cloud. Geometric line and surface features are extracted based on the initial dense point cloud map; the geometric line and surface features include three-dimensional straight line features and three-dimensional planar features. The optimal robot pose is solved using a nonlinear optimization method based on its geometric line and surface characteristics. Obtain the pose estimated by the IMU; The optimal odometry is obtained by fusing the robot's optimal pose with the pose estimated by the IMU. Based on the optimal odometer, a dense point cloud map of the lidar is determined; Determining the first fine-grid feature map based on the convolutional neural network specifically includes the following steps: The dense point cloud map of the lidar is projected onto a plane using a top-down view and divided into H*W grids, each grid containing K points, resulting in H*W*K points; The point cloud features of the H*W*K points are upscaled to obtain features of K*2m dimensions; The point cloud features of the K*2m dimension are adjusted using a self-attention mechanism to obtain features of a fixed size that are independent of K; The fixed-size features independent of K are processed through fully connected layers and convolutional layers to obtain a 48-dimensional vector; Repeat the above steps until all H*W grids have been processed, resulting in a fine grid feature map of size H*W*48, which is the first fine grid feature map.

2. The tree trunk feature recognition method according to claim 1, characterized in that, The LiDAR point cloud map is subjected to distortion correction processing, specifically by using cubic spline interpolation to transform the discretely estimated pose of the IMU into a continuously estimated pose.

3. The tree trunk feature recognition method according to claim 1, characterized in that, The expression for the loss function is as follows: Among them, L cls L represents the cross-entropy loss function for binary classification; cls (x seg y seg ) in x seg y seg This indicates the first output and first label predicted by the backbone network, i.e., foreground or background; L csl (x cls y cls ) in x cls y cls This indicates the second output and second label of the backbone network prediction, i.e., whether it is a tree; L s L represents the smoothing L1 loss function, L s (x center y center ) in x center y center This represents the output and label of the predicted center point of the backbone network; L s (x z y z ) in x z y z Z represents the height Z and label of the backbone network prediction, α1, α2, and α3 represent the weights, x represents the pixel points involved in the calculation, I represents the set of all points, and I' represents the set of positive sample points.

4. The tree trunk feature recognition method according to claim 1, characterized in that, Determining the trunk features based on the trunk center point of each grid specifically includes the following steps: Determine whether the center point of two adjacent grids is less than a preset threshold; If it is less than, then the two adjacent networks are merged into one trunk to obtain the trunk feature.

5. The tree trunk feature recognition method according to claim 4, characterized in that, The preset threshold is 0.5m.

6. A tree trunk feature recognition system, characterized in that, The identification system includes: The LiDAR dense point cloud map construction module is used to build LiDAR dense point cloud maps. The Convolutional Neural Network Building Block is used to build convolutional neural networks; The first fine-scale mesh feature map determination module is used to determine the first fine-scale mesh feature map based on the convolutional neural network; The second fine mesh feature map determination module is used to feed the fine mesh feature map into the convolutional neural network for training to obtain the second fine mesh feature map; The loss function determination module is used to determine the loss function; The trunk center point determination module is used to perform classification and regression detection on the second fine grid feature map using the loss function to obtain the trunk center point of each grid. The trunk feature determination module is used to determine the trunk features based on the trunk center point of each grid. The establishment of a dense point cloud map using lidar specifically includes the following steps: Obtain a LiDAR point cloud map; The LiDAR point cloud map is subjected to distortion correction processing to obtain an initial dense point cloud map; the initial dense point cloud map contains one frame of point cloud. Geometric line and surface features are extracted based on the initial dense point cloud map; the geometric line and surface features include three-dimensional straight line features and three-dimensional planar features. The optimal robot pose is solved using a nonlinear optimization method based on its geometric line and surface characteristics. Obtain the pose estimated by the IMU; The optimal odometry is obtained by fusing the robot's optimal pose with the pose estimated by the IMU. Based on the optimal odometer, a dense point cloud map of the lidar is determined; Determining the first fine-grid feature map based on the convolutional neural network specifically includes the following steps: The dense point cloud map of the lidar is projected onto a plane using a top-down view and divided into H*W grids, each grid containing K points, resulting in H*W*K points; The point cloud features of the H*W*K points are upscaled to obtain features of K*2m dimensions; The point cloud features of the K*2m dimension are adjusted using a self-attention mechanism to obtain features of a fixed size that are independent of K; The fixed-size features independent of K are processed through fully connected layers and convolutional layers to obtain a 48-dimensional vector; Repeat the above steps until all H*W grids have been processed, resulting in a fine grid feature map of size H*W*48, which is the first fine grid feature map.

7. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the tree trunk feature recognition method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the trunk feature recognition method as described in any one of claims 1-5.

Images