Self-propelled woodland multifunctional wrecker control system combined with visual detection

Abstract

The invention discloses a self-propelled woodland multifunctional wrecker control system combined with visual inspection, and relates to the technical field of intelligent control, the self-propelled woodland multifunctional wrecker control system comprises a wrecker control platform, the wrecker control platform is in communication connection with a visual inspection module, an operation positioning module, a path planning module and a wrecker operation execution module, the modules are in electric signal connection; the visual detection module is used for capturing and preprocessing image data of the forest environment and monitoring obstacles and operation areas around the wrecker in real time. The forest environment image is captured through the camera, the type, size and position of the obstacle can be accurately identified by using the image processing and feature extraction technology, and the accurate position information of the wrecker and the obstacle in the three-dimensional space is further provided by using the three-dimensional model constructed by the operation positioning module. Therefore, the obstacle clearing operation can be accurately performed aiming at a specific target, the operation precision is effectively improved, and the condition of incomplete clearing or misoperation is reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology, and in particular to a control system for a self-propelled multifunctional forest tow truck combined with visual detection. Background Art

[0002] Forest areas are usually faced with obstacles such as fallen trees, overgrown bushes, and dry trees. These obstacles will affect the passage, ecological environment and fire prevention management of the forest. Therefore, an efficient and environmentally friendly way is needed to clear these obstacles to ensure the smooth flow and safety of the forest area. Self-propelled forest multi-functional tow trucks came into being. They can efficiently handle various forest obstacles and meet the needs of forest management and maintenance. With the improvement of ecological environmental protection awareness, the management of forest areas pays more and more attention to the protection of the ecological environment. Self-propelled tow trucks have a lower environmental impact. Their use can reduce the disturbance of forest soil and the damage to the ecological environment, which helps to achieve sustainable forestry management.

[0003] In the existing technology, when performing delicate operations such as tree pruning and obstacle clearing, tow trucks have the problem of insufficient precision, resulting in low work efficiency or incomplete cleaning. Therefore, how to combine visual detection with machine learning algorithms to accurately locate the work area and conduct detailed analysis of obstacles to achieve precise cleaning and improve the accuracy of tow trucks when performing delicate tasks is the problem to be solved by the present invention. To this end, a self-propelled forest multi-functional control system combined with visual detection is proposed. Summary of the Invention

[0004] The object of the present invention is to provide a self-propelled multifunctional forest tow truck control system combined with visual detection to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A self-propelled multifunctional forest tow truck control system combined with visual detection includes a tow truck control platform, wherein the tow truck control platform is communicatively connected to a visual detection module, an operation positioning module, a path planning module, and an obstacle removal operation execution module, wherein the modules are electrically connected;

[0007] The visual detection module is used to capture and pre-process image data of the forest environment and monitor obstacles and operating areas around the tow truck in real time;

[0008] The operation positioning module is used to perform feature analysis on the pre-processed image data, build a three-dimensional model of the operation area, mark the positions of the tow truck and each obstacle target in the three-dimensional space, and realize the precise positioning of the tow truck;

[0009] The path planning module is used to plan the optimal working path for the tow truck based on the positioning information provided by the operation positioning module, combined with map data and real-time environment, and adjust the path in real time to avoid obstacles;

[0010] The obstacle removal operation execution module is used to combine the path planning and the positioning results of the obstacle target to generate corresponding operation instructions, and control the actuator of the tow truck to perform precise operations, thereby realizing the automation and intelligent operation of the tow truck.

[0011] A further improvement of the technical solution of the present invention is that the visual detection module specifically includes:

[0012] The high-resolution camera installed on the tow truck captures real-time image data of the forest environment, including various obstacles such as trees, rocks, low vegetation, and the general outline of the work area. The camera continuously collects images of the environment around the tow truck based on the preset sampling frequency and viewing angle range, and transmits the collected image data in the form of digital signals to the tow truck control platform;

[0013] Preprocessing the captured forest environment image data, including denoising, contrast enhancement and grayscale operations;

[0014] The pre-processed image data of the forest environment are integrated and stored in the data warehouse of the tow truck control platform, and are classified and sorted by camera number and timestamp.

[0015] A further improvement of the technical solution of the present invention is that: the operation positioning module includes a feature extraction unit and a three-dimensional modeling positioning unit;

[0016] The feature extraction unit is used to extract and identify features from the pre-processed image data, distinguish and mark the type, size and position of obstacles;

[0017] The three-dimensional modeling and positioning unit is used to combine the image data of visual detection and the identification and annotation results of obstacles to build a three-dimensional model of the working area, provide a three-dimensional visual representation of the working area, and enable the tow truck to accurately understand its own position and the target position.

[0018] A further improvement of the technical solution of the present invention is that the feature extraction unit specifically includes:

[0019] The pre-processed forest environment image data is segmented to identify the outlines of objects in the image, dividing continuous pixel areas into different objects. Then, a region growing algorithm is used to gradually expand the area starting from the seed point based on the pixel similarity criterion, further accurately distinguishing the boundary between obstacles and the surrounding environment, making obstacles appear more clearly as independent areas in the image.

[0020] Perform feature extraction on the segmented obstacle image to extract key feature information in the image, including shape features, texture features, and color features. Shape features include contour, area, and aspect ratio; texture features include gray-level co-occurrence matrix and texture direction; and color features include average color value and color distribution.

[0021] Analyze the extracted key feature information and use the obstacle classification model pre-built based on the decision tree model to classify obstacles, distinguish different types of obstacles, and identify their types;

[0022] The classified obstacle information is labeled, including the type, size and location of the obstacle, and the outline of each obstacle is marked in the image, recording its type, size and coordinate position in the image.

[0023] A further improvement of the technical solution of the present invention is that the process of distinguishing different types of obstacles using the obstacle classification model is as follows:

[0024] The extracted shape features, texture features, and color features are sorted and normalized to ensure that all feature information has a unified format and consistent dimensions. The sorted feature information is then input into a pre-built obstacle classification model, which then performs classification based on the input feature information and the preset decision rules.

[0025] The obstacle classification model starts from the root node and classifies obstacles based on preset feature thresholds. Obstacles are divided into three categories based on area: large, medium, and small. Within each subcategory, obstacles are further subdivided based on shape features. Finally, the specific type of obstacle is determined based on texture and color features.

[0026] After the obstacle classification model has been analyzed step by step, the final classification result of the obstacle is output to clarify the type of obstacle.

[0027] A further improvement of the technical solution of the present invention is that the three-dimensional modeling and positioning unit specifically includes:

[0028] Receives image data from the visual inspection module and obstacle identification and annotation results from the feature extraction unit, and adjusts the viewing angle and resolution of the image data through a calibration algorithm to eliminate deviations caused by differences in camera position and angle;

[0029] Based on the calibrated image data, a feature matching algorithm is used to find the same feature points in multiple images. By matching the feature points and combining the camera's intrinsic and extrinsic parameters, the depth information of the feature points (i.e., the distance from the feature points to the camera) is calculated using the triangulation principle. A stereo matching algorithm is then used to directly estimate the depth value of each pixel in the image, gradually constructing a three-dimensional depth map of the work area.

[0030] Based on the 3D depth map and combined with the annotation information of obstacles, a 3D model of the work area is constructed. The spatial coordinates of each point in the scene are determined based on the depth information. A 3D reconstruction algorithm is used to connect these points into surfaces, forming the 3D geometric shape of the obstacles and the work area. At the same time, the position information of the tow truck itself is also integrated into the 3D model to clarify the relative position of the tow truck in the work area, thus constructing a complete 3D model of the work area.

[0031] The constructed three-dimensional model of the work area is visually rendered, and the overall picture of the work area is displayed in an intuitive three-dimensional graphical interface, including the distribution, shape, size of obstacles and the location information of the tow truck. At the same time, the position information of the tow truck in the three-dimensional model of the work area is fed back to the control system of the tow truck, providing an accurate spatial reference for the navigation, obstacle avoidance and operation path planning of the tow truck.

[0032] A further improvement of the technical solution of the present invention is that the process of constructing the three-dimensional model of the working area is as follows:

[0033] Extract the depth value of each pixel from the 3D depth map, and use the depth map to provide distance information from the camera to each point in the scene;

[0034] Convert the pixel points in the depth map from the image coordinate system to the camera coordinate system, and for each pixel in the depth map in the image, calculate its three-dimensional coordinates in the camera coordinate system;

[0035] Combine the obstacle annotation information provided by the feature extraction unit with the depth information, where the obstacle annotation information includes the type, location, and range of the obstacle. For each obstacle, extract the corresponding depth value in the depth map based on its annotated outline and calculate the vertex coordinates of the obstacle outline;

[0036] The Poisson reconstruction algorithm is used to connect the point cloud data of obstacles and the working area into a surface to form a complete 3D geometric shape. By solving the Poisson equation, an implicit surface is obtained, and then a 3D mesh model is extracted.

[0037] Integrate the tow truck's own position information into the 3D model, obtain its position in the world coordinate system through the sensor, and convert it to the same coordinate system as the 3D model;

[0038] The location information of obstacles, work areas, and tow trucks are integrated into the same 3D model to form a complete 3D model of the work area. The point cloud data is aligned and optimized through an iterative closest point algorithm to ensure the accuracy and consistency of the model.

[0039] A further improvement of the technical solution of the present invention is that the path planning module specifically includes:

[0040] Receive the three-dimensional positioning information of the tow truck and obstacles provided by the operation positioning module, including the current position coordinates of the tow truck, the spatial distribution data of the obstacles, and the target position coordinates of the operation area. At the same time, load the pre-stored map data, where the map data contains the terrain elevation information of the operation area, the boundary range, and the position and attributes of fixed obstacles, and then initialize the path planning algorithm, setting the starting point to the current position of the tow truck and the end point to the target position of the operation area, providing basic input for path planning;

[0041] Based on the received positioning information and map data, the path generation algorithm is started. A heuristic search algorithm based on the A* algorithm is used to search for a feasible path from the starting point to the end point in the three-dimensional space of the operation area. The cost of the path is evaluated through a heuristic function to generate an optimal three-dimensional path from the starting point to the end point.

[0042] After the global path is generated, the path planning module enters the path adjustment and optimization stage, monitoring the dynamic changes of the working area in real time. When environmental changes are detected, the local path adjustment algorithm is used to dynamically adjust the path, replan the affected part of the path, and then transmit the optimized path to the control system of the tow truck.

[0043] A further improvement of the technical solution of the present invention is that the three-dimensional path generation process based on the A* algorithm is:

[0044] Receive the 3D coordinates of the tow truck's current position and the target position in the work area, as well as the 3D map data of the work area, including terrain elevation and obstacle distribution information. Discretize the 3D space into a grid. Each node n represents a position in the 3D space, including coordinates (x, y, z) and cost information. The current position is the starting point S, and the target position is the end point G.

[0045] The A* algorithm evaluates the cost of a path using the heuristic function f(n) = g(n) + h(n), where g(n) is the actual cost, representing the actual path cost from the starting point S to the current node n, and h(n) is the heuristic estimated cost, representing the estimated cost from the current node n to the end point G, calculated using Euclidean distance.

[0046] During the search process, an open list and a closed list are set. The open list is used to store the nodes to be expanded and sorted by the f(n) value, and the closed list is used to store the expanded nodes. In the initial state, the starting point S is added to the open list;

[0047] Select the node n with the smallest f(n) from the open list for expansion. For node n, generate its adjacent nodes in three-dimensional space and calculate the movement cost c(n, m) from n to the adjacent node m, taking into account the terrain elevation and the influence of obstacles. Update g(m) = g(n) + c(n, m) and calculate f(m) = g(m) + h(m).

[0048] If the adjacent node m is an obstacle or exceeds the map boundary, skip it. If m is not in the open list, add it to the open list. If m is already in the open list and the new g(m) is smaller, update g(m) and f(m), and then move node n to the closed list.

[0049] If the expanded node n is the end point G, the search is successful and the path is backtracked. If the open list is empty and the end point has not been reached, the path does not exist, and then backtrack from the end point G to the starting point S, and reconstruct the path based on the predecessor pointer of each node.

[0050] A further improvement of the technical solution of the present invention is that the obstacle removal operation execution module specifically includes:

[0051] The obstacle removal operation execution module receives the optimal three-dimensional path information transmitted by the path planning module, clarifies the specific route of the obstacle removal vehicle from its current position to the target position in the operation area, and obtains the positioning results of the obstacle target, including the spatial position, size and shape feature data of the obstacle, and converts it into a format that can be recognized by the obstacle removal vehicle actuator;

[0052] Based on the integrated path and obstacle information, combined with the tow truck's operational capability parameters, corresponding operation instructions are generated. The operation instructions include the tow truck's driving speed, steering angle, and the action sequence of the actuators. For each obstacle, a corresponding clearing action instruction is generated based on its type and location.

[0053] The generated operation instructions are sent to the actuators of the tow truck, including the robotic arm, cutting device, hydraulic system, etc. The control system accurately controls each actuator according to the operation instructions and controls the action of the tow truck to clear the obstacles at the appropriate time and location.

[0054] During the towing operation, the driving status of the towing vehicle, the working conditions of the actuators and the environmental changes in the operation area are monitored in real time. Relevant data are collected through sensors, including the actual position and speed of the towing vehicle and the operating parameters of the towing equipment. The real-time data is then compared and analyzed with the operation instructions to determine whether the operation is proceeding as expected. If there is any deviation, the operation instructions are adjusted, the local path is replanned or the operation mode of the towing equipment is adjusted to ensure that the towing operation can be completed smoothly.

[0055] Due to the adoption of the above technical solution, the present invention has the following technical advancements compared to the prior art:

[0056] 1. The present invention provides a self-propelled multifunctional forest tow truck control system combined with visual detection. It uses a high-resolution camera to capture forest environment images and uses image processing and feature extraction technology to accurately identify the type, size and position of obstacles. The three-dimensional model constructed by the operation positioning module further provides accurate position information of the tow truck and obstacles in three-dimensional space, allowing to carry out precise operations for specific targets, effectively improving operation accuracy and reducing the occurrence of incomplete cleaning or misoperation.

[0057] 2. The present invention provides a self-propelled multifunctional forest tow truck control system combined with visual detection. By real-time monitoring of obstacles and operating areas around the tow truck, it can provide early warning and plan the optimal path to avoid obstacles. Combined with map data and real-time environmental information, it can dynamically adjust the tow truck's driving route to ensure that it can reach the operating area safely and efficiently in complex forest environments, significantly enhancing the safety of tow clearance operations and reducing accident risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0059] Figure 1 This is a schematic diagram of the functional modules of the system of the present invention;

[0060] Figure 2 Schematic diagram of the workflow of the present invention. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0062] Example 1, as Figure 1 、 Figure 2As shown, the present invention provides a control system for a self-propelled forest multifunctional tow truck combined with visual detection, including a tow truck control platform, the tow truck control platform is communicatively connected to a visual detection module, an operation positioning module, a path planning module, and an obstacle removal operation execution module, wherein the modules are electrically connected;

[0063] The visual detection module is used to capture and pre-process image data of the forest environment, monitor obstacles and working areas around the tow truck in real time, and capture image data of the forest environment in real time through a high-resolution camera installed on the tow truck, covering various obstacles such as trees, rocks, low vegetation, and the general outline of the working area. The camera continuously collects images of the environment around the tow truck according to the preset sampling frequency and viewing angle range to ensure coverage of obstacles and working areas around the tow truck, and transmits the collected image data in the form of digital signals to the tow truck control platform. The captured image data of the forest environment is pre-processed, including denoising, contrast enhancement and grayscale operations. Among them, the denoising algorithm effectively removes random noise in the image to make the image clearer, and contrast enhancement is used to highlight key features in the image for subsequent recognition. Grayscale is used to convert color images into grayscale images, reducing data volume and improving processing efficiency. The pre-processed image data of the forest environment are integrated and stored in the data warehouse of the tow truck control platform, and classified and sorted by camera number and timestamp;

[0064] The operation positioning module is used to perform feature analysis on the pre-processed image data, build a three-dimensional model of the operation area, mark the positions of the tow truck and each obstacle target in three-dimensional space, and achieve accurate positioning of the tow truck. The operation positioning module includes a feature extraction unit and a three-dimensional modeling and positioning unit;

[0065] Among them, the feature extraction unit is used to extract and identify features of the preprocessed image data, distinguish and mark the type, size and position of obstacles, segment the preprocessed forest environment image data, identify the contours of objects in the image, divide continuous pixel areas into different objects, and use the region growing algorithm to gradually expand the area starting from the seed point based on the pixel similarity criterion, further accurately distinguish the boundary between the obstacle and the surrounding environment, so that the obstacle appears as a clearer independent area in the image, and perform feature extraction on the segmented obstacle image to extract key feature information in the image, including shape features, texture features and color features. Among them, shape features include contour, area and aspect ratio, texture features include grayscale co-occurrence matrix and texture direction, color features include average color value and color distribution, and for shape In terms of features, the outline information of the obstacle is calculated to determine its geometric shape, the number of pixels is counted to determine the area, and the length and width are measured to calculate the aspect ratio. In terms of texture features, the spatial dependence of pixel gray levels is analyzed through the gray-level co-occurrence matrix to obtain the coarseness and density characteristics of the texture. At the same time, the texture direction is analyzed to understand the texture direction of the obstacle surface. The color feature is analyzed by calculating the average color value of the pixels in the area and analyzing the distribution of colors in the area. The extracted key feature information is analyzed. The obstacle classification model pre-built based on the decision tree model is used to classify the obstacles, distinguish different types of obstacles, and identify their types. The classified obstacle information is annotated, including the type, size, and location of the obstacle. The outline of each obstacle is annotated in the image, and its type, size, and coordinate position in the image are recorded.

[0066] In addition, the process of distinguishing different types of obstacles using the obstacle classification model is as follows:

[0067] The extracted shape features, texture features and color features are sorted and normalized to ensure that all feature information formats are unified and the dimensions are consistent. The sorted feature information is input into the pre-built obstacle classification model. According to the input feature information, classification is performed according to the preset decision rules. A large amount of image data of different types of obstacles in forest environments is collected, covering various common obstacles, and the obstacles are labeled. The annotation content includes the type, shape features, texture features and color features of the obstacles, which are integrated into a comprehensive data set. The labeled image data is preprocessed, including denoising, contrast enhancement, grayscale and other operations. Key feature information including shape features, texture features and color features is extracted from the preprocessed image data as input features of the model. The dataset after feature extraction is divided into training set and test set. The training set is combined with the decision tree model to build an obstacle classification model. Starting from the root node, the optimal split feature is selected from all features according to the selected splitting criterion. The dataset is divided into several subsets. The above process is recursively performed on each subset until the stopping condition is met. If the samples in the node all belong to the same category, the number of samples in the node is less than a certain threshold or The decision tree reaches its maximum depth, and a complete decision tree model is constructed through continuous splitting and recursion. The trained decision tree model is evaluated using the test set. The evaluation indicators include accuracy, precision, recall, and F1 value. Accuracy indicates the proportion of correctly classified samples to the total number of samples, precision indicates the proportion of samples predicted to be positive that are actually positive, and recall indicates the proportion of samples actually positive that are correctly predicted to be positive. F1 value is the harmonic mean of precision and recall. By analyzing the evaluation indicators, we can understand the classification performance of the model in different categories. According to the model evaluation results, the model is evaluated. During tuning, if the model overfits, pruning is performed to reduce the complexity of the decision tree and improve the model's generalization ability. The trained obstacle classification model is then deployed to the tow truck's visual detection system. Starting from the root node, the obstacle classification model classifies obstacles based on preset feature thresholds. Obstacles are divided into three categories based on area: large, medium, and small. Within each subcategory, obstacles are further subdivided based on shape features. Finally, the specific type of obstacle is determined based on texture and color features. After step-by-step analysis, the obstacle classification model outputs the final classification result to clearly define the obstacle type.

[0068] The 3D modeling and positioning unit is used to combine the image data of visual detection and the identification and annotation results of obstacles to build a 3D model of the working area, provide a 3D visual representation of the working area, and enable the tow truck to accurately understand its own position and target position. It receives the image data provided by the visual detection module and the obstacle identification and annotation results of the feature extraction unit, and adjusts the viewing angle and resolution of the image data through the calibration algorithm to eliminate the deviation caused by the position and angle differences of the camera. Based on the calibrated image data, the feature matching algorithm is used to find the same feature points in multiple images. Through the matched feature points, combined with the internal and external parameters of the camera, the depth information of the feature points is calculated using the triangulation principle, that is, the distance from the feature point to the camera, and the stereo matching algorithm is used to directly estimate the depth value of each pixel in the image, and gradually construct a 3D depth map of the working area. Based on the 3D depth map, combined with the obstacle The annotation information of the objects is used to construct a three-dimensional model of the working area, wherein the spatial coordinates of each point in the scene are determined according to the depth information, and the three-dimensional reconstruction algorithm is used to connect the various points into surfaces to form the three-dimensional geometric shape of the obstacles and the working area. At the same time, the position information of the tow truck itself is also integrated into the three-dimensional model to clarify the relative position of the tow truck in the working area and construct a complete three-dimensional model of the working area. The constructed three-dimensional model of the working area is visualized and rendered to display the overall picture of the working area with an intuitive three-dimensional graphical interface, including the distribution, shape, size of the obstacles and the position information of the tow truck. By updating the three-dimensional model in real time, the operator can understand the dynamic changes of the working area at any time. At the same time, the position information of the tow truck in the three-dimensional model of the working area is fed back to the control system of the tow truck, providing accurate spatial reference for the navigation, obstacle avoidance and working path planning of the tow truck.

[0069] In addition, the construction process of the 3D model of the work area is as follows:

[0070] Extract the depth value d(x, y) of each pixel from the 3D depth map, where (x, y) is the coordinate of the pixel in the image. Use the depth map to provide distance information from the camera to each point in the scene. Convert the pixel points in the depth map from the image coordinate system to the camera coordinate system. The intrinsic parameter matrix of the camera is K, which is in the form of: Among them, f x and f y is the focal length of the camera, (c x , c y ) is the optical center coordinate. For each pixel in the depth map, its three-dimensional coordinate (X, y) in the camera coordinate system is c , Y c , Z c ) is calculated using the following formula: The obstacle annotation information provided by the feature extraction unit is combined with the depth information, where the obstacle annotation information includes the type, location and range of the obstacle. For each obstacle, the corresponding depth value is extracted from the depth map according to its annotated outline, and the vertex coordinates of the obstacle outline are calculated. The point cloud data of the obstacle and the working area are connected into a surface using the Poisson reconstruction 3D reconstruction algorithm to form a complete 3D geometric shape. The point cloud data is {p i}, the normal vector of each point is n i , then the goal of Poisson reconstruction is to solve a scalar function f(x) such that: By solving the Poisson equation, an implicit surface is obtained, and then a three-dimensional mesh model is extracted. The position information of the tow truck itself is integrated into the three-dimensional model, where the position of the tow truck is (X v , Y v , Z v ), obtain its position in the world coordinate system through the sensor and convert it to the same coordinate system as the 3D model. The position information of the obstacle, working area and tow truck are integrated into the same 3D model to form a complete 3D model of the working area. The point cloud data is aligned and optimized through the iterative closest point algorithm to ensure the accuracy and consistency of the model. The 3D model of the complete working area is represented as follows: R is the obstacle outline

[0071] The path planning module is used to plan the optimal operating path for the tow truck based on the positioning information provided by the operation positioning module, combined with map data and real-time environment, and adjust the path in real time to avoid obstacles, ensuring that the tow truck can reach the operation area efficiently and safely and operate according to the optimal path, thereby improving operation efficiency and safety;

[0072] The obstacle removal operation execution module is used to combine path planning with the positioning results of obstacle targets to generate corresponding operation instructions and control the actuators of the obstacle removal vehicle to perform precise operations, thereby realizing the automation and intelligent operation of the obstacle removal vehicle, improving operation accuracy and efficiency, and reducing manual intervention and labor intensity.

[0073] Example 2, as Figure 1 、 Figure 2 As shown, based on Example 1, the present invention provides a technical solution: preferably, the path planning module specifically includes:

[0074] The three-dimensional positioning information of the tow truck and obstacles provided by the operation positioning module is received, including the current position coordinates of the tow truck, the spatial distribution data of the obstacles and the target position coordinates of the operation area. At the same time, the pre-stored map data is loaded, wherein the map data contains the terrain elevation information, boundary range and the position and attributes of the fixed obstacles in the operation area, and then the path planning algorithm is initialized, and the starting point is set to the current position of the tow truck and the end point is set to the target position of the operation area, providing basic input for path planning. Based on the received positioning information and map data, the path generation algorithm is started, and a heuristic search algorithm based on the A* algorithm is used to search for a feasible path from the starting point to the end point in the three-dimensional space of the operation area. Through the heuristic function The cost of the path is evaluated numerically. Starting from the starting point, the search nodes are gradually expanded, the cost of each node is calculated, and the node with the smallest cost is selected as the next expansion direction until the end point is reached. An optimal three-dimensional path from the starting point to the end point is generated. After the global path is generated, the path planning module enters the path adjustment and optimization stage, monitoring the dynamic changes of the operating area in real time. When environmental changes are detected, the local path adjustment algorithm is used to dynamically adjust the path and replan the affected part of the path. At the same time, the smoothness and safety of the path are optimized to ensure that the tow truck can travel smoothly and efficiently when performing operations, avoiding unnecessary sharp turns or frequent obstacle avoidance, and then transmitting the optimized path to the tow truck's control system;

[0075] In addition, the three-dimensional path generation process based on the A* algorithm is:

[0076] Receive the three-dimensional coordinates of the current position of the tow truck and the target position of the working area, as well as the three-dimensional map data of the working area, including terrain elevation and obstacle distribution information, and discretize the three-dimensional space into a grid. Each node n represents a position in the three-dimensional space, including coordinates (x, y, z) and cost information. Among them, the current position is the starting point S and the target position is the end point G. The A* algorithm evaluates the cost of the path through the heuristic function f(n) = g(n) + h(n), where g(n) is the actual cost, which represents the actual path cost from the starting point S to the current node n, is the path length, and h(n) is the heuristic estimated cost, which represents the estimated cost from the current node n to the end point G. It is calculated using Euclidean distance. (x G ,y G , z G ) is the end point coordinate, (x n ,y n , z n) is the coordinate of the current node. During the search process, an open list and a closed list are set. The open list is used to store the nodes to be expanded and sorted by the f(n) value. The closed list is used to store the expanded nodes to avoid repeated calculations. In the initial state, the starting point S is added to the open list, g(S) = 0, f(S) = h(S), and the node n with the smallest f(n) is selected from the open list for expansion. For node n, its adjacent nodes in three-dimensional space are generated, and the movement cost c(n, m) from n to the adjacent node m is calculated in combination with the terrain elevation and the influence of obstacles. Update g(m)=g(n)+c(n,m), calculate f(m)=g(m)+h(m), if the adjacent node m is an obstacle or exceeds the map boundary, skip it, if m is not in the open list, add it to the open list, if m is already in the open list and the new g(m) is smaller, update g(m) and f(m), and then move node n to the closed list, if the expanded node n is the end point G, the search is successful, backtrack the path, if the open list is empty and the end point is not reached, then the path does not exist, and then backtrack from the end point G to the starting point S, and reconstruct the path based on the predecessor pointer of each node;

[0077] The obstacle removal operation execution module specifically includes:

[0078] The tow truck operation execution module receives the information of the optimal three-dimensional path transmitted by the path planning module, clarifies the specific route of the tow truck from the current position to the target position in the operation area, and obtains the positioning result of the obstacle target, including the spatial position, size and shape feature data of the obstacle, and converts it into a format that can be recognized by the tow truck actuator. Based on the integrated path and obstacle information, combined with the tow truck's operating capability parameters, the module generates corresponding operation instructions, where the operation instructions include the tow truck's driving speed, steering angle, and the action sequence of the actuator. For each obstacle, according to its type and position, the module generates corresponding cleaning action instructions, including cutting, grabbing or pushing, etc. At the same time, the module ensures that the operation instructions meet the mechanical performance and safety requirements of the tow truck, provides detailed operation guidelines for the precise control of the actuator, and sends the generated operation instructions to the actuator of the tow truck, including the mechanical arm, Cutting device, hydraulic system, etc. The control system precisely controls each actuator according to the operation instructions, adjusts the engine speed and power output, controls the transmission ratio of the transmission system to achieve speed regulation, and precisely controls the steering angle and steering speed of the steering system to ensure that the tow truck travels along the planned path. At the same time, it controls the action of the tow truck so that it can clear obstacles at the appropriate time and position. During the execution of the tow truck operation, it monitors the driving status of the tow truck, the working conditions of the actuators and the environmental changes in the operation area in real time. Relevant data is collected through sensors, including the actual position and speed of the tow truck and the operating parameters of the tow truck equipment. The real-time data is then compared and analyzed with the operation instructions to determine whether the operation is carried out as expected. If there is a deviation, the operation instructions are adjusted, the local path is replanned or the operation mode of the tow truck equipment is adjusted to ensure that the tow truck operation can be completed smoothly.

[0079] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A self-propelled multifunctional forest tow truck control system combined with visual detection, including a tow truck control platform, characterized by: The control platform of the tow truck is communicatively connected to a visual detection module, an operation positioning module, a path planning module and an obstacle removal operation execution module, wherein electrical signals are connected between the modules; The visual detection module is used to capture and pre-process image data of the forest environment and monitor obstacles and operating areas around the tow truck in real time; The operation positioning module is used to perform feature analysis on the pre-processed image data, build a three-dimensional model of the operation area, and mark the positions of the tow truck and each obstacle target in the three-dimensional space; The path planning module is used to plan the optimal operation path for the tow truck based on the positioning information provided by the operation positioning module, combined with map data and real-time environment; The obstacle removal operation execution module is used to combine the path planning and the positioning results of the obstacle target to generate corresponding operation instructions and control the actuator of the obstacle removal vehicle to perform precise operations.

2. The control system of a self-propelled multifunctional forest tow truck combined with visual detection according to claim 1, characterized in that: The visual detection module specifically includes: The high-resolution camera installed on the tow truck captures the image data of the forest environment in real time. The camera continuously collects images of the environment around the tow truck according to the preset sampling frequency and viewing angle range, and transmits the collected image data to the tow truck control platform in the form of digital signals; Preprocessing the captured forest environment image data, including denoising, contrast enhancement and grayscale operations; The pre-processed image data of the forest environment are integrated and stored in the data warehouse of the tow truck control platform, and are classified and sorted by camera number and timestamp.

3. The control system of a self-propelled multifunctional forest tow truck combined with visual detection according to claim 1, characterized in that: The operation positioning module includes a feature extraction unit and a three-dimensional modeling positioning unit; The feature extraction unit is used to extract and identify features from the pre-processed image data, distinguish and mark the type, size and position of obstacles; The three-dimensional modeling and positioning unit is used to combine the image data of visual detection and the identification and marking results of obstacles to construct a three-dimensional model of the working area and provide a three-dimensional visual representation of the working area.

4. The control system of a self-propelled multifunctional forest tow truck combined with visual detection according to claim 3 is characterized by: The feature extraction unit specifically includes: The pre-processed forest environment image data is segmented to identify the outlines of objects in the image, dividing continuous pixel areas into different objects. The region growing algorithm is then used to gradually expand the area from the seed point based on the pixel similarity criterion to further accurately distinguish the boundaries between obstacles and the surrounding environment. Perform feature extraction on the segmented obstacle image to extract key feature information in the image, including shape features, texture features, and color features. Shape features include contour, area, and aspect ratio; texture features include gray-level co-occurrence matrix and texture direction; and color features include average color value and color distribution. Analyze the extracted key feature information and use the obstacle classification model pre-built based on the decision tree model to classify obstacles, distinguish different types of obstacles, and identify their types; The classified obstacle information is labeled, including the type, size and location of the obstacle, and the outline of each obstacle is marked in the image, recording its type, size and coordinate position in the image.

5. The control system of a self-propelled multifunctional forest tow truck combined with visual detection according to claim 4, characterized in that: The process of using the obstacle classification model to distinguish different types of obstacles is as follows: The extracted shape features, texture features, and color features are sorted and normalized, and the sorted feature information is input into the pre-built obstacle classification model. Based on the input feature information, classification is performed according to the preset decision rules; The obstacle classification model starts from the root node and classifies obstacles based on preset feature thresholds. Obstacles are divided into three categories based on area: large, medium, and small. Within each subcategory, obstacles are further subdivided based on shape features. Finally, the specific type of obstacle is determined based on texture and color features. After the obstacle classification model has been analyzed step by step, the final classification result of the obstacle is output to clarify the type of obstacle.

6. The control system of a self-propelled multifunctional forest tow truck combined with visual detection according to claim 3, characterized in that: The three-dimensional modeling and positioning unit specifically includes: Receive the image data provided by the visual detection module and the obstacle differentiation and annotation results of the feature extraction unit, and adjust the viewing angle and resolution of the image data through the calibration algorithm; Based on the calibrated image data, a feature matching algorithm is used to find the same feature points in multiple images. By matching the feature points and combining the camera's intrinsic and extrinsic parameters, the depth information of the feature points (i.e., the distance from the feature points to the camera) is calculated using the triangulation principle. A stereo matching algorithm is then used to directly estimate the depth value of each pixel in the image, gradually constructing a three-dimensional depth map of the work area. Based on the 3D depth map and combined with the annotation information of obstacles, a 3D model of the work area is constructed. The spatial coordinates of each point in the scene are determined based on the depth information. A 3D reconstruction algorithm is used to connect the points into surfaces, forming the 3D geometric shape of the obstacles and the work area. At the same time, the position information of the tow truck itself is also integrated into the 3D model to clarify the relative position of the tow truck in the work area. The constructed three-dimensional model of the work area is visually rendered, and the overall picture of the work area is displayed in a three-dimensional graphical interface, including the distribution, shape, size of obstacles and the position information of the tow truck. At the same time, the position information of the tow truck in the three-dimensional model of the work area is fed back to the control system of the tow truck.

7. The control system of a self-propelled multifunctional forest tow truck combined with visual detection according to claim 6, characterized in that: The process of constructing the three-dimensional model of the working area is as follows: Extract the depth value of each pixel from the 3D depth map, and use the depth map to provide distance information from the camera to each point in the scene; Convert the pixel points in the depth map from the image coordinate system to the camera coordinate system, and for each pixel in the depth map in the image, calculate its three-dimensional coordinates in the camera coordinate system; Combine the obstacle annotation information provided by the feature extraction unit with the depth information, where the obstacle annotation information includes the type, location, and range of the obstacle. For each obstacle, extract the corresponding depth value in the depth map based on its annotated outline and calculate the vertex coordinates of the obstacle outline; The Poisson reconstruction algorithm is used to connect the point cloud data of obstacles and the working area into a surface to form a complete 3D geometric shape. By solving the Poisson equation, an implicit surface is obtained, and then a 3D mesh model is extracted. Integrate the tow truck's own position information into the 3D model, obtain its position in the world coordinate system through the sensor, and convert it to the same coordinate system as the 3D model; The location information of obstacles, work areas, and tow trucks are integrated into the same 3D model to form a complete 3D model of the work area, and the point cloud data is aligned and optimized through an iterative closest point algorithm.

8. The control system of a self-propelled multifunctional forest tow truck combined with visual detection according to claim 3, characterized in that: The path planning module specifically includes: Receive the three-dimensional positioning information of the tow truck and obstacles provided by the operation positioning module, including the current position coordinates of the tow truck, the spatial distribution data of the obstacles, and the target position coordinates of the operation area. At the same time, load the pre-stored map data, where the map data contains the terrain elevation information of the operation area, the boundary range, and the position and attributes of fixed obstacles, and then initialize the path planning algorithm, setting the starting point to the current position of the tow truck and the end point to the target position of the operation area; Based on the received positioning information and map data, the path generation algorithm is started. A heuristic search algorithm based on the A* algorithm is used to search for a feasible path from the starting point to the end point in the three-dimensional space of the operation area. The cost of the path is evaluated through a heuristic function to generate an optimal three-dimensional path from the starting point to the end point. After the global path is generated, the path planning module enters the path adjustment and optimization stage, monitoring the dynamic changes of the working area in real time. When environmental changes are detected, the local path adjustment algorithm is used to dynamically adjust the path, replan the affected part of the path, and then transmit the optimized path to the control system of the tow truck.

9. The control system of a self-propelled multifunctional forest tow truck combined with visual detection according to claim 8, characterized in that: The three-dimensional path generation process based on the A* algorithm is as follows: Receive the 3D coordinates of the tow truck's current position and the target position in the work area, as well as the 3D map data of the work area, including terrain elevation and obstacle distribution information. Discretize the 3D space into a grid. Each node n represents a position in the 3D space, including coordinates (x, y, z) and cost information. The current position is the starting point S, and the target position is the end point G. The A* algorithm evaluates the cost of a path using the heuristic function f(n) = g(n) + h(n), where g(n) is the actual cost, representing the actual path cost from the starting point S to the current node n, and h(n) is the heuristic estimated cost, representing the estimated cost from the current node n to the end point G, calculated using Euclidean distance. During the search process, an open list and a closed list are set. The open list is used to store the nodes to be expanded and sorted by the f(n) value. The closed list is used to store the expanded nodes to avoid repeated calculations. In the initial state, the starting point S is added to the open list, g(S) = 0, f(S) = h(S); Select the node n with the smallest f(n) from the open list for expansion. For node n, generate its adjacent nodes in three-dimensional space and calculate the movement cost c(n, m) from n to the adjacent node m, taking into account the terrain elevation and the influence of obstacles. Update g(m) = g(n) + c(n, m) and calculate f(m) = g(m) + h(m). If the adjacent node m is an obstacle or exceeds the map boundary, skip it. If m is not in the open list, add it to the open list. If m is already in the open list and the new g(m) is smaller, update g(m) and f(m), and then move node n to the closed list. If the expanded node n is the end point G, the search is successful and the path is backtracked. If the open list is empty and the end point has not been reached, the path does not exist, and then backtrack from the end point G to the starting point S, and reconstruct the path based on the predecessor pointer of each node.

10. The control system of a self-propelled multifunctional forest tow truck combined with visual detection according to claim 8, characterized in that: The obstacle removal operation execution module specifically includes: The obstacle removal operation execution module receives the optimal three-dimensional path information transmitted by the path planning module, clarifies the specific route of the obstacle removal vehicle from its current position to the target position in the operation area, and obtains the positioning results of the obstacle target, including the spatial position, size and shape feature data of the obstacle; Based on the integrated path and obstacle information, combined with the tow truck's operational capability parameters, corresponding operation instructions are generated. The operation instructions include the tow truck's driving speed, steering angle, and the action sequence of the actuators. For each obstacle, a corresponding clearing action instruction is generated based on its type and location. The generated operation instructions are sent to the actuators of the tow truck. The control system accurately controls each actuator according to the operation instructions and controls the action of the tow truck at the same time. During the towing operation, the driving status of the towing vehicle, the working conditions of the actuators and the environmental changes in the operation area are monitored in real time. Relevant data are collected through sensors, including the actual position and speed of the towing vehicle and the operating parameters of the towing equipment. The real-time data is then compared and analyzed with the operation instructions to determine whether the operation is proceeding as expected. If deviations occur, the operation instructions are adjusted, the local path is replanned or the operation mode of the towing equipment is adjusted.

Images