A hybrid terrain trajectory planning system, travel control system

Abstract

The application provides a mixed terrain trajectory planning system for planning a travel path for a travel target in a mixed terrain environment, the system comprising: a perception processing module configured with a pre-trained semantic segmentation network and an encoder; wherein the pre-trained semantic segmentation network is used for performing semantic feature extraction on a plurality of continuous RGB terrain images of the travel target collected at different moments to obtain a multi-label terrain feature map at each moment; the encoder is used for performing encoding processing on the multi-label terrain feature map at each moment to obtain a mixed terrain semantic feature map at each moment; and a trajectory planning module configured with a policy network, and used for processing a plurality of continuous travel states of the travel target, a terrain depth map and a plurality of continuous mixed terrain semantic feature maps corresponding thereto transmitted by the perception processing module to obtain a target planning trajectory.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of motion planning, in particular to trajectory planning technology in the field of motion planning, and more particularly to a mixed terrain trajectory planning system and a travel control system. BACKGROUND

[0002] In a navigation task in a wild environment, trajectory planning is a crucial link, which provides an efficient and safe execution path for tasks such as forest monitoring, mining, plateau transportation, search and rescue, etc. Due to the particularity of the wild environment, trajectory planning needs to consider not only the geometric characteristics of the terrain, but also rich semantic information; at the same time, due to the fact that the parameter space for trajectory planning in the wild environment is no longer the SE(2) space similar to the urban space, but needs to consider the SE(3) space with height variation, therefore, in the wild environment, the trajectory planning needs to obtain the wild environment information and process it to obtain the terrain information, and then plan a reasonable motion trajectory according to the obtained terrain information.

[0003] With the rapid development of computer vision technology, when performing a trajectory planning task, computer vision technology can be used to obtain terrain information in a wild environment, and the specific execution process is as follows: obtaining an RGB image collected in a wild environment, and assigning a single terrain class label to each pixel in the RGB image, such as assigning a terrain class label of grassland, soil, gravel, etc. In addition to this way of obtaining terrain information, terrain information can also be extracted by constructing a passable map, and the specific execution process is as follows: using a classification network or a passable standard defined by a human to determine whether a certain terrain in the image can be passed, and marking the passable area with a passable label and the impassable area with an impassable label to obtain a two-dimensional grid map.

[0004] After obtaining the terrain information, the current mainstream processing method is to perform trajectory planning by using a traditional planning method or a reinforcement learning method. For the traditional planning method, it usually uses RRT* or A* method based on the passable information represented in the two-dimensional grid map to perform trajectory planning, which simply searches for a feasible next position in the passable area to find an obstacle-free trajectory represented in the two-dimensional grid map, and adds linear constraints in the planning process to make the planned trajectory more reasonable and efficient. For the reinforcement learning method, it obtains terrain information through real-time collection of RGB images of a moving target in the real world, and uses the obtained terrain information as input and the planned trajectory as output to train the policy network online, and transmits the planned trajectory output by the policy network to the moving target for execution, and continuously optimizes the policy network according to the execution feedback of the moving target.

[0005] Although the prior art can plan a trajectory by acquiring terrain information, the planned trajectory obtained by the prior art has poor quality, which is difficult to control the motion target to move along the planned trajectory, and may also make the motion target face potential danger during the motion.

[0006] It should be noted that the background art is only used to introduce the related information of the present application, so as to help understand the technical solutions of the present application, but does not mean that the related information must be the prior art. In the absence of evidence that the related information has been disclosed before the filing date of the present application, the related information should not be considered as prior art. SUMMARY

[0007] Therefore, the purpose of the present application is to overcome the defects of the prior art, and to provide a mixed terrain trajectory planning system and a moving target moving control system.

[0008] The purpose of the present application is achieved by the following technical solutions.

[0009] According to a first aspect of the present application, a mixed terrain trajectory planning system is provided for planning a moving path for a moving target in a mixed terrain environment, the system comprising: a perception processing module configured with a pre-trained semantic segmentation network and an encoder; wherein the pre-trained semantic segmentation network is used to perform semantic feature extraction on a plurality of continuous time instant RGB terrain images collected by the moving target to obtain a multi-label terrain feature map of each time instant; the encoder is used to perform encoding processing on the multi-label terrain feature map of each time instant to obtain a mixed terrain semantic feature map of each time instant; wherein the multi-label terrain feature map represents the terrain class contained in each pixel of the RGB terrain image, and the mixed terrain semantic feature map represents the difficulty of passing through different terrain classes in the RGB terrain image; a trajectory planning module for processing the moving state of the moving target collected by the moving target at a plurality of continuous time instants, the terrain depth map and the plurality of continuous time instant mixed terrain semantic feature maps corresponding thereto transmitted by the perception processing module based on the policy network configured thereon to obtain a target planning trajectory; wherein the policy network takes the plurality of continuous time instant mixed terrain semantic feature maps, the terrain depth map and the moving state as input, and takes the target planning trajectory as output, and is obtained based on meta-reinforcement learning offline training.

[0010] In some embodiments of the present application, the pre-trained semantic segmentation network is configured to perform terrain classification on each pixel in the RGB terrain image of each time instant to obtain the probability of each pixel in the RGB terrain image of each time instant belonging to each terrain class, and the terrain class corresponding to all terrain class probabilities greater than a preset classification threshold in each pixel is taken as the terrain class label of the pixel to obtain the multi-label terrain feature map.

[0011] In some embodiments of the present application, the encoder is configured to encode the multi-label terrain feature map to obtain the hybrid terrain semantic feature map in the following manner: based on preset terrain category passing difficulty, calculating the sum mean of the passing difficulty of the terrain corresponding to all terrain category labels corresponding to each pixel in the multi-label terrain feature map.

[0012] In some embodiments of the present application, the policy network comprises a first pre-network and a second pre-network connected in parallel, and a plurality of fully connected layers, a long short-term memory network and an output layer connected in series after the first pre-network and the second pre-network; wherein the first pre-network is composed of a plurality of fully connected layers, for extracting state feature information of a plurality of continuous travel states at a plurality of time points; the second pre-network is composed of a plurality of convolutional layers, for extracting terrain semantic information of a plurality of continuous hybrid terrain semantic feature maps at a plurality of time points and terrain geometric information of a plurality of continuous terrain depth maps at a plurality of time points; the fully connected layers are used for fusion processing of the plurality of continuous state feature information transmitted by the first pre-network and the plurality of time point terrain semantic information and terrain geometric information transmitted by the second pre-network to obtain sequence fusion features; the long short-term memory network is used for extracting the dependency relationship in the sequence fusion features to obtain time sequence features; and the output layer is used for trajectory prediction based on the time sequence features to obtain the target planning trajectory.

[0013] In some embodiments of the present application, the policy network is configured to be trained in the following manner: based on existing path data, a training set is constructed, wherein the training set comprises a plurality of sequence data, each sequence data has a corresponding standard planning trajectory, and each sequence data comprises a plurality of continuous RGB terrain images at a plurality of time points, corresponding terrain depth maps and travel states; using a perception processing module to process the plurality of continuous RGB terrain images in each sequence data to obtain a hybrid terrain semantic feature map at each time point in the sequence data, using the plurality of processed sequence data as the input of the policy network and the target planning trajectory as the output to perform multiple rounds of training, and updating the parameters of the policy network based on a preset training target.

[0014] Preferably, the preset training target is:

[0015]

[0016] s.t

[0017] wherein, represents the policy network; represents the parameters of the policy network; represents the time length; represents the time point; represents the discount factor; represents the reward function; representing sequence data; representing sequence data corresponding standard planning trajectory; representing total reward obtained by executing the standard planning trajectory; representing sequence data obtained target planning trajectory; representing total reward obtained by executing the target planning trajectory; representing between and divergence, representing the policy network before updating, representing the policy network after being updated for multiple times based on ; representing hyperparameters.

[0018] In some embodiments of the present application, the system further comprises a self-correction module configured to collect all actual motion trajectories whose Euclidean distances from corresponding target planning trajectories are greater than or equal to a preset distance threshold as a correction trajectory set after the target planning trajectories are executed, and pass the correction trajectory set to the trajectory planning module so that the policy network configured on the trajectory planning module is retrained based on the correction trajectory set, wherein the Euclidean distance is used to measure the similarity between the actual motion trajectory and the corresponding target planning trajectory.

[0019] In some embodiments of the present application, the policy network is configured to be retrained in the following manner: taking the RGB terrain images, terrain depth maps and motion states at multiple time points corresponding to the target planning trajectory corresponding to each actual motion trajectory in the correction trajectory set as retraining sequence data, and taking the actual motion trajectory as the standard motion trajectory corresponding to the retraining sequence data, all the retraining sequence data and the corresponding standard motion trajectories constituting a retraining data set; processing the RGB terrain images at multiple time points in each retraining sequence data in the retraining data set by using the perception processing module to obtain mixed terrain semantic feature maps at each time point, taking the processed multiple retraining sequence data as input and the new target planning trajectory as output to perform multiple rounds of training, and updating the parameters of the policy network based on the new training target.

[0020] Preferably, the new training target is:

[0021]

[0022] wherein, representing the retrained policy network, representing the retraining sequence data, representing the retraining sequence data a corresponding standard motion trajectory, a reward obtained by executing the standard motion trajectory, a retraining sequence data based on which the policy network is trained, a new target planning trajectory obtained, a reward obtained by executing the new target planning trajectory, a Euclidean distance between the standard motion trajectory and the new target planning trajectory.

[0023] According to a second aspect of the present application, there is provided a traveling target traveling control system, comprising: a mixed terrain trajectory planning system as described in the first aspect of the present application, for planning a traveling path for a traveling target in a mixed terrain environment to obtain a target planning trajectory; a motion control module, for calculating an angular velocity and a linear velocity of the traveling target when executing the target planning trajectory based on the target planning trajectory; and a PID control module, for controlling the traveling target to move according to the target planning trajectory based on the angular velocity and the linear velocity obtained by the motion control module.

[0024] Compared with the prior art, the present application has the following advantages: (1) the RGB image is classified by a multi-label terrain classification method to obtain more comprehensive terrain information, thereby avoiding the problem of key information loss caused by insufficient terrain information extraction; (2) the policy network is trained by an offline meta-reinforcement learning method, which avoids the dangerous planning when interacting with the real world during online training, and the meta-reinforcement learning can establish an optimal planning strategy for different terrain types, and by regarding the planning tasks on different terrains as different meta-tasks, a policy network that can adapt to complex and variable terrain environments is trained; (3) a self-correction module is set to realize self-correction and optimization of the policy network, so that the policy network maintains good adaptability and stability in complex and variable terrain environments. BRIEF DESCRIPTION OF DRAWINGS

[0025] The embodiments of the present application will be further described below with reference to the accompanying drawings, in which:

[0026] Figure 1 FIG. 1 is a structural composition schematic diagram of a mixed terrain trajectory planning system according to an embodiment of the present application;

[0027] Figure 2 FIG. 2 is a mixed terrain semantic feature example schematic diagram according to an embodiment of the present application;

[0028] Figure 3 FIG. 3 is a policy network structural composition schematic diagram according to an embodiment of the present application;

[0029] Figure 4 FIG. 4 is a structural composition schematic diagram of a traveling target traveling control system according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0031] As mentioned in the background section, although the prior art can perform trajectory planning by obtaining terrain information, the planning trajectory obtained by the prior art has poor quality, which is difficult to control the motion target to move along the planning trajectory, and may also make the motion target face potential danger during the motion.

[0032] In order to solve the above-mentioned defects, the inventors analyze the prior art and find that the existing terrain information extraction and trajectory planning both have defects. The defects in the terrain information extraction and trajectory planning process are analyzed one by one as follows.

[0033] In the terrain information extraction process, the prior art mainly relies on two strategies: one is terrain category labeling strategy, and the other is passability labeling strategy. Among them, the terrain category labeling strategy is to assign a single terrain category label to each pixel in the RGB image, so as to identify and extract terrain information. Although this processing method can identify the basic type of terrain, it assumes that each pixel in the RGB image represents only one independent terrain category; however, in complex and variable natural environments, mixed terrain is extremely common, such as interlaced muddy ground and grassland, grassland mixed in gravel ground, and sandstone ground mixed with muddy ground. In these mixed terrain environments, simply classifying each pixel as a single terrain obviously cannot fully and accurately reflect the true situation of the terrain, resulting in biased or insufficient extracted terrain information, and further resulting in poor quality of planning trajectory. The passability labeling strategy uses a classification network or a human-defined passability standard to determine whether a certain terrain in the image is passable, so as to identify and extract terrain information. Although this processing method can identify passable areas and impassable areas, this processing method generates a two-dimensional grid map. However, in the complex and variable environment in the wild, the terrain fluctuates, and a large amount of key information is often lost in the two-dimensional grid map. Therefore, when relying only on the passability labeling on the two-dimensional grid map to plan a trajectory, it is difficult to ensure that a safe and reliable route is planned. As can be seen, the existing terrain information extraction strategies cannot fully and accurately reflect the true situation of the terrain, and the extracted terrain information all has deviations or deficiencies.

[0034] In the trajectory planning process, the existing technology mainly relies on two planning strategies: one is a traditional planning strategy, and the other is a planning strategy based on reinforcement learning. Among them, the traditional planning strategy usually relies on search or sampling methods combined with a two-dimensional grid map to plan a collision-free and destination-reaching trajectory. However, a significant limitation of this traditional planning strategy is that it cannot distinguish between different types of terrain such as grass, mud, and gravel, and only relies on the passable and impassable annotations on the two-dimensional grid map for planning, which results in suboptimal trajectory quality. In order to improve the safety and reliability of the planned trajectory, the traditional planning strategy also introduces linear constraints such as maximum speed, obstacle distance, etc. to adjust the planned trajectory. However, in natural environments, there are a large number of nonlinear constraints, such as complex and variable terrain constraints, nonholonomic motion constraints, etc. These nonlinear constraints have an important impact on the feasibility and safety of the trajectory. Therefore, the traditional planning strategy is limited to considering linear constraints, while ignoring or failing to effectively handle these nonlinear constraints. Therefore, the planned trajectory may not fully adapt to the complexity and variability of the natural environment, thereby affecting the actual travel effect and safety of the moving target. The planning strategy based on reinforcement learning trains the policy network online with terrain-labeled images, and uses the planning trajectory output by the policy network to control the motion of the moving target, and then optimizes the policy network according to the execution feedback of the moving target. Although the policy network can learn the mapping of terrain information to trajectory and handle various nonlinear constraints to some extent, and generate a more reasonable planning trajectory. However, due to the rich variety of terrains in the wild environment, and the optimal trajectory strategy for each terrain is different, which makes it difficult for the policy network to achieve optimal performance in all terrains during training. Moreover, online training requires interaction with the real world to obtain a large amount of data, resulting in difficulty and long training time. Furthermore, the continuous optimization of the policy network through online training means that the moving target needs to interact with the real environment in real time during training. However, before the policy network reaches a stable and reliable use requirement, this real-time interaction may cause dangerous planning, thereby affecting the actual operation effect and safety of the moving target. Therefore, the existing trajectory planning strategies cannot cope with complex and variable terrain environments, and it is difficult to plan a safe and reliable trajectory.

[0035] Based on the analysis of the prior art, in view of the defects existing in the existing terrain information extraction strategy, the inventor finds that more comprehensive terrain information can be obtained by multi-label terrain classification of the RGB image, so as to avoid the problem of key information loss caused by insufficient terrain information extraction. In view of the defects existing in the existing trajectory planning strategy, the inventor finds that the strategy network can be trained by using the offline meta-reinforcement learning method, wherein the offline training method can avoid the dangerous planning when interacting with the real world during online training, and the meta-reinforcement learning can establish the optimal planning strategy for different terrain types, and by regarding the planning tasks on different terrains as different meta-tasks, a strategy network capable of coping with complex and variable terrain environment is trained. Further, the inventor proposes a new trajectory planning system for complex outdoor environment, wherein a perception processing module capable of multi-label terrain classification processing is arranged, and a trajectory planning module configured with a strategy network obtained by offline training based on the meta-reinforcement learning method is arranged.

[0036] In summary, as shown in Figure 1 The present application provides a mixed terrain trajectory planning system for planning a travel path for a travel target in a mixed terrain environment, which comprises: a perception processing module configured with a pre-trained semantic segmentation network and an encoder; wherein the pre-trained semantic segmentation network is used for performing semantic feature extraction on a plurality of continuous RGB terrain images of the travel target collected at different time points to obtain a multi-label terrain feature map at each time point; the encoder is used for performing encoding processing on the multi-label terrain feature map at each time point to obtain a mixed terrain semantic feature map at each time point; wherein the multi-label terrain feature map represents the terrain category contained in each pixel of the RGB terrain image, and the mixed terrain semantic feature map represents the difficulty of passing through different terrain categories in the RGB terrain image; a trajectory planning module for processing the travel state of the travel target collected at different time points, the terrain depth map and the mixed terrain semantic feature map corresponding thereto transmitted by the perception processing module based on the strategy network arranged thereon to obtain a target planning trajectory; wherein the strategy network takes the plurality of continuous mixed terrain semantic feature maps, the terrain depth map and the travel state as input, and takes the target planning trajectory as output, and is obtained by offline training based on the meta-reinforcement learning method.

[0037] In order to better understand the present application, the components of the trajectory planning system will be described in detail below in combination with specific embodiments.

[0038] I. Perception processing module

[0039] The perception processing module is configured to preprocess the RGB image collected by the traveling target to extract terrain information in the RGB image. The perception processing module is configured with a pre-trained semantic segmentation network and an encoder. The pre-trained semantic segmentation network and the encoder are described below, respectively.

[0040] The pre-trained semantic segmentation network is configured to extract semantic features from a plurality of continuous RGB terrain images collected by the traveling target at different time points to obtain a multi-label terrain feature map at each time point. According to an embodiment of the present application, the pre-trained semantic segmentation network is configured to classify the terrain of each pixel in the RGB terrain image at each time point to obtain the probability that each pixel in the RGB terrain image at each time point belongs to each terrain category, and to take the terrain category corresponding to all terrain category probabilities greater than a preset classification threshold in each pixel as the terrain category label of the pixel to obtain the multi-label terrain feature map. In addition to taking the terrain category corresponding to all terrain category probabilities greater than a preset classification threshold in each pixel as the terrain category label of the pixel, the terrain category probability corresponding to each pixel can also be sorted from large to small, and the top three terrain categories greater than the preset classification threshold according to the sorting result are selected as the terrain category label of the pixel. For example, when the terrain classification result corresponding to a certain pixel is: gravel 0.7, grassland 0.8, mud land 0.7, cement land 0.2, asphalt land 0.1, and the preset classification threshold is 0.6, gravel, grassland, and mud land can be selected as the terrain category label of the pixel. It should be noted that the original single terrain label is transformed into a multi-terrain label, thereby distinguishing the case where the grassland on the gravel or the grassland on the mud land is classified as grassland by the prior art, and more accurate terrain information can be obtained to solve the problem of insufficient terrain information extraction in a mixed terrain environment.

[0041] The encoder is configured to encode the multi-label terrain feature map at each time point to obtain a mixed terrain semantic feature map at each time point. According to an embodiment of the present application, the encoder is configured to encode the multi-label terrain feature map to obtain the mixed terrain semantic feature map in the following manner: based on a preset terrain category difficulty, the sum of the difficulty of the corresponding terrain of all terrain category labels corresponding to each pixel in the multi-label terrain feature map is calculated. In order to better understand the mixed terrain semantic feature map, the mixed terrain semantic feature map is illustrated as follows: Figure 2 The mixed terrain is illustrated as an example, as follows: Figure 2 It can be seen that, Figure 2 (a) illustrates two different RGB images containing three types of fallen leaves, grassland, and land; Figure 2 (b) illustrates Figure 2(a) the mixed terrain semantic feature map corresponding to the two images in (a), wherein the red marked part represents the dangerous and difficult terrain, and the green marked part represents the safe and easy terrain; Figure 2 (c) shows the feature map after processing the two images in (a) by using the existing terrain information extraction strategy, wherein the yellow marked part represents the difficulty of terrain, and the green, yellow and red colors represent the safe, medium and dangerous terrains, respectively. It can be seen that the existing terrain information extraction strategy cannot accurately distinguish the difficulty of different terrains. Figure 2 (a) the feature map after processing the two images in (a), wherein the yellow marked part represents the difficulty of terrain (green for safe, yellow for medium, and red for dangerous), and the existing terrain information extraction strategy fails to accurately distinguish the difficulty of different terrains. It should be noted that, in the feature map in (a), the red marked part represents the dangerous terrain, and the green marked part represents the safe terrain. Figure 2 It can be seen that the mixed terrain semantic feature map is actually used to distinguish the difficulty of different terrains by using corresponding color marks, and the color mark corresponds to the sum average of the difficulty of the terrain corresponding to each pixel corresponding to all terrain class labels. The greater the sum average corresponding to a pixel, the redder the color corresponding to the pixel; on the contrary, the smaller the sum average corresponding to a pixel, the greener the color corresponding to the pixel. It should be noted that the terrain class difficulty is obtained by experiments of the moving target on different terrains, and the specific experimental process is as follows: the moving target is controlled to move on the flat ground corresponding to the terrain type, the linear speed of the moving target is controlled to be 2 m / s, the angular speed is gradually accelerated from 0 to 1.5 m / s, the angular speed when the moving target appears dangerous conditions such as slipping and vibration amplitude is recorded during the acceleration process, and the recorded angular speed is divided by the maximum angular speed 1.5 to obtain the difficulty of the current terrain.

[0042] II. Trajectory planning module

[0043] The trajectory planning module is configured to process the moving state of the moving target collected by the moving target at multiple continuous time points, the terrain depth map, and the mixed terrain semantic feature map corresponding thereto transmitted by the perception processing module based on the strategy network configured thereon to obtain the target planning trajectory. The strategy network configured thereon is described in detail below.

[0044] According to one embodiment of the present application, as shown in Figure 3As shown, the policy network comprises a first pre-network and a second pre-network connected in parallel, and a plurality of fully connected layers, a long short-term memory network and an output layer connected in series after the first pre-network and the second pre-network; wherein the first pre-network is composed of a plurality of fully connected layers, for extracting state feature information of a plurality of continuous travel states at different times; the second pre-network is composed of a plurality of convolutional layers, for extracting terrain semantic information of a plurality of continuous semantic feature maps of mixed terrains at different times and terrain geometric information of a plurality of continuous terrain depth maps at different times; the fully connected layers are used for fusion processing of the plurality of continuous state feature information transmitted by the first pre-network and the plurality of continuous terrain semantic information and terrain geometric information transmitted by the second pre-network to obtain sequence fusion features; the long short-term memory network is used for extracting dependency relationships in the sequence fusion features to obtain time sequence features; and the output layer is used for trajectory prediction based on the time sequence features to obtain a target planning trajectory. Wherein, in the policy network Figure 3 In the policy network shown, the first pre-network is composed of two fully connected layers containing 256 neurons and one fully connected layer containing 128 neurons; the second pre-network is composed of a convolutional layer configured with a convolution kernel size of 5x5 and a filter number of 32 and a convolutional layer configured with a convolution kernel size of 3x3 and a filter number of 64; a fully connected layer containing 512 neurons and a fully connected layer containing 256 neurons are connected in series after the first pre-network and the second pre-network; and the output layer is a fully connected layer containing 12 neurons, and the output layer outputs control points of a B-spline, and these control points uniquely determine a B-spline, i.e. the target planning trajectory. It should be noted that the policy network can adjust the network structure and the configuration of the network layers according to actual needs, and the present application does not specifically limit the policy network.

[0045] According to one embodiment of the present application, the policy network is configured to be trained in the following manner: a training set is constructed based on existing path data, wherein the training set includes a plurality of sequence data, each sequence data has a corresponding standard planning trajectory, and each sequence data includes a plurality of time-continuous RGB terrain images, a corresponding terrain depth map and a travel state; a perception processing module is used to process the plurality of time-continuous RGB terrain images in each sequence data to obtain a mixed terrain semantic feature map at each time in the sequence data, a plurality of sequence data after processing is used as the input of the policy network, the target planning trajectory is used as the output for multi-round training, and the parameters of the policy network are updated based on a preset training target. It should be noted that the travel state includes the speed of the travel target, specifically including the angular velocity, six-axis acceleration and linear velocity; also includes the attitude of the travel target, i.e. the configuration state of the direction and position of the travel target in space relative to a certain coordinate system, which describes the spatial pose information such as orientation, tilt or rotation of the travel target at a certain time; also includes the motion state of the travel target, which indicates the state of the travel target in the driving process, such as whether there is shaking, whether there is skidding, whether there is an obstacle, etc.

[0046] According to one embodiment of the present application, the preset training target is:

[0047]

[0048] s.t

[0049] wherein, denotes the policy network; denotes the parameters of the policy network; denotes the time length; denotes the time; denotes the discount coefficient; denotes the reward function; denotes the sequence data; denotes the sequence data the corresponding standard planning trajectory; denotes the total reward obtained by executing the standard planning trajectory; denotes the target planning trajectory obtained by the policy network based on the sequence data ; denotes the total reward obtained by executing the target planning trajectory; denotes the divergence between and ; denotes the policy network before updating, denotes the policy network after updating for multiple times based on ; represents a hyper-parameter. It should be noted that the preset training target is to minimize the difference between the total reward obtained by executing the standard planning trajectory and the total reward obtained by executing the target planning trajectory, and the smaller the difference is, the closer the target planning trajectory is to the standard planning trajectory, so as to make the policy network learn how to plan a trajectory; the preset training target also sets a constraint condition to constrain the update distance, so as to increase the training stability, wherein the value of the hyper-parameter is determined according to actual needs. It should be further noted that represents the total reward obtained by executing the standard planning trajectory from the time step to the time step , represents the immediate reward obtained by executing the corresponding action at the time step of the standard planning trajectory; represents the total reward obtained by executing the target planning trajectory from the time step to the time step , represents the immediate reward obtained by executing the corresponding action at the time step of the standard planning trajectory.

[0050] III. Self-correction module

[0051] According to one embodiment of the present application, as shown in Figure 1 , the system further comprises a self-correction module for collecting all actual motion trajectories with a Euclidean distance greater than or equal to a preset distance threshold from the corresponding target planning trajectory as a correction trajectory set after the moving target moves according to the target planning trajectory, and transmitting the correction trajectory set to the trajectory planning module to make the policy network configured on the trajectory planning module retrain based on the correction trajectory set, wherein the Euclidean distance is used to measure the similarity between the actual motion trajectory and the corresponding target planning trajectory. It should be noted that the preset distance threshold can be set to 4, or can be determined according to actual needs, and the present application does not make specific limitations thereon. It should be further noted that the main purpose of setting the self-correction module is to realize self-correction and optimization of the policy network, so that the policy network can maintain good adaptability and stability in complex and variable terrain environments, and output better target planning trajectories.

[0052] According to one embodiment of the present invention, the policy network is configured to be retrained in the following manner: the RGB terrain images, terrain depth maps, and travel states at multiple moments corresponding to the target planning trajectory corresponding to each actual motion trajectory in the corrected trajectory set are used as retraining sequence data, and the actual motion trajectory is used as the standard motion trajectory of its corresponding retraining sequence data. All retraining sequence data and their corresponding standard motion trajectory constitute a retraining data set; a perception processing module is used to process the RGB terrain images at multiple moments in each retraining sequence data in the retraining data set to obtain a mixed terrain semantic feature map at each moment, and multiple rounds of training are performed with the processed multiple retraining sequence data as input and the new target planning trajectory as output, and the parameters of the policy network are updated based on the new training objectives.

[0053] According to one embodiment of the present invention, the new training objective is:

[0054]

[0055] in, represents the retrained policy network, Represents the retraining sequence data, Represents retraining sequence data The corresponding standard motion trajectory, Represents the reward obtained by executing the standard motion trajectory, Represents the policy network based on retraining sequence data The new target planning trajectory obtained is represents the reward obtained by executing the new target planning trajectory, It represents the Euclidean distance between the standard motion trajectory and the new target planning trajectory. It should be noted that the Euclidean distance between the standard motion trajectory and the new target planning trajectory is added as a constraint in the new training objective to improve the stability of the policy network.

[0056] The trajectory planning system described in the above embodiment can plan a path for a moving target in a mixed terrain environment, so that the moving target moves based on the planned path, thereby achieving the moving target's task in a real environment. Figure 4 As shown, the present invention also provides a moving target travel control system, which includes: a mixed terrain trajectory planning system as described in the aforementioned embodiment, used to plan a travel path for a moving target in a mixed terrain environment to obtain a target planning trajectory; a motion control module, used to calculate the angular velocity and linear velocity of the moving target when executing the target planning trajectory based on the target planning trajectory; and a PID control module, used to control the moving target to move according to the target planning trajectory based on the angular velocity and linear velocity obtained by the motion control module.

[0057] In order to verify that the trajectory planning system proposed in the present application has better trajectory planning effect compared with the trajectory planning scheme proposed in the prior art, the experimental results shown in Table 1 are obtained by performing comparative experiments and ablation experiments.

[0058] The experimental results shown in Table 1 include five evaluation indexes, which are success rate, smoothness, standardized trajectory length, dangerous planning times and trajectory tracking difficulty. Among them, the success rate represents the probability that the moving target can successfully reach the destination according to the planned trajectory; the smoothness represents the continuity and fluency of the moving target in the actual movement process according to the planned trajectory (a smooth trajectory means that the moving target does not have a sharp change in direction during movement, and the smaller the smoothness, the better the planned trajectory); the standardized trajectory length represents the average length of the planned trajectory (the shorter the length, the better the planned trajectory); the dangerous planning times represent the number of times that the moving target encounters danger in the actual movement process according to the planned trajectory (the fewer the dangerous planning times, the better the planned trajectory); and the trajectory tracking difficulty represents the Euclidean distance between the planned trajectory and the actual movement trajectory (the smaller the distance, the better the planned trajectory).

[0059] The comparative experiments in Table 1 are used to evaluate the difference between the existing trajectory planning method and the trajectory planning scheme proposed in the present application. The comparative experiments include four existing trajectory planning methods, among which the traditional method uses fast RRT and model predictive control to plan a trajectory in a wild environment; the RAORN method uses single-class terrain classification combined with learning-based planning to plan a trajectory in a wild environment; the Terp method converts the terrain environment into a reward form of a passable map combined with reinforcement learning to plan a trajectory in a wild environment (the greater the reward, the easier it is to pass through); and the BADGR method directly uses the data collected by the original sensor combined with reinforcement learning to plan a trajectory in a wild environment.

[0060] The ablation experiments in Table 1 are used to evaluate the trajectory planning effect after simplifying part of the processing method in the trajectory planning scheme proposed in the present application. Among them, removing historical data means deleting part of the training data when training the strategy network, that is, reducing the amount of training data; removing self-correction means that the strategy network does not perform retraining processing; removing mixed terrain processing means only classifying single terrain for RGB images.

[0061] As can be seen from Table 1, compared with the other four existing trajectory planning methods, the present application is slightly inferior to the traditional method in smoothness, but the performance of the present application in the remaining indicators is better than that of the other four existing trajectory planning methods.

[0062] Table 1

[0063] Compared with the prior art, the present application has the beneficial effects that: (1) the RGB image is classified by a multi-label terrain classification method to obtain more comprehensive terrain information, so as to avoid the problem of key information loss caused by insufficient terrain information extraction; (2) the strategy network is trained by using offline meta-reinforcement learning, which avoids the dangerous planning when interacting with the real world during online training, and the meta-reinforcement learning can establish the optimal planning strategy for different terrain types, and by regarding the planning tasks on different terrains as different meta-tasks, a strategy network that can cope with complex and variable terrain environments is trained; (3) the self-correction module is set to realize the self-correction and optimization of the strategy network, so that the strategy network maintains good adaptability and stability in complex and variable terrain environments

[0064] It should be noted that although the above describes the steps in a specific order, it does not mean that the steps must be performed in the above specific order, in fact, some of the steps can be performed concurrently, or even in a changed order, as long as the required functions can be achieved.

[0065] The present application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0066] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, and any suitable combination of the foregoing. A non-transitory, computer-readable storage medium, as used herein, is expressly intended not to encompass transitory propagating signals per se.

[0067] The embodiments of the application have been described above, the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles, practical applications or technical improvements in the market of the embodiments, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A mixed terrain trajectory planning system for planning a path for a moving target in a mixed terrain environment, characterized in that: The system comprises: A perception processing module is configured with a pre-trained semantic segmentation network and an encoder; wherein the pre-trained semantic segmentation network is used to extract semantic features from multiple continuous RGB terrain images collected at multiple moments of the moving target to obtain a multi-label terrain feature map at each moment; the encoder is used to encode the multi-label terrain feature map at each moment to obtain a mixed terrain semantic feature map at each moment; wherein the multi-label terrain feature map represents the terrain category contained in each pixel in the RGB terrain image, and the mixed terrain semantic feature map represents the difficulty of passing different terrain categories in the RGB terrain image; The trajectory planning module is used to process the moving state of the moving target, the terrain depth map, and the corresponding mixed terrain semantic feature maps at multiple moments collected by the moving target based on the policy network configured thereon, and obtain the target planning trajectory; wherein, the policy network takes the mixed terrain semantic feature maps, the terrain depth map, and the moving state as input, and takes the target planning trajectory as output, and is obtained through offline training based on meta-reinforcement learning.

2. The system according to claim 1, wherein: The pre-trained semantic segmentation network is configured to perform terrain classification on each pixel in the RGB terrain image at each moment to obtain the probability that each pixel in the RGB terrain image at each moment belongs to each terrain category, and the terrain category corresponding to all terrain category probabilities greater than a preset classification threshold in each pixel is used as the terrain category label of the pixel to obtain a multi-label terrain feature map.

3. The system according to claim 2, characterized in that The encoder is configured to encode the multi-label terrain feature map in the following manner to obtain a mixed terrain semantic feature map: Based on the preset terrain category difficulty, the sum mean of the terrain difficulty corresponding to all terrain category labels corresponding to each pixel in the multi-label terrain feature map is calculated.

4. The system according to claim 3, characterized in that The strategy network includes a first front-end network and a second front-end network connected in parallel, and multiple fully connected layers, a long short-term memory network and an output layer connected in series after the first front-end network and the second front-end network; wherein the first front-end network is composed of multiple fully connected layers, which is used to extract state feature information of continuous travel states at multiple moments; the second front-end network is composed of multiple convolutional layers, which is used to extract terrain semantic information of continuous mixed terrain semantic feature maps at multiple moments and terrain geometric information of continuous terrain depth maps at multiple moments; the fully connected layer is used to fuse the state feature information of multiple moments continuously transmitted by the first front-end network and the terrain semantic information and terrain geometric information of multiple moments transmitted by the second front-end network to obtain sequence fusion features; the long short-term memory network is used to extract dependency relationships in the sequence fusion features to obtain time series features; the output layer is used to perform trajectory prediction based on the time series features to obtain the target planning trajectory.

5. The system according to claim 4, characterized in that The policy network is configured to be trained as follows: A training set is constructed based on the existing path data. The training set includes multiple sequence data, each of which has a corresponding standard planned trajectory. Each sequence data includes multiple continuous RGB terrain images, corresponding terrain depth maps, and travel status. The perception processing module is used to process the continuous RGB terrain images at multiple moments in each sequence data to obtain the mixed terrain semantic feature map at each moment in the sequence data. The processed multiple sequence data are used as the input of the policy network and the target planning trajectory is used as the output for multiple rounds of training. The parameters of the policy network are updated based on the preset training objectives.

6. The system according to claim 5, characterized in that The preset training objectives are: in, represents the policy network; Represents the parameters of the policy network; Indicates the length of time; Indicates the moment; represents the discount factor; represents the reward function; Represents sequence data; Representing sequence data The corresponding standard planning trajectory; represents the total reward obtained by executing the standard planning trajectory; Representation policy network based on sequence data The target planning trajectory obtained; represents the total reward obtained by executing the target planning trajectory; express and between Divergence, represents the policy network before updating, Indicates The policy network after multiple updates to the benchmark; represents a hyperparameter.

7. The system according to claim 6, characterized in that The system further comprises: The self-correction module is used to collect all actual motion trajectories obtained after the moving target moves along the target planning trajectory and whose Euclidean distance with the corresponding target planning trajectory is greater than or equal to a preset distance threshold as a correction trajectory set, and pass the correction trajectory set to the trajectory planning module so that the policy network configured on the trajectory planning module is retrained based on the correction trajectory set, wherein the Euclidean distance is used to measure the similarity between the actual motion trajectory and its corresponding target planning trajectory.

8. The system according to claim 7, characterized in that The policy network is configured to be retrained as follows: The RGB terrain images, terrain depth maps, and travel states at multiple moments corresponding to the target planned trajectory corresponding to each actual motion trajectory in the corrected trajectory set are used as retraining sequence data, and the actual motion trajectory is used as the standard motion trajectory of its corresponding retraining sequence data. All retraining sequence data and their corresponding standard motion trajectories constitute the retraining dataset; The perception processing module is used to process the RGB terrain images at multiple moments in each retraining sequence data in the retraining dataset to obtain the mixed terrain semantic feature map at each moment. Multiple rounds of training are carried out with the processed multiple retraining sequence data as input and the new target planning trajectory as output, and the parameters of the policy network are updated based on the new training objectives.

9. The system according to claim 8, characterized in that The new training objectives are: in, represents the retrained policy network, Represents the retraining sequence data, Represents retraining sequence data The corresponding standard motion trajectory, Represents the reward obtained by executing the standard motion trajectory, Represents the policy network based on retraining sequence data The new target planning trajectory obtained is represents the reward obtained by executing the new target planning trajectory, Represents the Euclidean distance between the standard motion trajectory and the new target planning trajectory.

10. A moving target travel control system, characterized in that: The system comprises: The mixed terrain trajectory planning system according to any one of claims 1 to 9, used for planning a travel path for a moving target in a mixed terrain environment to obtain a target planning trajectory; A motion control module is used to calculate the angular velocity and linear velocity of the moving target when executing the target planning trajectory based on the target planning trajectory; The PID control module is used to control the moving target to move according to the target planned trajectory based on the angular velocity and linear velocity obtained by the motion control module.

Images