High-speed rail car body plane spraying full coverage path planning method

Abstract

The application discloses a high-speed rail car body plane spraying full-coverage path planning method, comprising the following steps: selecting a high-speed rail plane car body to be planned, and determining a spraying area; building a grid map for path planning; different gray scale values represent the state, repetition degree and real-time position of different grid units, and form a gray scale state matrix; defining an Agent action direction; constructing a multi-level feature aggregation network model as an Agent structure; the gray scale values of the movable grid units around the current position of the Agent are converted into a direction probability matrix through calculation, and the direction probability matrix acts on the network output of the Agent; formulating a reward function and a path quality evaluation index; training the multi-level feature aggregation network model in the built spraying environment, and selecting an optimal path according to the path quality evaluation index. The application can be applied to any plane car body with different sizes and different door and window configurations, so that the spraying coverage rate reaches 100%, and the repetition rate is extremely low.

Description

Technical Field

[0001] This invention relates to the field of intelligent industry, specifically a method for full-coverage path planning of planar spraying on high-speed train bodies. Background Technology

[0002] Full-coverage path planning technology is an important research topic in many fields, such as industrial automation robots, autonomous driving, and signal communication. Currently, the painting process for high-speed rail bodies mainly relies on manual spraying, and the experience and physical condition of workers significantly affect the spraying quality, making it difficult to guarantee the uniformity and thickness of the coating. Automated spraying robots are considered to be able to replace manual spraying due to their efficiency and stability, but the automated putty spraying process is still in the exploratory and experimental stage, with few application examples.

[0003] In automated spraying, the spraying path significantly impacts the thickness and uniformity of the final coating. Repeated or incorrect paths can produce a "wavy" coating, thus affecting coating performance. Commonly used coverage path planning methods have two main drawbacks: insufficient coverage prevents spraying from being carried out, while excessive repetition leads to paint loss. Furthermore, to ensure a smooth vehicle surface, the spray gun must be temporarily closed along repeated paths. However, prolonged closure can cause the paint to harden, leading to spray gun clogging. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a path planning method for full coverage of high-speed train body surface spraying, which addresses the shortcomings of the prior art. This method can be applied to flat car bodies of any size and with different door and window configurations, so that the spraying coverage rate reaches 100% while ensuring an extremely low repetition rate.

[0005] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for full-coverage path planning of planar spraying on high-speed train bodies, including:

[0007] Step 1: Select the high-speed rail car body to be planned and determine the spraying area; select a suitable spray gun and build a grid map for path planning;

[0008] Step 2: Different grayscale values ​​represent the state, repetition level, and real-time location of different grid cells, forming a grayscale state matrix; define the agent's action direction;

[0009] Step 3: Construct a multi-level feature aggregation network model as the Agent structure in the Rainbow reinforcement learning framework;

[0010] Step 4: The gray values ​​of the movable grid cells around the Agent's current location are converted into a direction probability matrix and applied to the Agent's network output;

[0011] Step 5: Based on the comprehensive environmental coverage and the degree of grid cell repetition, formulate the reward function and path quality evaluation indicators;

[0012] Step 6: Train a multi-level feature aggregation network model in the constructed spraying environment, and select the optimal path based on the path quality evaluation index.

[0013] As a further improvement to the present invention, step 1 specifically comprises:

[0014] Step 1.1: Select the high-speed rail car body to be planned and determine the spraying position on the car body surface;

[0015] Step 1.2: Select the fan width size as... The spray gun is designed according to the planar dimensions of the high-speed train body. Based on the location of obstacles, a path planning grid map is created. The edge of the grid map is filled with obstacles. The grid cells in the grid map are squares with a side length of one unit. The dimensions of the grid map are:

[0016]

[0017] in, The vertical length of the vehicle body. The horizontal length of the vehicle body. This represents the number of raster cells in the vertical direction of the raster map. This represents the number of raster cells in the horizontal direction of the raster map.

[0018] As a further improvement to the present invention, step 2 specifically comprises:

[0019] Step 2.1: Use grayscale values Representing the state, repetition count, and real-time location of different grid cells, each grid cell in the raster map is assigned a grayscale value, thus forming a grayscale state matrix:

[0020]

[0021] in, Grayscale value The number of times a grid cell is repeated. This represents the attenuation rate of the grayscale value.

[0022] Step 2.2: The Agent's movement directions include four directions: up, down, left, and right, with a step size of 1 grid cell. During movement, the Agent cannot move into obstacle areas.

[0023] As a further improvement to the present invention, step 3 specifically comprises:

[0024] Step 3.1: Construct a multi-level feature aggregation network model as the Agent structure in the Rainbow reinforcement learning framework, including an input processing module, two basic modules, two tracking modules, an aggregation module, a flattening layer, an advantage value fully connected layer, and an action value fully connected layer;

[0025] Step 3.2: The input processing module includes a 3×3 convolutional layer with a stride of 1, layer normalization, and a ReLU activation function layer;

[0026] The grayscale state matrices calculated in steps 3.2.1 and 2.1 are processed by the input processing module, and the output features are... The feature map size remains unchanged, feature information is extracted, and spatial location information is preserved at the same time;

[0027] Step 3.3: The basic module includes two convolutional blocks consisting of a 3×3 convolutional layer, a layer normalization, a ReLU activation function layer, a 3×3 convolutional layer, and a layer normalization, as well as a channel adjustment layer consisting of a 3×3 convolutional layer and a layer normalization.

[0028] Output features of steps 3.3.1 and 3.2.1 After passing through the first convolutional block in the first basic module, the feature map size is halved, and the output feature map is... ;feature After passing through the channel adjustment layer in the first basic module, the feature map size is halved, and the output feature map is... The outputs of the two are merged. ,feature After passing through a ReLU activation layer and the second convolutional block in the first basic module, the feature map size remains unchanged, and the output feature map is... Residual join merging ,feature After passing through a ReLU activation function layer, activation features are obtained. ;

[0029] Step 3.3.2, Features After passing through the first convolutional block in the second basic module, the feature map size remains unchanged, but the number of channels doubles, and the output feature map is... ;feature After passing through the channel adjustment layer in the second basic module, the feature map size remains unchanged, but the number of channels doubles, resulting in the output feature map. The outputs of the two are merged. ,feature After passing through a ReLU activation layer and the second convolutional block in the second basic module, the feature map size and number of channels remain unchanged, and the output feature map is... Residual join merging ,feature After passing through a ReLU activation function layer, activation features are obtained. ;

[0030] Step 3.4: The first tracking module includes two convolutional blocks consisting of 3×3 convolutional layers, layer normalization, and ReLU activation function layers; a channel adjustment layer consisting of 3×3 convolutional layers and layer normalization; and an extraction layer consisting of 3×3 convolutional layers and ReLU activation function layers. The second tracking module includes two convolutional blocks consisting of 3×3 convolutional layers, layer normalization, and ReLU activation function layers; and an extraction layer consisting of 3×3 convolutional layers and ReLU activation function layers.

[0031] Step 3.4.1: The output features from step 3.3.2... After passing through the first convolutional block in the first tracking module, the feature map size remains unchanged, and the output feature map is... Residual join merging ,feature After passing through the second convolutional block in the first tracking module, the feature map size remains unchanged, and the output feature map is... Residual join merging ,feature After the extraction layer in the first tracking module, the feature map size is halved, the number of channels is doubled, and the output feature map is... ;feature After passing through the channel adjustment layer in the first tracking module, the feature map size is halved, the number of channels is doubled, and the output feature map is... The outputs of the two are merged. , to obtain features ;

[0032] Step 3.4.2, Features After the first convolutional block in the second tracking module, the feature map size and number of channels remain unchanged, and the output feature map is... Residual join merging ,feature After passing through the second convolutional block in the second tracking module, the feature map size and number of channels remain unchanged, and the output feature map is then processed. Residual join merging ,feature After extraction in the second tracking module, the feature map size and number of channels remain unchanged, and the output feature map is... Residual join merging , to obtain features ;

[0033] Step 3.5: The aggregation module contains two synthesis blocks consisting of a 3×3 convolutional layer and an upsampling layer;

[0034] Output features of steps 3.5.1 and 3.4.2 After the first synthesis block, the feature map size doubles, the number of channels is halved, and the output feature map... ;feature Output features of step 3.3.2 Perform residual join merging, i.e. , to obtain features ;

[0035] Step 3.5.2, Features After the second synthesis block, the feature map size remains the same, but the number of channels is halved, and the output feature map is... ;feature Output features of step 3.3.1 Perform residual join merging, i.e. , to obtain features ;

[0036] Step 3.6, Features After flattening in the flattening layer, the value is processed by the dominance value fully connected layer and the action value fully connected layer to obtain the dominance value. and action value After processing by formula (3), the Agent's output is obtained. :

[0037]

[0038] in, express The average value, Indicates the initial Value array.

[0039] As a further improvement to the present invention, step 4 specifically comprises:

[0040] Step 4.1: In the current grayscale state matrix, find the Agent's current position, i.e., the grid cell with a grayscale value of 170, and add the grayscale values ​​of its four adjacent grid cells (top, bottom, left, and right) to the array. middle;

[0041] Step 4.2, Array The elements in the formula (4) are modified to their corresponding values ​​according to the rules:

[0042]

[0043] in, express The first in the array The value of each element;

[0044] Step 4.3 array after The function is processed into a direction probability array. ;

[0045] Step 4.4: Perform the following operations according to the method in formula (5): The output of the Agent in step 3.6 After being corrected Value array, and the corrected Replace elements with a value of 0 in the value array. :

[0046]

[0047] As a further improvement to the present invention, step 5 specifically comprises:

[0048] Step 5.1: Define the reward function, assigning different reward values ​​to the Agent based on its different behaviors, thus quantifying the Agent's actions into specific numerical values.

[0049]

[0050] in, Indicates the first The rewards generated by taking action Indicates the first step, This indicates the base number for rewards for effective action steps. The discount factor representing the effective number of steps taken. Indicates the number of valid action steps. Indicates the first Step is an effective step. Indicates the base reward for invalid action steps. The discount factor representing the number of times a raster cell is repeated. Indicates the number of times a grid repeats. Indicates the first Step is an invalid step. Rewards are given for achieving full coverage. Indicates coverage rate;

[0051] Step 5.2: Develop route evaluation indicators, including environmental coverage. Path repetition rate Total Path Reward Environmental coverage Meeting the target is a prerequisite for completing the task; path repetition rate. Total Path Reward The criteria for judging the optimal solution of the coverage path are:

[0052]

[0053]

[0054]

[0055] in, This is the number of covered raster cells. It is the total number of grid cells that need to be covered in the environment. It is the number of steps consumed by the covered path. It is the first The rewards generated by taking action.

[0056] As a further improvement to the present invention, step 6 specifically comprises:

[0057] Step 6.1: Construct the spraying environment of Step 1.2 and define the training parameters for reinforcement learning;

[0058] Step 6.2: Reset the spraying environment, Agent starting point, path rewards, and... , Used to store past action information;

[0059] Step 6.3: Denote the grayscale state matrix from step 2.1 as... , As input to the multi-level feature aggregation network constructed in step 3, the network output... ;

[0060] Step 6.4: Follow the method in step 4 to... Acting on After being corrected The value array, and replace the elements with a value of 0 in the corrected Q-value array with Agent selection after correction The action corresponding to the largest element in the value array And execute;

[0061] Step 6.5: After the Agent takes action, the grayscale state matrix is ​​updated to... The environment provides a reward for each step of the action based on the reward function. Task completion indicator Current coverage Repetition rate Total Path Reward ;

[0062] Step 6.6 It was stored as a complete action process. ;

[0063] Step 6.7, if If the number of experiences in the pool reaches a preset value, then extract from it. One experience point is used to train the Agent. After training is complete, proceed to step 6.8; if If the number of experiences in the process does not reach the preset value, proceed directly to step 6.8;

[0064] Step 6.8: If the task is completed... for If the Agent reaches the maximum number of actions, the task is terminated, and the process jumps to step 6.2 to continue the loop.

[0065] Otherwise, denote the current grayscale state matrix as , As the input to the latest multi-level feature aggregation network, the network output Proceed to step 6.4;

[0066] Step 6.9: If the number of iterations in steps 6.2-6.8 reaches the maximum number of iterations, exit the task, select the optimal coverage path based on the path evaluation index, and draw the graph.

[0067] To achieve the above-mentioned technical objectives, another technical solution adopted by the present invention is as follows:

[0068] A high-speed rail vehicle body planar spraying full-coverage path planning system includes:

[0069] The spraying environment building module is used to build a grid map for path planning based on the high-speed rail car body plane dimensions, spray gun spraying fan width size, and obstacle positions.

[0070] The State-Action Space Construction Module is used to construct the grayscale state matrix and action space of the Agent.

[0071] The multi-level feature aggregation network module is used to construct a multi-level feature aggregation network model, and uses the multi-level feature aggregation network model as the Agent model structure in the Rainbow reinforcement learning framework.

[0072] The action mask module is used to convert the gray values ​​of the actionable grid cells around the Agent's current position into a direction probability matrix and apply it to the Agent's network output.

[0073] The Rewards and Indicators module is used to comprehensively consider environmental coverage and grid cell redundancy to formulate reward functions and path quality evaluation indicators.

[0074] The model training module is used to train a multi-level feature aggregation network model in the constructed spraying environment and select the optimal path based on the path quality evaluation index.

[0075] The beneficial effects of this invention are as follows:

[0076] (1) This invention optimizes the structure of the Agent in the Rainbow reinforcement learning framework and uses a multi-level feature aggregation network model to process environmental grayscale state information. Among them, the input processing module is used to extract simple feature information while retaining spatial location information; the basic module is used to extract shallow semantic information, improving the training stability and convergence speed of the network; the tracking module is used to efficiently capture multi-scale deep features and expand the receptive field. Its special structure enables it to process local details and broader contextual information at the same time, effectively balancing feature extraction capability, computational efficiency and network depth; the aggregation module is specifically used to fuse multi-scale features. This module not only improves the network's ability to fuse multi-scale information, but also optimizes the information flow between different levels.

[0077] (2) This invention proposes an observation-based approach. The action masking module transforms the grayscale matrix within the feasible range into an action probability matrix, which is then applied to the Agent's output to guide action selection. This process eliminates dangerous actions while guiding the Agent to complete the coverage task.

[0078] (3) This invention proposes a specific reward function to balance exploration rewards and inform the Agent of the task objective and the degree of environmental exploration. The reward function guides the Agent to perform effective coverage or escape from dead zones based on the path coverage and the number of times the cell is repeated. When the Agent's behavior is effective, the reward function feeds back a gradually increasing positive reward value; when the Agent's behavior is repeated, the reward function feeds back a negative reward value that is proportional to the number of times the grid cell is repeated; when the Agent completes the full coverage task, the reward function will give the Agent a huge reward.

[0079] (4) This invention addresses the problems of insufficient path coverage resulting from traditional commonly used coverage path planning methods, leading to the inability to carry out painting tasks, excessive repetition rates causing paint waste, and excessively long spray gun shut-off times causing spray gun clogging. It proposes a full-coverage, low-repetition-rate full-coverage path planning method for high-speed rail vehicle body planar painting. This method combines deep learning and reinforcement learning, integrating multiple innovative modules to improve the coverage of the planned path for the automated painting robot and reduce the repetition rate. Furthermore, this method and process are also applicable to planar surfaces of other vehicles or similar structures, providing an efficient coverage path planning method for automated painting. Attached Figure Description

[0080] Figure 1 A flowchart for the path planning method of full coverage spraying on the flat surface of high-speed rail bodies.

[0081] Figure 2 This is a flowchart illustrating the training process of the improved deep reinforcement learning agent of this invention.

[0082] Figure 3 For spraying colored environmental interfaces.

[0083] Figure 4 This represents the grayscale state matrix and the agent's action direction.

[0084] Figure 5 This is a flowchart of the algorithm for multi-level feature aggregation network and action mask.

[0085] Figure 6 This is a diagram of the network module structure in a multi-level feature aggregation network.

[0086] Figure 6 (a) represents the structure diagram of the basic module.

[0087] Figure 6 (b) shows the structure diagram of the tracking module.

[0088] Figure 6 (c) represents the structure diagram of the aggregation module.

[0089] Figure 7 This is a graph showing the changes in coverage and repetition rate corresponding to the number of iteration rounds.

[0090] Figure 8 This is a graph showing the change in total reward for each iteration round.

[0091] Figure 9 This is the optimal path diagram of the improved deep reinforcement learning algorithm in the spraying environment. Detailed Implementation

[0092] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0093] A method for full-coverage path planning of planar spraying on high-speed train bodies, such as... Figure 1 As shown, it includes the following steps:

[0094] Step 1: Model the spraying environment. Based on the high-speed rail car body dimensions, the spray gun spraying fan width, and the location of obstacles, build a grid map for path planning.

[0095] Step 1 specifically includes:

[0096] Step 1.1: Select the high-speed rail car body to be planned, and determine the areas to be painted on the car body surface (doors, windows, etc. are obstacles and should be avoided from being painted). The painting areas and obstacles are as follows: Figure 3 As shown, Figure 3 For spraying colored environmental interfaces;

[0097] Step 1.2: Select the fan width size as... The spray gun is designed according to the planar dimensions of the high-speed train body. Based on the location of obstacles, a path planning grid map is created (with obstacles filling the edges). The grid cells in the grid map are squares with a side length of one unit. The dimensions of the grid map are:

[0098]

[0099] in, The vertical length of the vehicle body. The horizontal length of the vehicle body. This represents the number of raster cells in the vertical direction of the raster map. This represents the number of raster cells in the horizontal direction of the raster map.

[0100] Step 2: Define the state and action space. Different gray values ​​represent the state of different grid cells, the degree of repetition of the region, and the real-time position of the Agent, forming a gray-scale state matrix. The direction of Agent action constitutes the action space (in reinforcement learning, the Agent processes environmental state information and acts in the environment. Its behavior and position in the environment represent the behavior and position of the spray gun in reality).

[0101] Step 2 specifically refers to:

[0102] Step 2.1, as follows Figure 4 As shown, using grayscale values Representing the state, repetition count, and real-time location of different grid cells, each grid cell in the raster map is assigned a grayscale value, thus forming a grayscale state matrix:

[0103]

[0104] in, Grayscale value The number of times a grid cell is repeated. This represents the attenuation rate of the grayscale value.

[0105] Step 2.2, as follows Figure 4 As shown, the Agent can move in four directions: up, down, left, and right. The Agent can move in these four directions, with a step size of 1 grid cell. When moving, the Agent cannot move into obstacle areas.

[0106] Step 3: Construct a multi-level feature aggregation network model based on convolutional neural networks and attention mechanisms, which serves as the Agent structure in the Rainbow reinforcement learning framework.

[0107] Step 3 specifically includes:

[0108] Step 3.1, as follows Figure 5 As shown, a multi-level feature aggregation network model is constructed as the Agent structure in the Rainbow reinforcement learning framework, including an input processing module, two basic modules, two tracking modules, an aggregation module, a flattening layer, and two fully connected layers (including a fully connected layer for advantage value and a fully connected layer for action value). Figure 6 (a) represents the structure diagram of the basic module. Figure 6 (b) shows the structure diagram of the tracking module. Figure 6 (c) represents the structure diagram of the aggregation module. These represent the parameters for kernel size, stride, padding edges, grouped convolution, and dilated convolution, respectively. Figure 6 In the basic module of (a), gray represents convolutional layers and orange represents channel adjustment layers. Figure 6 In the tracking module of (b), gray represents convolutional layers, orange represents channel adjustment layers, dashed lines indicate whether channel adjustment layers are needed depending on the situation, and yellow represents extraction layers. Figure 6 In the aggregation module (c), gray represents the convolutional layer, blue represents the upsampling layer, E is the output feature of step 3.4.2, C is the output feature of step 3.3.2, and B6 is the output feature of step 3.3.1.

[0109] Step 3.2: The input processing module includes a 3×3 convolutional layer with a stride of 1, layer normalization, and a ReLU activation function layer;

[0110] The grayscale state matrices calculated in steps 3.2.1 and 2.1 are processed by the input processing module, and the output features are... The feature map size remains unchanged, simple feature information is extracted, and spatial location information is preserved at the same time;

[0111] Step 3.3: The basic module includes two convolutional blocks consisting of a 3×3 convolutional layer, a layer normalization, a ReLU activation function layer, a 3×3 convolutional layer, and a layer normalization, as well as a channel adjustment layer consisting of a 3×3 convolutional layer and a layer normalization.

[0112] Output features of steps 3.3.1 and 3.2.1 After passing through the first convolutional block in the first basic module, the feature map size is halved, and the output feature map is... ;feature After passing through the channel adjustment layer in the first basic module, the feature map size is halved, and the output feature map is... The outputs of the two are merged. After passing through a ReLU activation function layer and then through the second convolutional block in the first basic module, the feature map size remains unchanged, and the output feature map is... Residual join merging ,feature After passing through a ReLU activation function layer, activation features are obtained. ;

[0113] Step 3.3.2, Features After passing through the first convolutional block in the second basic module, the feature map size remains unchanged, but the number of channels doubles, and the output feature map is... ;feature After passing through the channel adjustment layer in the second basic module, the feature map size remains unchanged, but the number of channels doubles, resulting in the output feature map. The outputs of the two are merged. After passing through a ReLU activation function layer and then through the second convolutional block in the second basic module, the feature map size and number of channels remain unchanged, and the output feature map is... Residual join merging ,feature After passing through a ReLU activation function layer, activation features are obtained. ;

[0114] Step 3.4: The first tracking module includes two convolutional blocks consisting of 3×3 convolutional layers, layer normalization, and ReLU activation function layers; a channel adjustment layer consisting of 3×3 convolutional layers and layer normalization; and an extraction layer consisting of 3×3 convolutional layers and ReLU activation function layers. The second tracking module includes two convolutional blocks consisting of 3×3 convolutional layers, layer normalization, and ReLU activation function layers; and an extraction layer consisting of 3×3 convolutional layers and ReLU activation function layers.

[0115] Step 3.4.1: The output features from step 3.3.2... After passing through the first convolutional block in the first tracking module, the feature map size remains unchanged, and the output feature map is... Residual join merging ,feature After passing through the second convolutional block in the first tracking module, the feature map size remains unchanged, and the output feature map is... Residual join merging ,feature After the extraction layer in the first tracking module, the feature map size is halved, the number of channels is doubled, and the output feature map is... ; After passing through the channel adjustment layer in the first tracking module, the feature map size is halved, the number of channels is doubled, and the output feature map is... The outputs of the two are merged. , to obtain features ;

[0116] Step 3.4.2, Features After the first convolutional block in the second tracking module, the feature map size and number of channels remain unchanged, and the output feature map is... Residual join merging ,feature After passing through the second convolutional block in the second tracking module, the feature map size and number of channels remain unchanged, and the output feature map is then processed. Residual join merging ,feature After extraction in the second tracking module, the feature map size and number of channels remain unchanged, and the output feature map is... Residual join merging , to obtain features ;

[0117] Step 3.5: The aggregation module contains two synthesis blocks consisting of a 3×3 convolutional layer and an upsampling layer;

[0118] Output features of steps 3.5.1 and 3.4.2 After the first synthesis block, the feature map size doubles, the number of channels is halved, and the output feature map... ; and the output features of step 3.3.2 Perform residual join merging. ;

[0119] Step 3.5.2, Features After the second synthesis block, the feature map size remains the same, but the number of channels is halved, and the output feature map is... ; and the output features of step 3.3.1 Perform residual join merging. , to obtain features ;

[0120] Step 3.6, Features After flattening (2D to 1D) in the flattening layer, the value is processed through fully connected layers for both dominance value and action value to obtain the dominance value. and action value The two are processed by formula (3) to obtain the Agent's output. .

[0121]

[0122] in, express The average value, This is the initial Q-value array.

[0123] Step 4: Construct an action mask. The grayscale values ​​of the actionable grid cells around the Agent's current location are calculated and converted into a directional probability distribution matrix. And it acts on the Agent's network output.

[0124] Step 4 specifically includes:

[0125] Step 4.1: In the current grayscale state matrix, find the Agent's current position, i.e., the grid cell with a grayscale value of 170, and add the grayscale values ​​of its four adjacent grid cells (top, bottom, left, and right) to the array. middle;

[0126] Step 4.2, Array The elements in the formula (4) are modified to their corresponding values ​​according to the rules:

[0127]

[0128] in, express The first in the array The value of each element;

[0129] Step 4.3 array after The function is processed into a direction probability array. ;

[0130] Step 4.4: Perform the following operations according to the method in formula (5): The output of the Agent in step 3.6 This yields the corrected Q-value array, and elements with a value of 0 in the corrected Q-value array are replaced with... :

[0131]

[0132] Step 5: Based on the comprehensive environmental coverage and the degree of grid cell repetition, formulate the reward function and path quality evaluation indicators;

[0133] Step 5 specifically includes:

[0134] Step 5.1: Define the reward function, assigning different reward values ​​to the Agent based on its different behaviors, thus quantifying the Agent's actions into specific numerical values.

[0135]

[0136] in, Indicates the first The rewards generated by taking action Indicates the first step, This indicates the base number for rewards for effective action steps. The discount factor representing the effective number of steps taken. Indicates the number of valid action steps. Indicates the first Step is an effective step. Indicates the base reward for invalid action steps. The discount factor representing the number of times a raster cell is repeated. Indicates the number of times a grid repeats. Indicates the first Step is an invalid step. Rewards are given for achieving full coverage. Indicates coverage rate;

[0137] Step 5.2: Develop route evaluation indicators, focusing on coverage route planning and environmental coverage rate. Meeting the target is a prerequisite for completing the task; path repetition rate. Total Path Reward The criteria for judging the optimal solution of the coverage path are:

[0138]

[0139]

[0140]

[0141] in, This is the number of covered raster cells. It is the total number of grid cells that need to be covered in the environment. It is the number of steps consumed by the covered path. It is the first The rewards generated by taking action.

[0142] Step 6: Train a multi-level feature aggregation network model in the constructed spraying environment, and select the optimal path based on the path quality evaluation index.

[0143] like Figure 2 As shown, step 6 specifically includes:

[0144] Step 6.1: Construct the spraying environment of Step 1.2 and define the training parameters for reinforcement learning (learning rate, discount rate, number of iterations, maximum number of steps per round, and experience capacity, etc.).

[0145] Step 6.2: Reset the spraying environment, Agent starting point (bottom left corner), path rewards, and... wait, Used to store past action information;

[0146] Step 6.3: Denote the grayscale state matrix from step 2.1 as... , As input to the multi-level feature aggregation network constructed in step 3, the network outputs an initial Q-value array;

[0147] Step 6.4: Follow the method in step 4 to... Applying this to the initial Q-value array yields the corrected Q-value array (the initial Q-value array and...). After multiplication, we obtain the corrected Q-value array, and then replace the elements with values ​​of 0 in the corrected Q-value array with... Agent selection The action corresponding to the largest element in the value array And execute;

[0148] Step 6.5: After the Agent takes action, the grayscale state matrix is ​​updated to... The environment provides a reward for each step of the action based on the reward function. Task completion indicator Current coverage Repetition rate Total Path Reward ;

[0149] Step 6.6 It was stored as a complete action process. ;

[0150] Step 6.7, if If the number of experiences in the pool reaches a preset value, then extract from it. One experience point is used to train the Agent. After training is complete, proceed to step 6.8; if If the number of experiences in the process does not reach the preset value, proceed directly to step 6.8;

[0151] Step 6.8: If the task is completed... for If the Agent reaches the maximum number of actions, the task is terminated, and the process jumps to step 6.2 to continue the loop.

[0152] Otherwise, denote the current grayscale state matrix as , As input to the latest multi-level feature aggregation network, the network outputs an initial Q-value array, and step 6.4 is executed;

[0153] Step 6.9: If the number of iterations in steps 6.2-6.8 reaches the maximum number of iterations, exit the task, select the optimal coverage path based on the path evaluation index, and draw the graph. Figure 7 This is a graph showing the changes in coverage and repetition rate corresponding to the number of iteration rounds. Figure 8 This is a graph showing the change in total reward for each iteration round. Figure 9 In the path diagram, green arrows indicate the direction of movement of the current grid cell, red arrows indicate repeated paths, and pentagrams indicate the end point of the path.

[0154] This embodiment also provides a high-speed rail vehicle body planar spraying full-coverage path planning system, including:

[0155] The spraying environment building module is used to build a grid map for path planning based on the high-speed rail car body plane dimensions, spray gun spraying fan width size, and obstacle positions.

[0156] The state and action space construction module is used to construct the grayscale state matrix and action space of the Agent;

[0157] The multi-level feature aggregation network module is used to construct a multi-level feature aggregation network model, and uses the multi-level feature aggregation network model as the Agent model structure in the Rainbow reinforcement learning framework, thereby improving the Agent structure and enhancing the environmental information processing and environmental exploration capabilities.

[0158] The action mask module is used to convert the gray values ​​of the actionable grid cells around the Agent's current position into a direction probability matrix and apply it to the Agent's network output. That is, an Action Mask is superimposed on the Agent's network output to guide the agent to achieve the goal of full coverage while avoiding dangerous actions.

[0159] The Rewards and Metrics module is used to comprehensively consider the environmental coverage and the degree of grid cell repetition to formulate reward functions and path quality evaluation metrics. The reward function quantifies the agent's actions into specific numerical values ​​to judge the quality of the actions. The path evaluation metrics are used to determine whether the full coverage task has been completed and the quality of the coverage path.

[0160] The model training module is used to train a multi-level feature aggregation network model in the constructed spraying environment. Based on the path quality evaluation index, it selects the optimal path, that is, it trains a reinforcement learning agent in the spraying environment, finds the best network parameters within a specified number of iterations, and finds the optimal path based on the path evaluation index.

[0161] The scope of protection of this invention includes, but is not limited to, the above embodiments. The scope of protection of this invention is defined by the claims. Any substitutions, modifications, or improvements to this technology that are easily conceived by those skilled in the art fall within the scope of protection of this invention.

Claims

1. A method for full-coverage path planning of planar spraying on high-speed train bodies, characterized in that, include: Step 1: Select the high-speed rail car body to be planned and determine the spraying area; select a suitable spray gun and build a grid map for path planning; Step 2: Different grayscale values ​​represent the state, repetition level, and real-time location of different grid cells, forming a grayscale state matrix; define the agent's action direction; Step 3: Construct a multi-level feature aggregation network model as the Agent structure in the Rainbow reinforcement learning framework; Step 4: The gray values ​​of the movable grid cells around the Agent's current location are converted into a direction probability matrix and applied to the Agent's network output; Step 5: Based on the comprehensive environmental coverage and the degree of grid cell repetition, formulate the reward function and path quality evaluation indicators; Step 6: Train a multi-level feature aggregation network model in the constructed spraying environment, and select the optimal path based on the path quality evaluation index; Step 1 specifically includes: Step 1.1: Select the high-speed rail car body to be planned and determine the spraying position on the car body surface; Step 1.2: Select the fan width size as... The spray gun is designed according to the planar dimensions of the high-speed train body. Based on the location of obstacles, a path planning grid map is created. The edge of the grid map is filled with obstacles. The grid cells in the grid map are squares with a side length of one unit. The dimensions of the grid map are: in, The vertical length of the vehicle body. The horizontal length of the vehicle body. This represents the number of raster cells in the vertical direction of the raster map. This represents the number of raster cells in the horizontal direction of the raster map.

2. The high-speed rail vehicle body planar spraying full-coverage path planning method according to claim 1, characterized in that, Step 2 specifically refers to: Step 2.1: Use grayscale values Representing the state, repetition count, and real-time location of different grid cells, each grid cell in the raster map is assigned a grayscale value, thus forming a grayscale state matrix: in, Grayscale value The number of times a grid cell repeats. This represents the attenuation rate of the grayscale value. Step 2.2: The Agent's movement directions include four directions: up, down, left, and right, with a step size of 1 grid cell. During movement, the Agent cannot move into obstacle areas.

3. The high-speed rail vehicle body planar spraying full-coverage path planning method according to claim 2, characterized in that, Step 3 specifically includes: Step 3.1: Construct a multi-level feature aggregation network model as the Agent structure in the Rainbow reinforcement learning framework, including an input processing module, two basic modules, two tracking modules, an aggregation module, a flattening layer, an advantage value fully connected layer, and an action value fully connected layer; Step 3.2: The input processing module includes a 3×3 convolutional layer with a stride of 1, layer normalization, and a ReLU activation function layer; The grayscale state matrices calculated in steps 3.2.1 and 2.1 are processed by the input processing module, and the output features are... The feature map size remains unchanged, feature information is extracted, and spatial location information is preserved at the same time; Step 3.3: The basic module includes two convolutional blocks consisting of a 3×3 convolutional layer, a layer normalization, a ReLU activation function layer, a 3×3 convolutional layer, and a layer normalization, as well as a channel adjustment layer consisting of a 3×3 convolutional layer and a layer normalization. Output features of steps 3.3.1 and 3.2.1 After passing through the first convolutional block in the first basic module, the feature map size is halved, and the output feature map is... ;feature After passing through the channel adjustment layer in the first basic module, the feature map size is halved, and the output feature map is... The outputs of the two are merged. ,feature After passing through a ReLU activation layer and the second convolutional block in the first basic module, the feature map size remains unchanged, and the output feature map is... Residual join merging ,feature After passing through a ReLU activation function layer, activation features are obtained. ; Step 3.3.2, Features After passing through the first convolutional block in the second basic module, the feature map size remains unchanged, but the number of channels doubles, and the output feature map is... ;feature After passing through the channel adjustment layer in the second basic module, the feature map size remains unchanged, but the number of channels doubles, resulting in the output feature map. The outputs of the two are merged. ,feature After passing through a ReLU activation layer and the second convolutional block in the second basic module, the feature map size and number of channels remain unchanged, and the output feature map is... Residual join merging ,feature After passing through a ReLU activation function layer, activation features are obtained. ; Step 3.4: The first tracking module includes two convolutional blocks consisting of 3×3 convolutional layers, layer normalization, and ReLU activation function layers; a channel adjustment layer consisting of 3×3 convolutional layers and layer normalization; and an extraction layer consisting of 3×3 convolutional layers and ReLU activation function layers. The second tracking module includes two convolutional blocks consisting of 3×3 convolutional layers, layer normalization, and ReLU activation function layers, and an extraction layer consisting of 3×3 convolutional layers and ReLU activation function layers; Step 3.4.1: The output features from step 3.3.2... After passing through the first convolutional block in the first tracking module, the feature map size remains unchanged, and the output feature map is... Residual join merging ,feature After passing through the second convolutional block in the first tracking module, the feature map size remains unchanged, and the output feature map is... Residual join merging ,feature After the extraction layer in the first tracking module, the feature map size is halved, the number of channels is doubled, and the output feature map is... ;feature After passing through the channel adjustment layer in the first tracking module, the feature map size is halved, the number of channels is doubled, and the output feature map is... The outputs of the two are merged. , to obtain features ; Step 3.4.2, Features After the first convolutional block in the second tracking module, the feature map size and number of channels remain unchanged, and the output feature map is... Residual join merging ,feature After passing through the second convolutional block in the second tracking module, the feature map size and number of channels remain unchanged, and the output feature map is then processed. Residual join merging ,feature After extraction in the second tracking module, the feature map size and number of channels remain unchanged, and the output feature map is... Residual join merging , to obtain features ; Step 3.5: The aggregation module contains two synthesis blocks consisting of a 3×3 convolutional layer and an upsampling layer; Output features of steps 3.5.1 and 3.4.2 After the first synthesis block, the feature map size doubles, the number of channels is halved, and the output feature map... ;feature Output features of step 3.3.2 Perform residual join merging, i.e. , to obtain features ; Step 3.5.2, Features After the second synthesis block, the feature map size remains the same, but the number of channels is halved, and the output feature map is... ;feature Output features of step 3.3.1 Perform residual join merging, i.e. , to obtain features ; Step 3.6, Features After flattening in the flattening layer, the value is processed by the dominance value fully connected layer and the action value fully connected layer to obtain the dominance value. and action value After processing by formula (3), the Agent's output is obtained. : in, express The average value, Indicates the initial Value array.

4. The high-speed rail vehicle body planar spraying full-coverage path planning method according to claim 3, characterized in that, Step 4 specifically includes: Step 4.1: In the current grayscale state matrix, find the Agent's current position, i.e., the grid cell with a grayscale value of 170, and add the grayscale values ​​of its four adjacent grid cells (top, bottom, left, and right) to the array. middle; Step 4.2, Array The elements in the formula (4) are modified to their corresponding values ​​according to the rules: in, express The first in the array The value of each element; Step 4.3 array after The function is processed into a direction probability array. ; Step 4.4: Perform the following operations according to the method in formula (5): The output of the Agent in step 3.6 After being corrected Value array, and the corrected Replace elements with a value of 0 in the value array. :

5. The high-speed rail vehicle body planar spraying full-coverage path planning method according to claim 1, characterized in that, Step 5 specifically includes: Step 5.1: Define the reward function, assigning different reward values ​​to the Agent based on its different behaviors, thus quantifying the Agent's actions into specific numerical values. in, Indicates the first The rewards generated by taking action Indicates the first step, This indicates the base number for rewards for effective action steps. The discount factor representing the effective number of steps taken. Indicates the number of valid action steps. Indicates the first Step is an effective step. Indicates the base reward for invalid action steps. The discount factor representing the number of times a raster cell is repeated. Indicates the number of times a grid repeats. Indicates the first Step is an invalid step. Rewards are given for achieving full coverage. Indicates coverage rate; Step 5.2: Develop route evaluation indicators, including environmental coverage. Path repetition rate Total Path Reward Environmental coverage Meeting the target is a prerequisite for completing the task; path repetition rate. Total Path Reward The criteria for judging the optimal solution of the coverage path are: in, This is the number of covered raster cells. It is the total number of grid cells that need to be covered in the environment. It is the number of steps consumed by the covered path. It is the first The rewards generated by taking action.

6. The high-speed rail vehicle body planar spraying full-coverage path planning method according to claim 4, characterized in that, Step 6 specifically includes: Step 6.1: Construct the spraying environment of Step 1.2 and define the training parameters for reinforcement learning; Step 6.2: Reset the spraying environment, Agent starting point, path rewards, and... , Used to store past action information; Step 6.3: Denote the grayscale state matrix from step 2.1 as... , As input to the multi-level feature aggregation network constructed in step 3, the network output... ; Step 6.4: Follow the method in step 4 to... Acting on After being corrected The value array, and replace the elements with a value of 0 in the corrected Q-value array with Agent selection after correction The action corresponding to the largest element in the value array And execute; Step 6.5: After the Agent takes action, the grayscale state matrix is ​​updated to... The environment provides a reward for each step of the action based on the reward function. Task completion indicator Current coverage Repetition rate Total Path Reward ; Step 6.6 It was stored as a complete action process. ; Step 6.7, if If the number of experiences in the pool reaches a preset value, then extract from it. One experience point is used to train the Agent. After training is complete, proceed to step 6.8; if If the number of experiences in the process does not reach the preset value, proceed directly to step 6.8; Step 6.8: If the task is completed... for If the Agent reaches the maximum number of actions, the task is terminated, and the process jumps to step 6.2 to continue the loop. Otherwise, denote the current grayscale state matrix as , As the input to the latest multi-level feature aggregation network, the network output Proceed to step 6.4; Step 6.9: If the number of iterations in steps 6.2-6.8 reaches the maximum number of iterations, then exit the task, select the optimal coverage path based on the path evaluation index, and draw the graph.

7. A path planning system for full coverage of flat spraying on high-speed train bodies, characterized in that, include: The spraying environment building module is used to build a grid map for path planning based on the high-speed rail car body plane dimensions, spray gun spraying fan width size, and obstacle positions. Specifically, building a grid map for route planning involves: selecting the high-speed rail vehicle body to be planned and determining the spraying positions on the vehicle body surface; selecting the fan-shaped size... The spray gun is designed according to the planar dimensions of the high-speed train body. Based on the location of obstacles, a path planning grid map is created. The edge of the grid map is filled with obstacles. The grid cells in the grid map are squares with a side length of one unit. The dimensions of the grid map are: in, The vertical length of the vehicle body. The horizontal length of the vehicle body. This represents the number of raster cells in the vertical direction of the raster map. This represents the number of raster cells in the horizontal direction of the raster map. The State-Action Space Construction Module is used to construct the grayscale state matrix and action space of the Agent. The multi-level feature aggregation network module is used to construct a multi-level feature aggregation network model, and uses the multi-level feature aggregation network model as the Agent model structure in the Rainbow reinforcement learning framework. The action mask module is used to convert the gray values ​​of the actionable grid cells around the Agent's current position into a direction probability matrix and apply it to the Agent's network output. The Rewards and Indicators module is used to comprehensively consider environmental coverage and grid cell redundancy to formulate reward functions and path quality evaluation indicators. The model training module is used to train a multi-level feature aggregation network model in the constructed spraying environment and select the optimal path based on the path quality evaluation index.

Images