A lightweight terrain segmentation method and system based on category attention and a terminal

Abstract

The application discloses a kind of based on category attention lightweight terrain segmentation method, system and terminal, the method includes: the RGB original image is input to multi-scale feature extractor layer and is carried out feature extraction, obtains multi-layer feature map, and each layer feature map is fused and is handled to obtain multiple fusion features;Decoder module obtains corresponding prediction feature map by attention calculation to each fusion feature, calculates class loss according to each prediction feature map and corresponding label, and calculates auxiliary loss according to each prediction feature map and original label, and the overall loss is calculated according to class loss and auxiliary loss, when overall loss meets the requirement, the trained semantic segmentation framework is obtained;Unstructured outdoor terrain environment information is input to semantic segmentation framework, and terrain segmentation result map is generated, and terrain segmentation result map is used for local navigation planning.The application improves the performance and efficiency of semantic segmentation framework in unstructured terrain segmentation.

Description

Technical Field

[0001] This invention relates to the field of robot vision navigation technology, and in particular to a lightweight terrain segmentation method, system, terminal, and computer-readable storage medium based on category attention. Background Technology

[0002] Environmental perception is a crucial component of visual navigation, involving the capture and analysis of image information about the robot's surrounding environment. This process includes image analysis and processing, feature extraction, and obstacle identification. In the field of robot visual navigation, combining deep learning for environmental perception has become a mainstream research direction. Road segmentation is a key step in environmental perception, its main purpose being to distinguish safe, drivable areas and provide a safe and reliable navigation basis for the robot's decision-making system. Existing segmentation efforts primarily focus on handling structured scenes like CityScapes, while unstructured scenes present several challenges.

[0003] First, compared to structured scenes, unstructured outdoor scenes contain many terrain categories with intersecting boundaries and highly similar appearances, making accurate segmentation of such terrain extremely challenging. Errors are also unavoidable when manually creating labels. This difficulty is further amplified when such terrain environments occupy only a small portion of the image. Existing methods often suffer from significant classification errors due to the overlapping features between classes when handling such scenes. In machine vision navigation, if a robot mistakenly confuses a dangerous area with a safe area, it can cause irreparable damage. Second, previous research has shown that in unstructured terrain, due to its complex and overlapping characteristics, fine-grained segmentation is inefficient for machine vision navigation systems. Coarse-grained classification is sufficient for navigation tasks. For example, the navigation system only needs to classify all trees, rocks, and houses as impassable obstacles, without needing to classify them into their respective categories.

[0004] Finally, since robotic devices are typically embedded computers, their computing resources and memory space are limited to a certain extent, making it impossible to test large models. While lightweighting the model can effectively reduce resource consumption, it inevitably leads to a loss of performance.

[0005] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0006] The main objective of this invention is to provide a lightweight terrain segmentation method, system, terminal, and computer-readable storage medium based on category attention, aiming to solve the problems in the prior art, such as insufficient unstructured semantic segmentation, low efficiency of fine-grained segmentation, and insufficient computing resources, which result in the inability to provide effective terrain segmentation results for navigation.

[0007] To achieve the above objectives, this invention provides a lightweight terrain segmentation method based on category attention, which includes the following steps:

[0008] The original RGB image is acquired and input into a multi-scale feature extractor layer for feature extraction to obtain multi-layer feature maps. Each feature map is then fused to obtain multiple fused features, which are then input into the decoder module.

[0009] The decoder module sets the number of heads for multi-head attention according to the navigation region type, obtains the corresponding predicted feature map for each fused feature through attention calculation, calculates the category loss based on each predicted feature map and the corresponding label, calculates the auxiliary loss based on each predicted feature map and the original label, and calculates the overall loss based on the category loss and the auxiliary loss. When the overall loss meets the requirements, the trained semantic segmentation framework is obtained.

[0010] Unstructured outdoor terrain environment information is acquired and input into a semantic segmentation framework to generate a terrain segmentation result map, which is used for local navigation planning.

[0011] Optionally, in the lightweight terrain segmentation method based on category attention, the multi-scale feature extractor includes a lightweight Transformer module; the lightweight Transformer module includes a lightweight multi-head attention module and a lightweight forward feedback module.

[0012] The decoder module includes a category attention module, which in turn includes a multi-head attention module.

[0013] Optionally, in the lightweight terrain segmentation method based on category attention, the lightweight multi-head attention module consists of multiple attention heads, and the attention mechanism formula is as follows:

[0014] (1)

[0015] in, There are three learnable matrices. for In the corresponding dimensions, Softmax represents the activation function, and T represents the transpose;

[0016] Multi-head attention calculation is defined as follows:

[0017] (2)

[0018] (3)

[0019] in, This indicates multi-head attention calculation. Let m represent the i-th attention head, and m represent the number of attention heads. Let i be the feature matrix corresponding to the i-th attention head. This indicates attention calculation.

[0020] Optionally, the lightweight terrain segmentation method based on category attention, wherein acquiring the original RGB image, inputting the original RGB image into a multi-scale feature extractor layer for feature extraction to obtain multi-layer feature maps, fusing each layer of feature maps to obtain multiple fused features, and inputting them into the decoder module, specifically includes:

[0021] The lightweight multi-head attention module divides the sequence features of the original RGB image into K groups. The different sequence features in the K groups are then subjected to attention operations and then stacked together to obtain stacked features.

[0022] The lightweight forward feedback module divides the stacked features output by the lightweight multi-head attention module into... Grouping, the grouped features are dimensionally expanded and then fed into a convolutional layer, compressed back to their original size, and then... The group features are spliced ​​together to obtain spliced ​​features. The extracted spliced ​​features at each level are then fused according to a preset resolution to obtain fused features.

[0023] Optionally, the lightweight terrain segmentation method based on category attention, wherein the lightweight multi-head attention module divides the sequence features of the original RGB image into K groups, and performs attention operations on the different sequence features of the K groups respectively before stacking and concatenating them to obtain stacked features, specifically includes:

[0024] For any input sequence features, The grouping transformation operation is defined as follows:

[0025] (4)

[0026] (5)

[0027] in, This represents the sequence features before grouping. This represents the sequence features after the attention calculation for the i-th group. Indicates input features, This indicates that a convolution operation is performed on the input features. This indicates that the feature map is expanded along a certain dimension. This represents the attention calculation after grouping. This indicates a feature concatenation operation. This represents the stacked features obtained by stacking features after grouping calculations are completed;

[0028] The computational complexity is The number of parameters is from Descending to ,in, The dimension representing the learning vector. This represents the number of channels.

[0029] Optionally, in the lightweight terrain segmentation method based on category attention, the lightweight forward feedback module divides the stacked features output by the lightweight multi-head attention module into... Grouping, the grouped features are dimensionally expanded and then fed into a convolutional layer, compressed back to their original size, and then... The group features are concatenated to obtain concatenated features. The extracted concatenated features at each level are then fused according to a preset resolution to obtain fused features, specifically including:

[0030] The specific definition of the lightweight forward feedback module is as follows:

[0031] (6)

[0032] (7)

[0033] in, This represents the serialization features of the input. Indicates feature segmentation, This indicates an adjustment of the dimension size. For the defined convolutional layer operations, For the output serialization features, This indicates a feature concatenation operation. This indicates that a lightweight forward feedback operation is being performed.

[0034] The splicing features at each level are fused according to a preset resolution to obtain the fused features. ;

[0035] The lightweight forward feedback module reduces the number of parameters from the original Descending to The computational complexity is from Descending to .

[0036] Optionally, the lightweight terrain segmentation method based on category attention, wherein the decoder module sets the number of heads for multi-head attention according to the navigation region type, obtains the corresponding predicted feature map for each fused feature through attention calculation, calculates the category loss based on each predicted feature map and the corresponding label, calculates the auxiliary loss based on each predicted feature map and the original label, calculates the overall loss based on the category loss and the auxiliary loss, and obtains the trained semantic segmentation framework when the overall loss meets the requirements, specifically including:

[0037] The number of heads for multi-head attention is set according to the pre-defined navigation region types. Each head is used to perform multi-head attention based on each fusion feature. Generate a feature map for a category;

[0038] Adjust the dimensions of each feature map, and obtain the predicted feature map corresponding to each feature map through attention calculation. Extract the corresponding category from each original label and recreate the binary classification label;

[0039] Calculate the predicted feature map The cross-entropy loss with the original label is used as the basis for calculating the class loss by summing the calculated losses for each class. ;

[0040] The cross-entropy loss is defined as follows:

[0041] (8)

[0042] in, The dimension of the feature map. Indicates the number of categories. This represents the output probability corresponding to each category. Indicates the truth value of the label;

[0043] Category loss The definition is as follows:

[0044] (9)

[0045] A deep supervised approach is used to calculate the cross-entropy loss between the generated predicted feature map and the original label, thus obtaining an auxiliary loss. ;

[0046] Then, based on category loss and auxiliary losses Calculate the overall loss :

[0047] (10)

[0048] When the overall loss When the requirements are met, a well-trained semantic segmentation framework is obtained.

[0049] Furthermore, to achieve the above objectives, the present invention also provides a lightweight terrain segmentation system based on category attention, wherein the lightweight terrain segmentation system based on category attention includes:

[0050] The feature extraction and fusion module is used to acquire the original RGB image, input the original RGB image into the multi-scale feature extractor layer for feature extraction, obtain multi-layer feature maps, fuse each feature map to obtain multiple fused features, and input them into the decoder module;

[0051] The loss calculation and model training module is used by the decoder module to set the number of heads of multi-head attention according to the navigation region type, obtain the corresponding prediction feature map for each fused feature through attention calculation, calculate the category loss based on each prediction feature map and the corresponding label, calculate the auxiliary loss based on each prediction feature map and the original label, calculate the overall loss based on the category loss and the auxiliary loss, and obtain the trained semantic segmentation framework when the overall loss meets the requirements.

[0052] The terrain segmentation and navigation planning module is used to acquire unstructured outdoor terrain environment information, input the unstructured outdoor terrain environment information into the semantic segmentation framework, and generate a terrain segmentation result map, which is used for local navigation planning.

[0053] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a lightweight terrain segmentation program based on category attention stored in the memory and executable on the processor, wherein when the lightweight terrain segmentation program based on category attention is executed by the processor, it implements the steps of the lightweight terrain segmentation method based on category attention as described above.

[0054] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a lightweight terrain segmentation program based on category attention, which, when executed by a processor, implements the steps of the lightweight terrain segmentation method based on category attention as described above.

[0055] In this invention, an RGB original image is acquired and input into a multi-scale feature extractor layer for feature extraction, resulting in multi-layer feature maps. Each feature map is fused to obtain multiple fused features, which are then input into a decoder module. The decoder module sets the number of heads for multi-head attention according to the navigation region type. Each fused feature is used to obtain a corresponding predicted feature map through attention calculation. A category loss is calculated based on each predicted feature map and its corresponding label, and an auxiliary loss is calculated based on each predicted feature map and its original label. The overall loss is calculated based on the category loss and the auxiliary loss. When the overall loss meets the requirements, a trained semantic segmentation framework is obtained. Unstructured outdoor terrain environment information is acquired and input into the semantic segmentation framework to generate a terrain segmentation result map, which is used for local navigation planning. This invention establishes global dependencies, more effectively captures global information in the image, and generates feature maps with a global receptive field. Channel decomposition reduces the number of parameters in the self-attention calculation process, thereby improving the performance and efficiency of the semantic segmentation framework in unstructured terrain segmentation. The generated terrain segmentation result map can be applied to local navigation planning. Attached Figure Description

[0056] Figure 1 This is a flowchart of a preferred embodiment of the lightweight terrain segmentation method based on category attention of the present invention;

[0057] Figure 2 This is a schematic diagram of the semantic segmentation framework in a preferred embodiment of the lightweight terrain segmentation method based on category attention of the present invention.

[0058] Figure 3 This is a schematic diagram of the lightweight multi-head attention module processing process in a preferred embodiment of the lightweight terrain segmentation method based on category attention of the present invention.

[0059] Figure 4 This is a schematic diagram of the lightweight forward feedback module processing in a preferred embodiment of the lightweight terrain segmentation method based on category attention of the present invention.

[0060] Figure 5 This is a schematic diagram of the processing procedure of the category attention module in a preferred embodiment of the lightweight terrain segmentation method based on category attention of the present invention;

[0061] Figure 6 This is a structural diagram of a preferred embodiment of the lightweight terrain segmentation system based on category attention of the present invention;

[0062] Figure 7 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0064] To address the issues of insufficient work on unstructured semantic segmentation, low efficiency of fine-grained segmentation, and inadequate computational resources, this invention proposes the following solutions. Firstly, a lightweight Transformer module is proposed, primarily comprising a lightweight multi-head attention module and a lightweight forward feedback module. The lightweight multi-head attention module divides the original image sequence features into K groups before performing the attention operation. Different features in each of the K groups undergo attention operations before being stacked and concatenated. Similarly, the lightweight forward feedback module uses depthwise separable convolution to extract local information and introduces residual connections to ensure that performance is not excessively compromised. Before entering the forward feedback, the original features are first divided into K groups. To make the sequence features suitable for depthwise separable convolution, they need to be dimensionally expanded before being fed into the convolutional layer. Finally, they are compressed back to their original size and the K groups of features are concatenated. Secondly, this invention also proposes a category-based attention module. First, the features extracted from the multi-scale feature extractor at various levels need to be fused at a certain resolution. The feature maps pass through the multi-head attention module before output. The number of heads in multi-head attention depends on the pre-defined navigation region types, with each head generating a feature map for a specific category. Then, the dimensions of each feature map are adjusted to obtain a corresponding predicted feature map through attention computation. The category is then extracted from the original labels to reconstruct binary labels. Finally, the cross-entropy loss between the predicted feature map and the label is calculated. The calculated losses for each category are summed to obtain the class loss, thereby enhancing the accuracy of coarse-grained semantic segmentation. This improves the model's performance and efficiency in unstructured terrain segmentation.

[0065] The lightweight terrain segmentation method based on category attention described in the preferred embodiment of the present invention, such as... Figure 1 and Figure 2 As shown, the lightweight terrain segmentation method based on category attention includes the following steps:

[0066] Step S10: Obtain the original RGB image, input the original RGB image into the multi-scale feature extractor layer for feature extraction, obtain multi-layer feature maps, perform fusion processing on each layer feature map to obtain multiple fused features, and input them into the decoder module.

[0067] Specifically, such as Figure 2The diagram illustrates the semantic segmentation framework used in this invention. This framework is an improvement upon existing excellent semantic segmentation frameworks, incorporating a lightweight Transformer module (used to reduce the number of parameters in the self-attention computation process) and a category attention module. The multi-scale feature extractor includes a lightweight Transformer module; the lightweight Transformer module includes a lightweight multi-head attention module (i.e.,...). Figure 2 The lightweight multi-head attention mechanism and the lightweight forward feedback module (i.e.) Figure 2 The lightweight forward feedback network in the decoder module includes a category attention module (i.e., a category attention network). Figure 2 The category attention mechanism in the text includes a multi-head attention module.

[0068] like Figure 3 As shown, this represents the lightweight multi-head attention module of the present invention. The lightweight multi-head attention module consists of multiple attention heads, and the attention mechanism formula is as follows:

[0069] (1)

[0070] in, There are three learnable matrices. for In the corresponding dimensions, Softmax represents the activation function, and T represents the transpose;

[0071] Multi-head attention calculation is defined as follows:

[0072] (2)

[0073] (3)

[0074] in, This indicates multi-head attention calculation. Let m represent the i-th attention head, and m represent the number of attention heads. Let i be the feature matrix corresponding to the i-th attention head. This indicates attention calculation.

[0075] To conserve computational resources and reduce dot product operations in attention, this invention proposes a lightweight multi-head attention mechanism. For example... Figure 3 As shown, before performing the attention operation, the lightweight multi-head attention module divides the sequence features of the original RGB image into K groups. The different sequence features in each of the K groups undergo an attention operation, and then are stacked and concatenated to obtain stacked features; for example... Figure 4As shown, the lightweight forward feedback module divides the stacked features output by the lightweight multi-head attention module into... Grouping, the grouped features are dimensionally expanded and then fed into a convolutional layer, compressed back to their original size, and then... The group features are spliced ​​together to obtain spliced ​​features. The extracted spliced ​​features at each level are then fused according to a preset resolution to obtain fused features.

[0076] Figure 3 and Figure 4 In N This indicates the number of input feature maps in a single operation. H and W Indicates the height and width of the feature map; C This indicates the number of channels in the image, and the subscripts are used to distinguish different feature maps. L Indicates to pressing the image H and W Compression of these two dimensions creates a new dimension. L The actual size is H * W .

[0077] For any input sequence features, The grouping transformation operation is defined as follows:

[0078] (4)

[0079] (5)

[0080] in, This represents the sequence features before grouping. Represents the sequence features after the attention calculation for the i-th group ( and (The meanings are the same, k represents the number of input features). Indicates input features, This indicates that a convolution operation is performed on the input features. This indicates that the feature map is expanded along a certain dimension. This represents the attention calculation after grouping. This indicates a feature concatenation operation. This represents the stacked features obtained by stacking features after grouping calculations are completed.

[0081] This invention reduces computational complexity by almost half through grouping, resulting in a computational complexity of... The number of parameters is from Descending to ,in, The dimension representing the learning vector. This represents the number of channels.

[0082] Lightweight forward feedback module such as Figure 4 As shown, depthwise separable convolutions are used to extract local information, and residual connections are introduced to ensure that performance is not excessively compromised. Before entering the lightweight feedforward module, the stacked features output by the lightweight multi-head attention module first need to be divided into... To make sequence features suitable for depthwise separable convolutions, they need to be dimensionally expanded before being fed into the convolutional layer, and finally compressed back to their original size. Group features are spliced ​​together. Figure 4 As shown The specific definition of the lightweight forward feedback module is as follows:

[0083] (6)

[0084] (7)

[0085] in, This represents the serialization features of the input. Indicates feature segmentation, This indicates an adjustment of the dimension size. For the defined convolutional layer operations, For the output serialization features, This indicates a feature concatenation operation. This indicates a lightweight forward feedback operation. The stitched features from each level are fused at a preset resolution to obtain the fused features. .

[0086] In this invention, the lightweight forward feedback module reduces the number of parameters from the original... Descending to The computational complexity is from Descending to .

[0087] Step S20: The decoder module sets the number of heads for multi-head attention according to the navigation region type, obtains the corresponding predicted feature map for each fused feature through attention calculation, calculates the category loss based on each predicted feature map and the corresponding label, calculates the auxiliary loss based on each predicted feature map and the original label, calculates the overall loss based on the category loss and the auxiliary loss, and obtains the trained semantic segmentation framework when the overall loss meets the requirements.

[0088] Specifically, to improve the model's segmentation accuracy for each category, this invention utilizes the characteristics of multi-head attention mechanisms to propose a category attention module. Figure 5 As shown.

[0089] First, the features extracted from each level of the multi-scale feature extractor need to be fused at a certain resolution. The fused features are as follows: The number of heads for multi-head attention is set according to the pre-defined navigation region types (i.e., the number of heads for multi-head attention depends on the pre-defined navigation region types). Each head is used to perform multi-head attention based on each fusion feature. Generate a feature map for a category.

[0090] Then, the dimensions of each feature map are adjusted so that the predicted feature map corresponding to each feature map is obtained through attention calculation. Extract the corresponding category from each original label and recreate the binary classification label.

[0091] Finally, the predicted feature map is calculated. The cross-entropy loss with the original label is used as the basis for calculating the class loss by summing the calculated losses for each class. .

[0092] The cross-entropy loss is defined as follows:

[0093] (8)

[0094] in, The dimension of the feature map. Indicates the number of categories. This represents the output probability corresponding to each category. Indicates the truth value of the label.

[0095] Category loss The definition is as follows:

[0096] (9)

[0097] This invention also employs a deep supervision method to calculate the cross-entropy loss between the generated predicted feature map and the original label, thereby obtaining an auxiliary loss. Auxiliary loss can assist in model training.

[0098] In summary, based on category loss and auxiliary losses Calculate the overall loss :

[0099] (10)

[0100] When the overall loss A well-trained semantic segmentation framework is obtained when the requirements are met (e.g., the smaller the overall loss, the better).

[0101] This invention first inputs the original RGB image into a multi-scale feature extractor. The input RGB image undergoes feature extraction, with each feature map layer's channel count doubled after downsampling, while its scale is halved. During the feature fusion stage, the features extracted from each layer are fused. Due to the different feature resolutions of each layer, all features need to be compressed to a single dimension. Secondly, to reduce the excessive computational burden during feature extraction, this invention employs a cleverly designed lightweight structure to avoid this impact while ensuring performance during the fusion process. Furthermore, since coarse-grained terrain classification can assist in visual navigation, a category attention mechanism is used for coarse-grained classification. This mechanism primarily operates in the decoder module. After the feature map output from the decoder module passes through the multi-head attention mechanism, each channel of the output feature map focuses on a specific category for loss calculation. Finally, all category features are combined for auxiliary loss calculation.

[0102] Step S30: Obtain unstructured outdoor terrain environment information, input the unstructured outdoor terrain environment information into the semantic segmentation framework, and generate a terrain segmentation result map. The terrain segmentation result map is used for local navigation planning.

[0103] Specifically, after processing in steps S10 and S20, a semantic segmentation framework (model) that meets the requirements is obtained, which can then be used. Figure 2 The entire semantic segmentation framework shown is used to obtain the terrain segmentation result map, thus acquiring unstructured outdoor terrain environment information. The unstructured outdoor terrain environment information is input into the semantic segmentation framework to generate the terrain segmentation result map, which is used for local navigation planning.

[0104] This invention proposes a lightweight terrain segmentation method based on category attention, which focuses on performing segmentation tasks in unstructured outdoor terrain environments and achieves good performance. Terrain segmentation fulfills the environmental perception task in navigation, which greatly helps robots understand the different terrain environments in the current scene and avoid dangerous situations. Therefore, the terrain segmentation result map generated by this invention can be applied to local navigation planning. For example, the terrain segmentation result generated by this invention can divide the terrain into passable and impassable areas, and this result can be used to construct a semantic grid map required for navigation. Initially, each position in the semantic grid map represents only two values, 0 and 1, indicating passability or impassability. Next, different distance values ​​are calculated for each passable position. This distance value describes the distance from that position to the impassable area. Finally, a semantic grid map containing different distance values ​​is obtained. Local navigation planning can be completed by applying some path planning algorithms on this semantic grid map.

[0105] This invention proposes a lightweight terrain segmentation method based on category attention. First, it proposes a Transformer-based multi-scale feature extraction network (i.e., a multi-scale feature extractor). This network establishes global dependencies, more effectively capturing global information in the image and generating feature maps with a global receptive field. To ensure operational efficiency on mobile devices, this invention further proposes a lightweight strategy for the Transformer model. This strategy employs a channel-splitting self-attention module to decompose the correlation of channel dimensions, thereby reducing the number of parameters in the self-attention computation process. Furthermore, this invention innovatively employs a category attention method to further improve the prediction accuracy for different types of terrain. Finally, comparative experiments are conducted on three publicly available unstructured datasets—RUGD, RELLIS-3D, and WVD—to compare the proposed method with current leading semantic segmentation algorithms. The results show that the proposed method exhibits better performance and can be further deployed on mobile devices.

[0106] Furthermore, such as Figure 6 As shown, based on the above-described lightweight terrain segmentation method based on category attention, this invention also provides a lightweight terrain segmentation system based on category attention, wherein the lightweight terrain segmentation system based on category attention includes:

[0107] The feature extraction and fusion module 51 is used to acquire the original RGB image, input the original RGB image into the multi-scale feature extractor layer for feature extraction, obtain multi-layer feature maps, perform fusion processing on each layer feature map to obtain multiple fused features, and input them into the decoder module;

[0108] The loss calculation and model training module 52 is used by the decoder module to set the number of heads of multi-head attention according to the navigation region type, obtain the corresponding prediction feature map for each fused feature through attention calculation, calculate the category loss based on each prediction feature map and the corresponding label, calculate the auxiliary loss based on each prediction feature map and the original label, calculate the overall loss based on the category loss and the auxiliary loss, and obtain the trained semantic segmentation framework when the overall loss meets the requirements.

[0109] The terrain segmentation and navigation planning module 53 is used to acquire unstructured outdoor terrain environment information, input the unstructured outdoor terrain environment information into the semantic segmentation framework, and generate a terrain segmentation result map, which is used for local navigation planning.

[0110] Furthermore, such as Figure 7 As shown, based on the above-mentioned lightweight terrain segmentation method and system based on category attention, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 7 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0111] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a lightweight terrain segmentation program 40 based on category attention, which can be executed by the processor 10 to implement the lightweight terrain segmentation method based on category attention in this application.

[0112] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the lightweight terrain segmentation method based on category attention.

[0113] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0114] In one embodiment, the steps of the lightweight terrain segmentation method based on category attention described above are implemented when the processor 10 executes the lightweight terrain segmentation program 40 based on category attention in the memory 20.

[0115] The present invention also provides a computer-readable storage medium storing a lightweight terrain segmentation program based on category attention, wherein the lightweight terrain segmentation program based on category attention, when executed by a processor, implements the steps of the lightweight terrain segmentation method based on category attention as described above.

[0116] In summary, this invention provides a lightweight terrain segmentation method, system, terminal, and storage medium based on category attention. The method includes: acquiring an RGB original image; inputting the RGB original image into a multi-scale feature extractor layer for feature extraction to obtain multi-layer feature maps; fusing each layer of feature maps to obtain multiple fused features, and inputting them into a decoder module; the decoder module sets the number of heads for multi-head attention according to the navigation region type; obtaining a corresponding predicted feature map for each fused feature through attention calculation; calculating a category loss based on each predicted feature map and its corresponding label; calculating an auxiliary loss based on each predicted feature map and its original label; calculating an overall loss based on the category loss and the auxiliary loss; and obtaining a trained semantic segmentation framework when the overall loss meets the requirements; acquiring unstructured outdoor terrain environment information; inputting the unstructured outdoor terrain environment information into the semantic segmentation framework to generate a terrain segmentation result map, which is used for local navigation planning. This invention establishes global dependencies to more effectively capture global information in images and generate feature maps with global receptive fields. It also decomposes the correlation of channel dimensions, thereby reducing the number of parameters in the self-attention calculation process and improving the performance and efficiency of the semantic segmentation framework in unstructured terrain segmentation. The generated terrain segmentation result map can be applied to local navigation planning.

[0117] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0118] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0119] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A lightweight terrain segmentation method based on category attention, characterized in that, The lightweight terrain segmentation method based on category attention includes: The original RGB image is acquired and input into a multi-scale feature extractor layer for feature extraction to obtain multi-layer feature maps. Each feature map is then fused to obtain multiple fused features, which are then input into the decoder module. The decoder module sets the number of heads for multi-head attention according to the navigation region type, obtains the corresponding predicted feature map for each fused feature through attention calculation, calculates the category loss based on each predicted feature map and the corresponding label, calculates the auxiliary loss based on each predicted feature map and the original label, and calculates the overall loss based on the category loss and the auxiliary loss. When the overall loss meets the requirements, the trained semantic segmentation framework is obtained. Unstructured outdoor terrain environment information is acquired, and the unstructured outdoor terrain environment information is input into a semantic segmentation framework to generate a terrain segmentation result map, which is used for local navigation planning. The multi-scale feature extractor includes a lightweight Transformer module; the lightweight Transformer module includes a lightweight multi-head attention module and a lightweight forward feedback module; the decoder module includes a category attention module, and the category attention module includes a multi-head attention module. The process of acquiring the original RGB image, inputting the original RGB image into a multi-scale feature extractor layer for feature extraction to obtain multi-layer feature maps, fusing each feature map to obtain multiple fused features, and inputting them into the decoder module, specifically includes: The lightweight multi-head attention module divides the sequence features of the original RGB image into K groups. The different sequence features in the K groups are then subjected to attention operations and then stacked together to obtain stacked features. The lightweight forward feedback module divides the stacked features output by the lightweight multi-head attention module into... Grouping, the grouped features are dimensionally expanded and then fed into a convolutional layer, compressed back to their original size, and then... The group features are spliced ​​together to obtain spliced ​​features. The extracted spliced ​​features at each level are then fused according to a preset resolution to obtain fused features.

2. The lightweight terrain segmentation method based on category attention according to claim 1, characterized in that, The lightweight multi-head attention module consists of multiple attention heads, and the attention mechanism formula is as follows: ；（1） in, There are three learnable matrices. for In the corresponding dimensions, Softmax represents the activation function, and T represents the transpose; Multi-head attention calculation is defined as follows: ；（2） ；（3） in, This indicates multi-head attention calculation. Let m represent the i-th attention head, and m represent the number of attention heads. For the first The feature matrix corresponding to each attention head This indicates attention calculation.

3. The lightweight terrain segmentation method based on category attention according to claim 2, characterized in that, The lightweight multi-head attention module divides the sequence features of the original RGB image into K groups. Different sequence features in each of the K groups undergo an attention operation, and then are stacked and concatenated to obtain stacked features. Specifically, this includes: For any input sequence features, The grouping transformation operation is defined as follows: ；（4） ；（5） in, This represents the sequence features before grouping. This represents the sequence features after the attention calculation for the i-th group. Indicates input features, This indicates that a convolution operation is performed on the input features. This indicates that the feature map is expanded along a certain dimension. This represents the attention calculation after grouping. This indicates a feature concatenation operation. This represents the stacked features obtained by stacking features after grouping calculations are completed; The computational complexity is The number of parameters is from ,in, The dimension representing the learning vector. This represents the number of channels.

4. The lightweight terrain segmentation method based on category attention according to claim 3, characterized in that, The lightweight forward feedback module divides the stacked features output by the lightweight multi-head attention module into... Grouping, the grouped features are dimensionally expanded and then fed into a convolutional layer, compressed back to their original size, and then... The group features are concatenated to obtain concatenated features. The extracted concatenated features at each level are then fused according to a preset resolution to obtain fused features, specifically including: The specific definition of the lightweight forward feedback module is as follows: ；（6） ；（7） in, This represents the serialization features of the input. Indicates feature segmentation, This indicates an adjustment of the dimension size. For the defined convolutional layer operations, For the output serialization features, This indicates a feature concatenation operation. This indicates that a lightweight forward feedback operation is being performed. The splicing features at each level are fused according to a preset resolution to obtain the fused features. ; The lightweight forward feedback module reduces the number of parameters from the original Descending to The computational complexity is from Descending to .

5. The lightweight terrain segmentation method based on category attention according to claim 4, characterized in that, The decoder module sets the number of heads for multi-head attention according to the navigation region type. It obtains the corresponding predicted feature map for each fused feature through attention calculation. It calculates the category loss based on each predicted feature map and its corresponding label, and the auxiliary loss based on each predicted feature map and its original label. The overall loss is then calculated based on the category loss and the auxiliary loss. When the overall loss meets the requirements, the trained semantic segmentation framework is obtained, specifically including: The number of heads for multi-head attention is set according to the pre-defined navigation region types. Each head is used to perform multi-head attention based on each fusion feature. Generate a feature map for a category; Adjust the dimensions of each feature map, and obtain the predicted feature map corresponding to each feature map through attention calculation. Extract the corresponding category from each original label and recreate the binary classification label; Calculate the predicted feature map The cross-entropy loss with the original label is used as the basis for calculating the class loss by summing the calculated losses for each class. ; The cross-entropy loss is defined as follows: ；（8） in, The dimension of the feature map. Indicates the number of categories. This represents the output probability corresponding to each category. Indicates the truth value of the label; Category loss The definition is as follows: ；（9） A deep supervised approach is used to calculate the cross-entropy loss between the generated predicted feature map and the original label, thus obtaining an auxiliary loss. ; Then, based on category loss and auxiliary losses Calculate the overall loss : ；（10） When the overall loss When the requirements are met, a well-trained semantic segmentation framework is obtained.

6. A lightweight terrain segmentation system based on category attention, characterized in that, The lightweight terrain segmentation system based on category attention is used to implement the lightweight terrain segmentation method based on category attention as described in any one of claims 1-5, wherein the lightweight terrain segmentation system based on category attention includes: The feature extraction and fusion module is used to acquire the original RGB image, input the original RGB image into the multi-scale feature extractor layer for feature extraction, obtain multi-layer feature maps, fuse each feature map to obtain multiple fused features, and input them into the decoder module; The loss calculation and model training module is used by the decoder module to set the number of heads of multi-head attention according to the navigation region type, obtain the corresponding prediction feature map for each fused feature through attention calculation, calculate the category loss based on each prediction feature map and the corresponding label, calculate the auxiliary loss based on each prediction feature map and the original label, calculate the overall loss based on the category loss and the auxiliary loss, and obtain the trained semantic segmentation framework when the overall loss meets the requirements. The terrain segmentation and navigation planning module is used to acquire unstructured outdoor terrain environment information, input the unstructured outdoor terrain environment information into the semantic segmentation framework, and generate a terrain segmentation result map, which is used for local navigation planning.

7. A terminal, characterized in that, The terminal includes: a memory, a processor, and a lightweight terrain segmentation program based on category attention stored in the memory and executable on the processor. When executed by the processor, the lightweight terrain segmentation program based on category attention implements the steps of the lightweight terrain segmentation method based on category attention as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a lightweight terrain segmentation program based on category attention, which, when executed by a processor, implements the steps of the lightweight terrain segmentation method based on category attention as described in any one of claims 1-5.

Images