Parallel attention convergence high-precision map construction method for assisting visually impaired people

Abstract

The invention provides a parallel attention convergence high-precision map construction method for assisting visually impaired people. The method comprises the following steps: firstly, collecting multi-view and multi-scale data of a traffic environment through multi-camera or multi-sensor equipment; a multi-scale feature fusion network EDS is utilized to integrate data features of different view angles, and semantic understanding of a complex scene is enhanced; then, accurate spatial position attention features are generated through a parallel attention aggregation module PAAM, and the feature aggregation effect and semantic segmentation performance are improved in combination with multi-scale context information; then optimizing multi-scale features by using a multi-resolution feature aggregation module MRFA to ensure the spatial consistency and real-time performance of the high-precision semantic map; and finally, applying the generated semantic map to a real-time navigation system for visually impaired people. According to the method, the semantic understanding and real-time map construction capabilities in a complex traffic environment are remarkably improved, and safe and accurate navigation assistance is provided for travel of visually impaired people.

Description

Technical Field

[0001] The present invention relates to the field of intelligent travel assistance technology, and in particular to a method for constructing a parallel attention-aggregated high-precision map for assisting visually impaired people. Background Art

[0002] With the acceleration of urbanization and the rapid increase in the number of motor vehicles, the traffic environment has become increasingly complex, which poses a major challenge to the travel safety of the visually impaired. The visually impaired mainly rely on external environmental information for navigation and decision-making during travel. However, existing technologies have significant limitations in providing detailed road conditions information in real time. Especially in complex traffic scenarios, existing semantic map construction methods are difficult to effectively handle spatial cognition problems, which makes it easy for the visually impaired to encounter safety hazards when traveling.

[0003] In existing technologies, semantic maps are usually constructed by integrating data from multiple cameras and sensor systems. These systems use environmental data captured from different angles and at different resolutions, and integrate them into maps through algorithms, aiming to provide information about elements such as roads, obstacles, and traffic signs. However, these methods are inefficient in data integration and have difficulty achieving consistent spatial understanding of complex road environments. Especially in environments with more visual impairments, existing perception systems often have difficulty accurately identifying and locating obstacles and key signs on the road, which increases the difficulty of providing accurate navigation assistance for the visually impaired. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a method for constructing a high-precision map by parallel attention aggregation to assist the visually impaired. By combining a multi-resolution feature aggregation module and a parallel attention aggregation module, the construction of a high-precision semantic map is realized, which can effectively respond to the real-time perception needs in complex traffic scenarios, provide real-time travel assistance services for the visually impaired, and help them safely and efficiently cope with various challenges in complex traffic environments. Through the present invention, the visually impaired can significantly improve their perception of the surrounding traffic environment, provide them with accurate road information and navigation services, and improve travel safety.

[0005] The present invention proposes a method for constructing a parallel attention aggregation high-precision map for assisting visually impaired people, comprising the following steps:

[0006] Step S1: Collect traffic environment data of different perspectives and scales through multiple cameras or multiple sensor devices, pre-process the collected data, and use a convolutional neural network to extract features from each data;

[0007] Step S2: Use the multi-scale feature fusion network EDS to perform multi-scale feature fusion processing on the extracted features, use deep separable convolution to perform spatial and channel optimization processing on the extracted multi-scale features, first perform independent convolution on each channel, then complete channel fusion through point-by-point convolution, and output feature maps;

[0008] Step S3: Perform multi-path convolution operations on the feature map through the parallel attention aggregation module PAAM, use the spatial fusion mechanism to aggregate information of different spatial dimensions, and fuse the output results of the 1×1 and 3×3 convolution paths through the matrix dot product operation to generate the first spatial attention map, then perform global average pooling on the output results of the 3×3 convolution path, and adjust the dimension of the output of the 1×1 convolution path, generate the second spatial attention map through the channel joint activation mechanism, and finally fuse the two spatial attention maps and output;

[0009] Step S4: The multi-resolution feature aggregation module MRFA performs convolution operations on the features of different resolutions output by PAAM based on the elastic perception field convolution to extract multi-scale features, and the extracted multi-scale features are fused through 1x1 convolution. The fused feature map is passed to the decoding network. Through layer-by-layer upsampling and skip connections, the spatial resolution is gradually restored and the shallow details and deep semantic features are fused. The classification head is used to classify the target category pixel by pixel, and finally a high-definition semantic map is generated;

[0010] Step S5: Apply the generated high-precision semantic map to the real-time navigation assistance system for the visually impaired, so as to provide the visually impaired users with detailed traffic environment perception information.

[0011] Furthermore, the EDS includes an input layer, a preprocessing layer, a multi-scale feature extraction layer, a deep separable convolution layer, a feature fusion module and an output layer, wherein the input layer receives raw data from multiple cameras and sensors; the preprocessing layer performs standardization on the input data; the multi-scale feature extraction layer extracts multi-scale features of the data through a multi-layer convolutional neural network structure, uses convolution kernels of different sizes to capture spatial features, and enhances the model nonlinearity through the activation function ReLU; the deep separable convolution decomposes the traditional convolution into deep convolution and point-by-point convolution, first performs independent convolution on each channel, and then completes channel fusion through point-by-point convolution; the feature fusion module adopts a Transformer-based multimodal fusion method, models the global correlation between modalities through a multi-head self-attention mechanism, uses a feedforward neural network to perform nonlinear mapping on the self-attention output, and uses a cross-modal attention mechanism, with the main modal features as queries and the sub-modal features as keys and values, to strengthen the information interaction and complementarity between modalities, and gradually realizes deep fusion through multi-layer Transformer modules; the output layer directly outputs feature representations for constructing high-precision semantic maps, namely feature maps.

[0012] Furthermore, the workflow of the PAAM is:

[0013] First, the input feature map obtained from EDS processing is pooled in the X and Y directions. Then, a 1×1 convolution is applied to the feature map after the pooling operation to reduce the dimension of the feature channel. Then, two tensors are generated for the outputs of the 1×1 and 3×3 convolution paths, one containing the global spatial information of the 1×1 convolution output and the other containing the local spatial information captured by the 3×3 convolution path. The output of the 1×1 path is encoded by global average pooling to generate a global feature matrix. At the same time, the output of the 3×3 path is appropriately adjusted to ensure that its dimension matches the output of the 1×1 path. Before spatially fusing the outputs of the 1×1 and 3×3 convolutional paths, global average pooling is used to process the local features extracted by the 3×3 path. Then, the output results of the 1×1 and 3×3 paths are fused through a matrix dot product operation to generate the first spatial attention map. The output results of the 3×3 path are then globally average pooled, and the output of the 1×1 path is dimensionally adjusted before entering the channel joint activation mechanism. The result of global average pooling is then combined with the adjusted 1×1 path output to generate the second spatial attention map through the channel joint activation mechanism. Finally, the two spatial attention maps are fused.

[0014] Furthermore, the step S4 further comprises:

[0015] Step S41: The multi-resolution feature aggregation module MRFA performs convolution operation on the features of different resolutions output by the PAAM through elastic perception field convolution to realize the extraction of multi-scale features;

[0016] Step S42: The multi-scale features extracted in step S41 are fused through 1x1 convolution, and the fused feature map is output;

[0017] Step S43: The fused feature map is passed into the decoding network. Through layer-by-layer upsampling and skip connections, the spatial resolution is gradually restored and shallow details and deep semantic features are fused. During the decoding process, the classification head is used to classify the target category pixel by pixel, and finally a high-definition semantic map is generated.

[0018] The beneficial technical effects of the present invention are as follows: the present invention introduces an innovative multi-scale feature fusion network (EDS) and a parallel attention aggregation module (PAAM). The EDS network effectively improves the processing speed and feature extraction accuracy by combining deep separable convolution and point-by-point convolution, as well as adaptive feature fusion technology, and optimizes the model's responsiveness to complex traffic scenarios. The PAAM module uses multiple parallel attention sub-networks to independently process information of different scales and channels, and significantly improves the feature aggregation effect and semantic segmentation accuracy by integrating global and local information. The scheme of the present invention not only improves the perception ability and computing efficiency of the navigation system, but also greatly enhances the travel safety of the visually impaired by providing accurate real-time road information and optimized data processing accuracy, and provides accurate environmental perception and safe navigation services for the visually impaired in complex traffic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a structural schematic diagram of a parallel attention aggregation high-precision map construction system for assisting visually impaired people provided by an embodiment of the present invention;

[0021] Figure 2 It is a flow chart of a method for constructing a parallel attention-aggregated high-precision map for assisting visually impaired people provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of the structure of a multi-scale feature fusion (EDS) network provided in an embodiment of the present invention;

[0023] Figure 4 A flow chart of generating spatial position attention features by a parallel attention aggregation module (PAAM) provided in an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of the layout of a real-time traffic environment data collection device provided by an embodiment of the present invention;

[0025] Figure 6 A feature optimization flow chart of a multi-resolution feature aggregation module (MRFA) provided in an embodiment of the present invention;

[0026] Figure 7 A schematic diagram of the application of a high-precision semantic map provided by an embodiment of the present invention in an urban road scenario. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] like Figure 1 As shown, the parallel attention aggregation high-precision semantic map construction system for assisting the visually impaired in traveling proposed in the present invention includes a data acquisition layer, a feature processing layer and a semantic map generation layer. The data acquisition layer collects traffic environment data and performs preprocessing. The feature processing layer extracts and processes the collected data to generate fused features for semantic map construction. The semantic map generation layer generates a high-precision semantic map based on the fused features and is used for navigation assistance for the visually impaired to help them travel safely and accurately in complex traffic environments.

[0029] Figure 2 The present invention provides a flow chart of a method for constructing a parallel attention aggregation high-precision map for assisting visually impaired people, and the method comprises the following steps:

[0030] Step S1: Traffic environment data of different perspectives and scales are collected through multi-camera or multi-sensor equipment, wherein the equipment includes high-definition cameras, laser radars, millimeter-wave radars, infrared sensors and GPS receivers, which can capture detailed information of key traffic elements such as road signs, traffic signals, pedestrians, vehicles and obstacles, and ensure that these devices work synchronously to achieve all-round and seamless coverage of traffic scenes. The collected data include high-resolution images and video streams, point cloud data, heat maps, geographic coordinate information, and traffic flow statistics. The collected data has high resolution, diversity and multi-scale features, and can provide high-quality original input data for subsequent processing; then the collected data is preprocessed, and a convolutional neural network is used to extract features from each data;

[0031] Step S2: Use a multi-scale feature fusion network (EDS) to perform multi-scale feature fusion processing on the collected data. The feature fusion is completed through the multi-level structure of the deep convolutional neural network to ensure that semantic information such as road boundaries, lane lines, pedestrians and vehicles in the traffic environment can be accurately extracted at different scales. The network effectively captures semantic information of different scales through multi-layer convolution and pooling operations; deep separable convolution is used to perform spatial and channel optimization processing on the extracted multi-scale features, first performing independent convolution on each channel, and then completing channel fusion through point-by-point convolution to output a feature map;

[0032] Step S3: Generate accurate attention features of spatial positions through the parallel attention aggregation module (PAAM). The module introduces multiple parallel sub-network blocks to perform feature aggregation and attention calculation in the spatial dimension and channel dimension respectively, which can effectively capture the key elements and their interactive relationships in the traffic environment. At the same time, the PAAM module integrates multi-scale context information by fusing the output results of 1×1 and 3×3 convolution paths, improves the global perception ability of the network, and significantly improves its feature aggregation and semantic segmentation performance in complex traffic scenarios;

[0033] Step S4: The features of different resolutions are further optimized through the multi-resolution feature aggregation module (MRFA). This module is based on the elastic perception field convolution technology and can dynamically adjust the perception field size to ensure accurate spatial information at different resolutions. It also uses 1x1 convolution to fuse global features, thereby improving the spatial consistency and real-time performance of high-precision semantic maps. The MRFA module uses an enhanced feature fusion mechanism to ensure stable performance and fast response capabilities when processing complex traffic scenes. The fused feature map is passed to the decoding network. Through layer-by-layer upsampling and jump connections, the spatial resolution is gradually restored and shallow details and deep semantic features are fused. The classification head is used to classify the target category pixel by pixel, and finally a high-definition semantic map is generated.

[0034] Step S5: Apply the generated high-precision semantic map to the real-time navigation assistance system for the visually impaired. The system can be dynamically adjusted according to the real-time traffic environment data to provide the visually impaired users with detailed traffic environment perception information, including road signs, traffic signals, pedestrians, vehicles and surrounding obstacles, etc., to ensure that the visually impaired users can safely and efficiently cope with the complex traffic environment.

[0035] The step S1 further comprises:

[0036] Step S11: Configuration and initialization of sensor equipment: accurately configure the sensor equipment, and manage the data collection process of each sensor through a dedicated hardware interface and software protocol to ensure efficient data collection;

[0037] Step S12: multi-sensor data preprocessing and integration, the preprocessing includes data filtering, denoising and synchronization, and then the convolutional neural network (CNN) is used to extract features from each sensor data, and then the fusion algorithm is used to integrate each feature to form a unified multi-level data representation for subsequent semantic analysis;

[0038] Step S13: Data synchronization and calibration technology. To ensure the timing consistency and spatial accuracy of data collection, all sensor devices undergo strict time synchronization and spatial calibration. Time synchronization uses the Network Time Protocol (NTP) to ensure that the time tags between devices are consistent, while spatial calibration uses calibration software to accurately correct the viewing angle and position of each sensor to ensure spatial alignment of the data.

[0039] The step S2 further comprises:

[0040] Step S21: using a multi-layer convolutional neural network (CNN) to apply convolution kernels of different sizes to extract multi-scale spatial features of the preprocessed data, and enhancing nonlinearity through an activation function ReLU;

[0041] Step S22: using depthwise separable convolution to perform spatial and channel optimization processing on the extracted multi-scale features, first performing independent convolution on each channel, and then completing channel fusion through point-by-point convolution;

[0042] Step S23: adopt a hybrid activation function strategy and select appropriate activation functions at different levels to enhance the network's expressive power and nonlinear characteristics. The optimized features are integrated through a feature fusion module, and finally output as feature representations for building a high-precision semantic map, i.e., feature maps.

[0043] Traditional convolution operations simultaneously process the interaction information between channels and spatial feature extraction, while the depthwise separable convolution significantly improves the computational efficiency by separating these two processes. In order to solve the problem of insufficient feature extraction of traditional two-layer depthwise separable convolution in complex road environments, the present invention proposes a multi-scale feature fusion (EDS) network to process the collected multi-view and multi-scale data. The EDS network significantly improves the model's ability to accurately recognize various semantic information (such as road boundaries, lane lines, etc.) by fusing features of different scales, and can ensure the computational efficiency of data processing. Figure 3 As shown, the network consists of the following parts:

[0044] Input layer: Receives raw data from multiple cameras and sensors and provides basic input for deep network processing.

[0045] Preprocessing layer: Standardizes input data to optimize data quality and prepare for deep processing.

[0046] Multi-scale feature extraction layer: The multi-scale features of the data are extracted through a multi-layer convolutional neural network (CNN) structure, convolution kernels of different sizes are used to capture spatial features, and the activation function ReLU is used to enhance the nonlinearity of the model.

[0047] Depthwise separable convolution layer: Depthwise separable convolution is used to optimize computational efficiency. This technology decomposes traditional convolution into depthwise convolution and pointwise convolution, specifically dealing with spatial relationships and channel fusion. Depthwise separable convolution decomposes traditional convolution into depthwise convolution and pointwise convolution, first performing independent convolution on each channel, and then completing channel fusion through pointwise convolution.

[0048] Feature fusion module: A Transformer-based multimodal fusion method is used to model the global correlation between modalities through a multi-head self-attention mechanism, and dynamically adjust feature weights to highlight key task-related information. The fusion network uses a feedforward neural network to perform nonlinear mapping on the self-attention output to further optimize feature expression. Using a cross-modal attention mechanism, with the main modal features as queries and the sub-modal features as keys and values, information interaction and complementarity between modalities are strengthened. Deep fusion is gradually achieved through multi-layer Transformer modules to generate high-quality joint features, providing accurate and robust semantic expression support for semantic map construction.

[0049] Output layer: Directly outputs feature representations used to build high-precision semantic maps, namely feature maps. These features are used in real-time navigation assistance systems to provide road information, traffic signals, and obstacle perception to ensure accurate and reliable navigation assistance.

[0050] The above modules are closely connected according to the data flow and processing logic to form a continuous processing flow. In the initial stage, the input layer first receives the raw data collected by multiple cameras and sensors, and then passes it to the preprocessing layer, where the data is preliminarily processed through steps such as standardization and denoising. The processed data flows into the multi-scale feature extraction layer, where different sizes of convolution kernels are applied through multi-layer convolutional neural networks (CNNs) to extract multi-scale features from fine-grained to coarse-grained. Subsequently, the depthwise separable convolution layer further optimizes these features in space and channels, and improves computational efficiency by separating depthwise convolution and pointwise convolution. The optimized features are integrated through the feature fusion module, which uses feature pyramid networks or adaptive techniques to fuse features at different levels to ensure consistency and integrity. Finally, the output layer summarizes this series of refined features to generate detailed road and obstacle information for real-time navigation assistance, directly supporting the construction of high-precision semantic maps. Through this orderly module connection and data processing method, the EDS network can effectively process and analyze multi-source data in complex scenarios and provide accurate environmental perception output.

[0051] The step S3 further comprises:

[0052] Step S31: Perform multi-path convolution operation on the input feature map. The PAAM module performs attention calculation on different dimensions of the feature map, uses 1×1 and 3×3 convolution kernels to extract global and local features respectively, and generates attention features with precise spatial position information;

[0053] Step S32: Generate two tensors for the outputs of the 1×1 and 3×3 convolution paths respectively, one containing the global spatial information of the 1×1 convolution output, and the other containing the local spatial information captured by the 3×3 convolution path, encode the output of the 1×1 path through global average pooling to generate a global feature matrix, and then use global average pooling to process the local features extracted by the 3×3 path;

[0054] Step S33: The output results of the 1×1 and 3×3 paths are fused through a matrix dot product operation to generate the first spatial attention map, and the results of the 3×3 path output are globally averaged pooled to generate global spatial information, and the output of the 1×1 path is dimensionally adjusted before entering the channel joint activation mechanism. The result of the global average pooling is combined with the adjusted 1×1 path output, and the second spatial attention map is generated through the channel joint activation mechanism. Finally, the two spatial attention maps are fused and the fused feature map is output.

[0055] The workflow of the Parallel Attention Aggregation Module (PAAM) is as follows Figure 4 As shown in the figure, the parallel attention aggregation module includes an input layer, a parallel sub-unit network, a feature weighting mechanism, a multi-scale feature processing unit, a feature fusion module, and an output layer. The parallel sub-network blocks capture the cross-dimensional interactions between data and effectively generate more accurate spatial position attention features. The PAAM module can integrate multi-scale contextual information, enhance the aggregation effect of features and the accuracy of semantic segmentation, and is particularly suitable for road and obstacle perception in complex traffic scenarios. The workflow of PAAM is:

[0056] The PAAM network consists of three parallel 1×1 convolution paths and one 3×3 convolution path. First, the input feature map obtained from EDS processing is Pooling operations are performed in the X and Y directions, where C represents the number of input channels, H and W represent the spatial dimensions of the input features, respectively. For a given input feature map , PAAM will divide C into G sub-features along the channel dimension to learn different semantics, where the grouping style is represented as ,in , in order to enhance the generalization of the module, the value of G is stipulated not to exceed C, and it is assumed that the learned attention weights will be used to enhance the feature expression of key areas in each sub-feature; then 1×1 convolution is applied to the feature map after pooling operation in the X and Y directions to reduce the dimension of the feature channel. The use of 1×1 convolution can effectively reduce the computational complexity and is particularly suitable for large-scale datasets, such as the NuScenes dataset. One of the three 1×1 convolution paths is combined with 1D global average pooling to capture global channel information to provide contextual information; the remaining 1×1 convolutions are mainly used to adjust the channel dimension and extract deep features. The 3×3 convolution path captures local spatial features through a larger receptive field, further supplementing and enriching the spatial expression ability of the network, thereby enhancing the overall feature extraction ability of the network;

[0057] After performing multi-path convolution parallel feature extraction, a spatial fusion mechanism is used to aggregate information of different spatial dimensions. First, two tensors are generated for the outputs of the 1×1 and 3×3 convolution paths, one containing the global spatial information of the 1×1 convolution output and the other containing the local spatial information captured by the 3×3 convolution path. The output of the 1×1 path is encoded by global average pooling to generate a global feature matrix. At the same time, the output of the 3×3 path is appropriately adjusted to ensure that its dimension matches the output of the 1×1 path. Before the outputs of the 1×1 and 3×3 convolution paths are spatially fused, global average pooling is used to process the local features extracted by the 3×3 path, simplifying the local spatial information into more easily processable global information so that it can be more effectively combined with the global channel information extracted by the 1×1 path. The formula for the global average pooling operation is:

[0058]

[0059] This formula helps capture the global spatial information in the feature map, where i, j represents the coordinates of the feature map. Represents the input characteristics of the cth channel.

[0060] The output results of the 1×1 and 3×3 paths are then fused through a matrix dot product operation to generate the first spatial attention map, which captures spatial features of different scales. Furthermore, the output results of the 3×3 path are globally averaged pooled to extract global spatial information, while the output of the 1×1 path is dimensionally adjusted before entering the channel joint activation mechanism. Subsequently, the result of global average pooling is combined with the adjusted 1×1 path output to generate a second spatial attention map through the channel joint activation mechanism, which retains the precise spatial position information of the input features while enhancing the expression of global spatial information. Finally, PAAM fuses the two spatial attention maps to generate an output feature map that is not only the same size as the original input feature map, but also has the ability to efficiently aggregate multi-scale spatial information. The channel joint activation mechanism includes: fusing the global feature vector with the adjusted 1×1 path output feature, and the fusion methods include element-by-element addition fusion, element-by-element multiplication fusion and splicing fusion, and then processing the fused feature vector. First, the feature dimension is reduced through a fully connected layer, and then the feature expression ability is enhanced through a nonlinear activation function (such as ReLU). Then, the original dimension is restored through another fully connected layer, and the output value is mapped to a specific range (usually between 0 and 1) using an activation function (such as Sigmoid) to generate an attention weight for each channel. Finally, the generated attention weight is applied to the input feature. Specifically, each channel feature map of the input feature is multiplied by the corresponding attention weight to achieve dynamic weighted adjustment at the channel level. This can strengthen the network's attention to important channels while suppressing irrelevant or minor channels.

[0061] The step S4 further comprises:

[0062] Step S41: The multi-resolution feature aggregation module MRFA performs convolution operation on the features of different resolutions output by the PAAM through elastic perception field convolution to realize the extraction of multi-scale features;

[0063] Step S42: The multi-scale features extracted in step S41 are fused through 1x1 convolution, and the fused feature map is output;

[0064] Step S43: The fused feature map is passed to the decoding network. Through layer-by-layer upsampling and skip connections, the spatial resolution is gradually restored and shallow details and deep semantic features are fused. During the decoding process, the classification head is used to classify the target category (such as roads, obstacles, traffic signs, etc.) pixel by pixel, and finally a high-definition semantic map is generated.

[0065] The present invention utilizes a multi-resolution feature aggregation module (MRFA) to further optimize and fuse features of different resolutions, ensuring that the data has good spatial consistency and real-time response capability when generating a high-precision semantic map. Figure 6The feature optimization process of the MRFA module is shown. The specific implementation steps are as follows:

[0066] Step 1: After inputting the image feature map processed by PAAM, the elastic perception field convolution module is used to perform convolution operation. The convolution process is performed by setting different elastic coefficients. , dynamically adjust the size of the perception field to achieve multi-scale feature extraction, ensuring that a wider range of feature information is obtained at the same resolution;

[0067] Step 2: The multi-scale features extracted in step 1 are fused through 1x1 convolution to combine global information with local details. The specific expression is as follows:

[0068]

[0069]

[0070] in, is the input feature map, is the convolution kernel weight, is the elastic modulus, Defines the size of the convolution kernel. P(i,j) is the dot product result of the local area of ​​the input feature map and the convolution kernel. is the output feature map.

[0071] Step 3: The fused feature map is passed to the decoding network. Through layer-by-layer upsampling and skip connections, the spatial resolution is gradually restored and shallow details and deep semantic features are integrated. During the decoding process, the classification head is used to classify the target categories (such as roads, obstacles, traffic signs, etc.) pixel by pixel, and finally a high-definition semantic map is generated. This map combines multi-scale contextual information and fine-grained semantic annotations to meet the high-precision requirements of autonomous driving scenarios for environmental perception.

[0072] The generated high-precision semantic map is applied to the navigation assistance system for the visually impaired, such as Figure 7 As shown in Figure 1, the semantic map presents accurate information about roads, traffic signals, and surrounding obstacles to visually impaired users. Through this map, visually impaired users can obtain timely information about changes in the traffic environment to ensure safe and effective navigation.

[0073] The step S5 further comprises:

[0074] Step S51: The system combines the high-precision semantic map with the real-time collected traffic data, and regularly updates the map information to reflect the current road conditions;

[0075] Step S52: Integrate various types of detailed traffic information in multiple dimensions and process the traffic information in layers, including road signs, obstacles, traffic signals, etc.;

[0076] Step S53: The system dynamically generates a reminder based on the obstacle information in the semantic map and the obstacle position detected by the sensor in real time;

[0077] Step S54: providing a real-time update function through periodic data collection and instant feature analysis to ensure that the visually impaired user obtains the latest road and surrounding environment information;

[0078] Step S55: Adjust the user's navigation path in real time to avoid obstacles and complex road sections.

[0079] This method is also particularly suitable for processing complex traffic scenarios, such as urban congested roads and signal-intensive areas. Through the PAAM and MRFA modules, the system can maintain high semantic recognition capabilities and accuracy when processing these complex scenarios. Figure 7 The semantic segmentation results of the present invention when processing complex traffic scenes are demonstrated.

[0080] In the process of generating high-precision semantic maps, data from different perspectives obtained by multiple cameras or multi-sensor devices complement each other to achieve multi-perspective data fusion, ensuring the integrity and accuracy of map data. Figure 5 An example of a multi-sensor data collection screen is shown.

[0081] The semantic map generated by the present invention can also be used in real time in a variety of terminal devices, and users can obtain current road condition information and navigate at any time through mobile devices.

[0082] Through innovative multi-scale feature fusion networks, parallel attention aggregation modules and multi-resolution feature aggregation modules, the present invention significantly improves the efficiency and accuracy of constructing high-precision semantic maps in complex traffic scenarios, providing safer and more accurate travel assistance services for the visually impaired.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a high-precision map with parallel attention aggregation to assist visually impaired people, characterized in that: The method comprises: Step S1: Collect traffic environment data of different perspectives and scales through multiple cameras or multiple sensor devices, pre-process the collected data, and use a convolutional neural network to extract features from each data; Step S2: Use the multi-scale feature fusion network EDS to perform multi-scale feature fusion processing on the extracted features, use deep separable convolution to perform spatial and channel optimization processing on the extracted multi-scale features, first perform independent convolution on each channel, then complete channel fusion through point-by-point convolution, and output feature maps; Step S3: Perform multi-path convolution operations on the feature map through the parallel attention aggregation module PAAM, use the spatial fusion mechanism to aggregate information of different spatial dimensions, and fuse the output results of the 1×1 and 3×3 paths through the matrix dot product operation to generate the first spatial attention map, then perform global average pooling on the output results of the 3×3 path, and adjust the dimension of the output of the 1×1 path, generate the second spatial attention map through the channel joint activation mechanism, and finally fuse the two spatial attention maps and output; Step S4: The multi-resolution feature aggregation module MRFA performs convolution operations on the features of different resolutions output by PAAM based on the elastic perception field convolution to extract multi-scale features, and the extracted multi-scale features are fused through 1x1 convolution. The fused feature map is passed to the decoding network. Through layer-by-layer upsampling and skip connections, the spatial resolution is gradually restored and the shallow details and deep semantic features are fused. The classification head is used to classify the target category pixel by pixel, and finally a high-definition semantic map is generated; Step S5: Apply the generated high-precision semantic map to the real-time navigation assistance system for the visually impaired, so as to provide the visually impaired users with detailed traffic environment perception information.

2. The method according to claim 1, characterized in that The EDS includes an input layer, a preprocessing layer, a multi-scale feature extraction layer, a deep separable convolution layer, a feature fusion module and an output layer. The input layer receives raw data from multiple cameras and sensors; the preprocessing layer performs standardization on the input data; the multi-scale feature extraction layer extracts multi-scale features of the data through a multi-layer convolutional neural network structure, uses convolution kernels of different sizes to capture spatial features, and enhances the model nonlinearity through the activation function ReLU; the deep separable convolution decomposes the traditional convolution into deep convolution and point-by-point convolution, first performs independent convolution on each channel, and then completes channel fusion through point-by-point convolution; the feature fusion module adopts a Transformer-based multimodal fusion method, models the global association between modalities through a multi-head self-attention mechanism, uses a feedforward neural network to perform nonlinear mapping on the self-attention output, uses a cross-modal attention mechanism, takes the main modal features as queries, and the sub-modal features as keys and values, strengthens the information interaction and complementarity between modalities, and gradually realizes deep fusion through multi-layer Transformer modules; the output layer directly outputs the feature representation used to construct a high-precision semantic map, that is, a feature map.

3. The method according to claim 1, characterized in that The workflow of PAAM is: First, the input feature map obtained from EDS processing is pooled in the X and Y directions. Then, a 1×1 convolution is applied to the feature map after the pooling operation to reduce the dimension of the feature channel. Then, two tensors are generated for the outputs of the 1×1 and 3×3 convolution paths, one containing the global spatial information of the 1×1 convolution output and the other containing the local spatial information captured by the 3×3 convolution path. The output of the 1×1 path is encoded by global average pooling to generate a global feature matrix. At the same time, the output of the 3×3 path is appropriately adjusted to ensure that its dimension matches the output of the 1×1 path. Before spatially fusing the outputs of the 1×1 and 3×3 convolutional paths, global average pooling is used to process the local features extracted by the 3×3 path. Then, the output results of the 1×1 and 3×3 paths are fused through a matrix dot product operation to generate the first spatial attention map. The output results of the 3×3 path are then globally average pooled, and the output of the 1×1 path is dimensionally adjusted before entering the channel joint activation mechanism. The result of global average pooling is then combined with the adjusted 1×1 path output to generate the second spatial attention map through the channel joint activation mechanism. Finally, the two spatial attention maps are fused.

4. The method according to claim 1, characterized in that: The step S4 further comprises: Step S41: The multi-resolution feature aggregation module MRFA performs convolution operations on the features of different resolutions output by the PAAM through elastic perception field convolution to achieve multi-scale feature extraction; Step S42: The multi-scale features extracted in step S41 are fused through 1x1 convolution, and the fused feature map is output; Step S43: The fused feature map is passed into the decoding network. Through layer-by-layer upsampling and skip connections, the spatial resolution is gradually restored and shallow details and deep semantic features are fused. During the decoding process, the classification head is used to classify the target category pixel by pixel, and finally a high-definition semantic map is generated.

Images