A method, device, storage medium, and program product for understanding road topology.

Abstract

This application relates to a road topology understanding method, device, storage medium, and program product, belonging to the field of road topology technology. The method includes: acquiring a traffic scene map of a target vehicle during its journey; determining corresponding multi-scale prior features based on the traffic scene map, wherein the multi-scale prior features are image features aligned to different scales of a clean image; fusing the multi-scale prior features with degradation features in the traffic scene map to obtain fused features; performing fusion optimization processing on the fused features to obtain an optimized scene map corresponding to the traffic scene map, wherein the optimized scene map is a scene map whose clarity has been optimized from the traffic scene map; and generating at least one lane centerline based on the optimized scene map using a preset Bezier deformation attention. This application improves the vehicle's road topology understanding capability.

Description

Technical Field

[0001] This application relates to the field of road topology technology, specifically to a road topology understanding method, device, storage medium, and program product. Background Technology

[0002] Road topology refers to a topological structure that describes the network structure of roads and their interconnections. For automobiles, especially autonomous vehicles, understanding road topology means being able to grasp their own position within the road network, as well as the layout and connections of their surroundings. This understanding is crucial for functions such as navigation, route planning, and autonomous driving, helping vehicles accurately determine driving styles, choose optimal routes, and avoid getting lost or causing traffic accidents in complex road environments. With the continuous development of artificial intelligence technology, autonomous driving technology is rapidly moving towards widespread adoption and high intelligence, thus placing increasingly higher demands on vehicles' road topology understanding capabilities.

[0003] Currently, in the field of autonomous driving, the real-time images of the vehicle's surroundings collected by the vehicle are typically subjected to general image processing, such as preprocessing, feature extraction, object detection and recognition, and scene understanding, to help the vehicle understand the road topology. However, in adverse weather conditions such as rain, fog, or snow, or when lighting is poor, the image quality or clarity of the real-time images of the vehicle's surroundings are poor. This reduces the vehicle's ability to understand the surrounding road topology, resulting in poor road topology understanding capabilities. Summary of the Invention

[0004] To improve the road topology understanding capability of automobiles, this application provides a road topology understanding method, device, system, storage medium, and program product.

[0005] The first aspect of this application provides a road topology understanding method, specifically including:

[0006] Obtain a traffic scene map of the target car during its driving process;

[0007] Based on the traffic scene map, corresponding multi-scale prior features are determined. The multi-scale prior features are image features of different scales aligned to a clean image.

[0008] The multi-scale prior features and the degradation features in the traffic scene map are fused to obtain the fused features;

[0009] The fused features are subjected to fusion optimization processing to obtain an optimized scene map corresponding to the traffic scene map. The optimized scene map is a scene map with improved clarity from the traffic scene map.

[0010] Based on the optimized scene map, at least one lane centerline is generated through a preset Bezier deformation attention.

[0011] By employing the aforementioned technical solution, after acquiring a traffic scene image of the target vehicle during its journey, image features of different scales aligned to a clean image are extracted from the traffic scene image. This yields high-quality prior image features corresponding to the traffic scene image, namely multi-scale prior features, which provide useful prior knowledge for subsequent image quality restoration and optimization. Furthermore, the multi-scale prior features are fused with degradation features. Guided by prior knowledge, this process transforms degradation regions in the traffic scene image into clean regions, thereby optimizing blurry or low-quality areas. The fused features are then subjected to fusion optimization processing to alleviate the structural warping and texture distortion problems caused by directly fusing high-quality prior features and low-quality degradation features. This results in an optimized scene image with better clarity. Finally, based on this optimized scene image, lane centerlines are generated more accurately, thereby improving the vehicle's road topology understanding capabilities.

[0012] In one possible design, determining the corresponding multi-scale prior features based on the traffic scene map specifically includes:

[0013] Initial image features are extracted from the traffic scene map through a preset convolutional layer, and the initial image features are mapped to the pre-trained VQVAE codebook to obtain a single-scale mapping result;

[0014] Input the mapping results of all current scales into the preset vector autoregressive model to obtain the mapping results of the next scale, and repeat the step of inputting the mapping results of all current scales into the preset vector autoregressive model to obtain the mapping results at a preset number of scales.

[0015] Decode all the mapping results to obtain the multi-scale prior features corresponding to the traffic scene map.

[0016] In one possible design, fusing the multi-scale prior features and the degenerate features in the traffic scene map to obtain the fused features specifically includes:

[0017] The traffic scene map is input into a preset residual block to obtain degradation features;

[0018] The degraded features are input into the weight predictor to obtain the processed features, and the processed features are mapped through a preset projection convolutional layer to obtain the mapped weights.

[0019] The multi-scale prior features are weighted using the mapped weights and then transformed using a preset lightweight projection head to obtain the processed prior features.

[0020] The result of the previous fusion and optimization of the degradation features and the multi-scale prior features in the traffic scene map is determined as the previous optimization result. The processed prior features and the previous optimization result are substituted into the preset weight calculation formula to obtain the first weight and the second weight.

[0021] Based on the first weight and the second weight, the processed prior features and the previous optimization result are weighted and summed to obtain the fused features.

[0022] In one possible design, the weight calculation formula is: ;

[0023] In the formula, concat represents the concatenation operation, RSTBs represents a series of residual Swin-Transformer blocks, conv represents a convolutional layer, and softmax represents a softmax activation layer. Indicates the first weight. Indicates the second weight. This indicates the result of the previous optimization. This represents the prior features after processing.

[0024] In one possible design, the step of performing fusion optimization processing on the fused features to obtain an optimized scene map corresponding to the traffic scene map specifically includes:

[0025] The previous optimization result and the fused features are then concatenated to obtain the concatenated result;

[0026] The previous optimization result is subjected to a projection transformation to obtain a first transformation result, and the splicing result is subjected to a projection transformation to obtain a second transformation result;

[0027] The fused features are subjected to projection transformation to obtain a third transformation result and a fourth transformation result. Based on the first transformation result, the second transformation result, the third transformation result, and the fourth transformation result, the fused features are subjected to fusion optimization processing through a two-layer attention mechanism to obtain an optimized result.

[0028] Repeat the step of inputting the traffic scene map into the preset residual block to obtain the optimized scene map corresponding to the traffic scene map.

[0029] In one possible design, generating at least one lane centerline based on the optimized scene map using a preset Bezier deformation attention mechanism specifically includes:

[0030] The optimized scene map is converted into perspective view features using a preset feature extraction function;

[0031] The perspective view features are aggregated and projected onto the BEV feature map using a preset projection function to obtain the projection result.

[0032] The projection result is input into the Transformer decoder based on Bézier deformation attention to obtain the final set of Bézier control points generated by the last layer of the Transformer decoder;

[0033] Based on the final set of Bezier control points, at least one lane centerline corresponding to the optimized scene map is generated.

[0034] In one possible design, inputting the projection result into a Transformer decoder based on Bézier deformation attention to obtain the final set of Bézier control points generated by the last layer of the Transformer decoder specifically includes:

[0035] The initial set of Bessel control points for the first query embedding of the Transformer decoder single layer is generated by using a multilayer perceptron and a sigmoid function.

[0036] Based on the first query embedding and the position embedding of the same layer of the Transformer decoder, the second query embedding of the same layer of the Bezier deformation attention is determined;

[0037] Based on the projection result, the second query embedding, and the initial set of Bézier control points, the output result is obtained using Bézier deformation attention;

[0038] Based on the output and the position embedding, self-attention calculation is performed to obtain the calculation result;

[0039] The calculation result is used as the first query embedding of the next layer of the Transformer decoder, and the step of generating the initial set of Bezier control points for the first query embedding of a single layer of the Transformer decoder through a multilayer perceptron and a sigmoid function is repeated to obtain the final set of Bezier control points generated by the last layer of the Transformer decoder.

[0040] A second aspect of this application provides an electronic device, specifically comprising:

[0041] A processor, a memory, and a computer program stored in the memory and capable of running on the processor, the processor being configured to load and execute the computer program stored in the memory to cause the electronic device to perform the method as described in any one of the first aspects.

[0042] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when loaded and executed by a processor, performs the steps of the method described in any one of the first aspects.

[0043] A fourth aspect of this application provides a computer program product including a computer program that, when executed by a processor, implements the method as described in any one of the first aspects.

[0044] A fifth aspect of this application provides a road topology understanding system, comprising:

[0045] The information acquisition module is used to acquire traffic scene images of the target vehicle during its driving process;

[0046] The prior determination module is used to determine the corresponding multi-scale prior features based on the traffic scene map. The multi-scale prior features are image features of different scales aligned to a clean image.

[0047] The feature fusion module is used to fuse the multi-scale prior features and the degradation features in the traffic scene map to obtain the fused features;

[0048] The image optimization module is used to perform fusion optimization processing on the fused features to obtain an optimized scene image corresponding to the traffic scene image. The optimized scene image is a scene image after the traffic scene image has been optimized in terms of clarity.

[0049] The topology understanding module is used to generate at least one lane centerline based on the optimized scene graph and through a preset Bezier deformation attention.

[0050] By adopting the above technical solution, after the information acquisition module obtains the traffic scene map of the target car during its driving process, the prior determination module determines the corresponding multi-scale prior features based on the traffic scene map. Then, the feature fusion module fuses the multi-scale prior features and the degradation features in the traffic scene map to obtain the fused features. Next, the image optimization module determines the optimized scene map corresponding to the traffic scene map. Finally, the topology understanding module generates at least one lane centerline through the preset Bezier deformation attention.

[0051] In summary, this application includes at least one of the following beneficial technical effects: After acquiring a traffic scene image of the target vehicle during its driving process, image features of different scales aligned to a clean image are extracted from the traffic scene image, thereby obtaining high-quality prior image features corresponding to this traffic scene image, namely multi-scale prior features, which facilitates the provision of useful prior knowledge for subsequent restoration and optimization of the image quality of the traffic scene image. Furthermore, the multi-scale prior features are fused with degradation features, thereby, guided by prior knowledge, promoting the transformation of degradation regions in the traffic scene image into clean regions, thus optimizing blurry or poor-quality areas in the traffic scene image. The fused features are then subjected to fusion optimization processing, thereby alleviating the structural warping and texture distortion problems caused by the direct fusion of high-quality prior features and low-quality degradation features, resulting in an optimized scene image with better clarity optimization. Finally, based on this optimized scene image, lane centerlines are generated more accurately, thereby improving the vehicle's road topology understanding ability. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a road topology understanding method provided in an embodiment of this application;

[0053] Figure 2 This is a schematic diagram of the structure of a road topology understanding system provided in an embodiment of this application.

[0054] Explanation of reference numerals in the attached figures: 11. Information acquisition module; 12. Prior determination module; 13. Feature fusion module; 14. Image optimization module; 15. Topology understanding module. Detailed Implementation

[0055] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0056] In the description of the embodiments of this application, words such as "exemplarily," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0057] In the description of the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, or A and B existing simultaneously. Furthermore, unless otherwise stated, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0058] See Figure 1 This application discloses a flowchart of a road topology understanding method, which can be implemented using a computer program or run on a road topology understanding system based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application, specifically including:

[0059] S101: Obtain a traffic scene map of the target car during its driving process.

[0060] Specifically, in this embodiment, the target vehicle is a car in autonomous driving mode driving on the road. In other embodiments, it can also be a car in intelligent assisted driving mode driving on the road. The traffic scene map can be understood as an image reflecting the road conditions around the target vehicle. It should be noted that the car in autonomous driving mode needs to capture real-time images of the surrounding road to perceive or understand the surrounding road conditions, such as identifying lane center lines, thereby better autonomously planning driving paths and making reasonable driving decisions.

[0061] The road topology understanding method disclosed in this application is implemented by a target vehicle itself. The target vehicle is equipped with a camera that can acquire real-time video of the surrounding road conditions while the target vehicle is driving, and extract the current frame from it, i.e., the traffic scene map of the target vehicle's current driving situation. However, the target vehicle may be affected by environmental factors such as rain, fog, and poor lighting, resulting in a blurry traffic scene map with poor image quality. This leads to a poor ability for the target vehicle to understand the road topology and make better driving decisions.

[0062] S102: Based on the traffic scene map, determine the corresponding multi-scale prior features.

[0063] Specifically, multi-scale prior features are image features aligned to different scales of a clean image. A clean image can be understood as an image of the traffic scene map in a normally clear state. Multi-scale prior features can be understood as prior knowledge corresponding to the traffic scene map in a normally clear state, which can be used to guide subsequent clarity restoration of the traffic scene map. After the traffic scene map is determined, image preprocessing is required. That is, initial image features are extracted from the traffic scene map through a series of convolutional layers, and then the initial image features are mapped to a pre-trained Vector Quantized Variational Autoencoder (VQVAE) codebook to obtain a single-scale mapping result, i.e., a single-scale token map. Token maps are a technique used in image processing, computer vision, and natural language processing for multi-scale analysis and representation of data. The VQVAE codebook is an important component of the Vector Quantized Variational Autoencoder. Mapping the initial image features to this VQVAE codebook aims to transform the initial image features into a low-dimensional discrete vector space, reducing the amount of data to some extent. Furthermore, the initial image features are mapped into this VQVAE codebook to obtain mapping results, providing input for the subsequent Vector Autoregression (VAR) model. This facilitates the subsequent implementation of multi-scale prior features corresponding to this traffic scene map in a scale-by-scale prediction manner. The Vector Autoregression model is a statistical model used to analyze the interdependencies between multiple time series. In the VAR model, the current value of each time series is a function of its past values ​​and the past values ​​of other time series.

[0064] Further, the mapping results of all current scales are input into the vector autoregressive (VAR) model to obtain the mapping result corresponding to the next scale. Then, the step of inputting the mapping results of all current scales into the preset VAR model is repeated to continue determining the mapping result corresponding to the next scale through the VAR model, ultimately resulting in mapping results at a preset number of scales. For example, if the mapping result obtained based on the initial image feature mapping is r1, then r1 is input into the VAR model to obtain the mapping result corresponding to the next scale, denoted as r2. Then, the mapping results corresponding to all current scales, i.e., r1 and r2, are simultaneously input into the VAR model to obtain the mapping result corresponding to the next scale after that, and so on, obtaining mapping results corresponding to multiple scales. Further, in the decoder stage, all mapping results, i.e., the mapping results corresponding to the preset number of scales, are decoded to obtain the multi-scale prior features corresponding to the traffic scene map, and then... S v express, , kThe number represents a preset number. The multi-scale prior features include image feature representations from low resolution to high resolution, and have been gradually aligned to the distribution of clean images during the autoregressive generation process, providing useful prior knowledge for the subsequent restoration of the clarity of the traffic scene map.

[0065] S103: Fuse multi-scale prior features and degradation features in traffic scene maps to obtain fused features.

[0066] Specifically, degraded features in traffic scene images refer to features in traffic scene images that have experienced quality degradation, information loss, or reduced resolution due to various reasons. The optimized scene image is the traffic scene image after its clarity has been improved. After determining the multi-scale prior features, the traffic scene image is then input into a pre-defined residual block to extract the degraded features. A residual block is a convolutional neural network structure widely used in deep learning, offering significant advantages in extracting image degraded features. Each residual block includes two convolutional layers (for feature extraction) and a residual connection (to address the vanishing and exploding gradient problems). Further, the degraded features are input into a pre-defined weight predictor, undergoing feature processing via a self-attention mechanism and a feedforward neural network to obtain the processed features. The weight predictor is a tool or algorithm used in machine learning and data modeling to estimate and predict weight parameters.

[0067] The processed features are mapped using a pre-defined projection convolutional layer to obtain corresponding mapped weights; multiple mapped weights exist. These mapped weights are then used to weight the multi-scale prior features, i.e., to... Sv middle The various features are weighted and summed to obtain weighted prior features. Finally, a lightweight projection head is used to perform a dimensionality transformation on the weighted prior features to obtain processed prior features, which are represented as follows: ,in, This represents a lightweight projection head, where j represents the j-th scale. This represents the prior feature corresponding to the j-th scale. This represents the weights after mapping.

[0068] Furthermore, the processed prior features and the previous optimization result are substituted into the preset weight calculation formula to obtain the first weight and the second weight. In this embodiment, the process of restoring the clarity of the traffic scene image involves fusing degraded features with multi-scale prior features. After fusion, to alleviate the structural warping and texture distortion caused by directly fusing high-quality prior features with low-quality degraded features, further fusion optimization is required. This process is repeated multiple times, involving feature fusion and subsequent optimization. Therefore, the previous optimization result is the result of the previous fusion and optimization of the degraded features and multi-scale prior features in the traffic scene image. The weight calculation formula is as follows: ;

[0069] In the formula, concat represents the concatenation operation, RSTBs represents a series of residual Swin-Transformer blocks, conv represents a convolutional layer, and softmax represents a softmax activation layer. Indicates the first weight. Indicates the second weight. This indicates the result of the previous optimization. Let i represent the prior features after processing, and let i represent the i-th feature fusion.

[0070] Furthermore, the previous optimization result and the processed prior features are weighted and summed using the first weight and the second weight to obtain the fused features, specifically represented as follows: In the formula, This represents the fused features after this feature fusion.

[0071] S104: Perform fusion optimization processing on the fused features to obtain the optimized scene map corresponding to the traffic scene map.

[0072] Specifically, after determining the fused features, a fusion optimization process needs to be performed on the fused features to alleviate the structural warping and texture distortion problems caused by directly fusing high-quality prior features and low-quality degenerate features corresponding to the traffic scene map, thereby improving the clarity recovery effect of the traffic scene map. In this embodiment, a feasible processing method is to concatenate the previous optimization result with the fused features to obtain the concatenated result, i.e., Then, a projection transformation is performed on the previous optimization result to obtain the first transformation result, that is, the result of the projection transformation is obtained. Projected to The spliced ​​result is then subjected to a projection transformation to obtain the second transformation result, i.e., the result is... Projected to ; Perform projection transformation on the fused features to obtain the third and fourth transformation results, i.e. Projected onto K= and Among them, W q W m W k W v Each represents the weight matrix during the projection transformation. Finally, based on the determined first, second, third, and fourth transformation results, a two-layer attention mechanism is used to perform fusion optimization processing on the fused features to obtain the optimized result. The specific process is described below: In the formula, Q represents the first transformation result, M represents the second transformation result, K represents the third transformation result, V represents the fourth transformation result, T represents the transpose, and d represents the scalable training factor. Simultaneously, the obtained optimization result is used as the previous optimization result in the next feature fusion and fusion optimization process.

[0073] Finally, the step of inputting the traffic scene map into the preset residual block in step S103 is repeated to extract the degradation features again. Based on the mapped weights after the degradation feature mapping process, the multi-scale prior features are weighted and their dimensions are changed to obtain the processed prior features. The processed prior features and the previous optimization result are then weighted and summed to achieve the fusion of the multi-scale prior features and the degradation features in the traffic scene map once again, resulting in fused features. Finally, the fusion optimization process is performed based on these fused features and the previous optimization result. This process is repeated a preset number of times to obtain the final optimization result. This final optimization result is then passed through a 3x3 convolutional layer to obtain an optimized scene map with better clarity optimization, i.e., a scene map with improved clarity after optimizing the traffic scene map.

[0074] In other embodiments, based on the historical accident records of the current road, the first occurrence frequency of road areas where inter-lane vehicle accidents have historically occurred is counted. A first number of road areas are selected from each road area according to the descending order of the first occurrence frequency to determine the target road area, i.e., the area prone to inter-lane vehicle accidents. Historical time periods in which inter-lane vehicle accidents occurred in a single target road area are obtained. The second occurrence frequency of each historical time period is counted. A second number of historical time periods are selected from each historical time period according to the descending order of the second occurrence frequency to determine the target time period corresponding to the single target road area, i.e., the time period prone to inter-lane vehicle accidents. The historical accident records include the road areas where inter-lane vehicle accidents have historically occurred on the current road and the corresponding occurrence time periods.

[0075] Further, a first weight is determined for each target road area. This first weight is the ratio of the first occurrence count of each target road area to the sum of the first occurrence counts of all target road areas. Then, a second weight is determined for each target road area corresponding to a target time period. This second weight is the ratio of the second occurrence count of a single target time period corresponding to the target road area to the sum of the second occurrence counts of all corresponding target time periods. Further, the actual area where the target vehicle is currently located is obtained. When the actual area is a target road area, the product of the first weight of this actual area and the second weight of the corresponding key time period is calculated. The larger the product, the greater the probability of the target vehicle being involved in an inter-lane collision. The key time period is the target time period in which the current time is located. Finally, a preset number of occurrences corresponding to this product is determined from a preset number of occurrences matching table. The larger the product and the larger the preset number of occurrences, the more accurate the generated lane centerline, the better it is at preventing inter-lane collisions and improving the safety of the target vehicle. The number of occurrences matching table includes different products and their corresponding preset number of occurrences, all set based on human experience.

[0076] S105: Based on the optimized scene map, at least one lane centerline is generated through a preset Bezier deformation attention.

[0077] Specifically, Bézier Deformable Attention combines the properties of Bézier curves with a self-attention mechanism. Specifically, it uses Bézier curves to define the distribution or deformation of attention, allowing the attention mechanism to more flexibly adapt to the geometric shape or structural features of the input data. After the optimized scene map is determined, it is converted into perspective view features using a preset feature extraction function, where the SIFT function is used. Then, the perspective view features are aggregated and projected onto the BEV feature map using a preset projection function, obtaining the projection result. The LSS (Lift-Splat-Shoot) projection function is used. It should be noted that the BEV feature map is a widely used representation method in the fields of autonomous driving and computer vision. It generates feature maps by projecting object information in three-dimensional space onto a bird's-eye view (BEV), thus facilitating subsequent processing and analysis.

[0078] Furthermore, the projection results are input into a Transformer decoder based on Bézier deformation attention to determine the final set of Bézier control points generated by the last layer of the Transformer decoder. This final set of Bézier control points is a collection of multiple Bézier control points. Finally, at least one lane centerline is generated based on these multiple Bézier control points. The Bézier control points are key parameters for defining and adjusting the shape of the Bézier curve. By controlling the position and number of control points, the direction, curvature, and smoothness of the curve can be precisely controlled. In this embodiment, the lane centerline can be considered a curve defined by Bézier control points. Based on the Bézier control points in the final set, the shape and curvature of the lane centerline can be accurately simulated, helping to improve the target vehicle's road topology understanding ability. It should be noted that the lane centerline can be generated using the preset Adobe Illustrator tool based on the Bézier control points in the final set, assisting the target vehicle in road topology understanding. In other embodiments, the lane centerline can also be generated using the preset CorelDRAW tool.

[0079] Furthermore, a feasible way to determine the final set of Bezier control points is as follows: Using a multilayer perceptron and a sigmoid function, an initial set of Bezier control points for the first query embedding of a single layer of the Transformer decoder is generated. This set is a collection of Bezier control points that define the shape of the lane centerline. For example, the initial set of Bezier control points for the first query embedding of the first layer of the Transformer decoder can be generated, and the specific generation process is as follows: , This represents the first query embedding of the first layer. Query embedding is a technique that transforms query information into a low-dimensional vector representation, used to describe and locate the desired features. The Multilayer Perceptron (MLP) is a feedforward neural network model consisting of an input layer, an output layer, and at least one hidden layer.

[0080] Next, based on this first query embedding and the position embedding at the same layer of the Transformer decoder, the second query embedding at the same layer of the Bezier deformation attention is determined. The specific process is described as follows: , This represents the first query embedding of the l-th layer of the Transformer decoder. This indicates the position embedding of the l-th layer of the Transformer decoder. Let represent the second query embedding of the l-th layer of the Bezier deformation attention. Next, based on the projection result, the second query embedding, and the initial set of Bezier control points, Bezier deformation attention is used to guide attention with Bezier control points that define the shape of the lane centerline, yielding the output result. This achieves accurate capture of the position and shape of the lane centerline involved in the traffic scene map. Then, based on the output result and the position embedding, self-attention is calculated to obtain the calculation result, thereby optimizing the feature representation. The specific process is as follows: , This represents the output of the l-th layer.

[0081] Finally, the computation result of the current layer is used as the first query embedding of the next layer of the Transformer decoder, and the step of generating the initial set of Bezier control points for the first query embedding of a single layer of the Transformer decoder through a multilayer perceptron and a sigmoid function is repeated until the final set of Bezier control points generated by the last layer of the Transformer decoder is obtained. It should be noted that the computation process for the set of Bezier control points for the first query embedding of the next layer during repeated execution is as follows: In the formula, This represents the set of Bessel control points embedded in the first query at level l. This represents the set of Bessel control points embedded in the first query of the layer above the l-th layer.

[0082] The implementation principle of the road topology understanding method in this application is as follows: After acquiring a traffic scene map of the target vehicle during its driving process, image features of different scales aligned to a clean image are extracted from the traffic scene map to obtain high-quality prior image features corresponding to this traffic scene map, namely multi-scale prior features. This provides useful prior knowledge for subsequent image quality restoration and optimization of the traffic scene map. Furthermore, the multi-scale prior features are fused with degradation features, thereby guiding the transformation of degradation regions in the traffic scene map to clean regions under the guidance of prior knowledge, thus optimizing blurry and low-quality areas in the traffic scene map. The fused features are then subjected to fusion optimization processing to alleviate the structural warping and texture distortion problems caused by the direct fusion of high-quality prior features and low-quality degradation features, resulting in an optimized scene map with better clarity optimization. Finally, based on this optimized scene map, lane centerlines are generated more accurately, thereby improving the vehicle's road topology understanding capability.

[0083] The following are system embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of this application.

[0084] Please see Figure 2This is a schematic diagram of the road topology understanding system provided in an embodiment of this application. This road topology understanding system can be implemented as all or part of a system through software, hardware, or a combination of both. The system includes an information acquisition module 11, a priori determination module 12, a feature fusion module 13, an image optimization module 14, and a topology understanding module 15.

[0085] Information acquisition module 11 is used to acquire traffic scene images of the target vehicle during its driving process;

[0086] The prior determination module 12 is used to determine the corresponding multi-scale prior features based on the traffic scene map. The multi-scale prior features are image features of different scales aligned to the clean image.

[0087] Feature fusion module 13 is used to fuse multi-scale prior features and degenerate features in traffic scene map to obtain fused features;

[0088] Image optimization module 14 is used to perform fusion optimization processing on the fused features to obtain an optimized scene map corresponding to the traffic scene map. The optimized scene map is a scene map after the traffic scene map has been optimized in terms of clarity.

[0089] The topology understanding module 15 is used to generate at least one lane centerline based on the optimized scene graph and through a preset Bezier deformation attention.

[0090] Optional, the prior determination module 12 is specifically used for:

[0091] Initial image features are extracted from the traffic scene map through a pre-defined convolutional layer, and the initial image features are mapped to the pre-trained VQVAE codebook to obtain a single-scale mapping result.

[0092] Input the mapping results of all current scales into the preset vector autoregression model to obtain the mapping results of the next scale, and repeat the step of inputting the mapping results of all current scales into the preset vector autoregression model to obtain the mapping results at a preset number of scales.

[0093] Decode all mapping results to obtain multi-scale prior features corresponding to the traffic scene map.

[0094] Optional, feature fusion module 13, specifically used for:

[0095] Input the traffic scene map into the preset residual block to obtain the degradation features;

[0096] The degraded features are input into the weight predictor to obtain the processed features, and the processed features are mapped through a preset projection convolutional layer to obtain the mapped weights.

[0097] The multi-scale prior features are weighted by mapping and then transformed by a preset lightweight projection head to obtain the processed prior features.

[0098] The result of the previous fusion and optimization of the degradation features and multi-scale prior features in the traffic scene map is determined as the previous optimization result. The processed prior features and the previous optimization result are substituted into the preset weight calculation formula to obtain the first weight and the second weight.

[0099] Based on the first and second weights, the processed prior features and the previous optimization result are weighted and summed to obtain the fused features.

[0100] Optional, image optimization module 14, specifically used for:

[0101] The previous optimization result and the fused features are then concatenated to obtain the concatenated result;

[0102] The previous optimization result is transformed by projection to obtain the first transformation result, and the splicing result is transformed by projection to obtain the second transformation result.

[0103] The fused features are subjected to projection transformation to obtain the third and fourth transformation results. Based on the first, second, third, and fourth transformation results, the fused features are optimized through a two-layer attention mechanism to obtain the optimized result.

[0104] Repeat the steps of inputting the traffic scene map into the preset residual block to obtain the optimized scene map corresponding to the traffic scene map.

[0105] Optional, topology understanding module 15, specifically used for:

[0106] The optimized scene map is converted into perspective view features using a preset feature extraction function;

[0107] The perspective view features are aggregated and projected onto the BEV feature map using a preset projection function to obtain the projection result.

[0108] The projection results are input into the Transformer decoder based on Bézier deformation attention to obtain the final set of Bézier control points generated by the last layer of the Transformer decoder;

[0109] Based on the final set of Bezier control points, generate at least one lane centerline corresponding to the optimized scene map.

[0110] Optional, topology understanding module 15, specifically used for:

[0111] By using a multilayer perceptron and a sigmoid function, the initial set of Bezier control points for the first query embedding of a single layer of the Transformer decoder is generated;

[0112] Based on the first query embedding and the position embedding at the same layer of the Transformer decoder, the second query embedding at the same layer of the Bezier deformation attention is determined;

[0113] Based on the projection results, the second query embedding, and the initial set of Bezier control points, the output results are obtained by utilizing Bezier deformation attention.

[0114] Based on the output and position embedding, self-attention calculation is performed to obtain the calculation result;

[0115] The computation result is used as the first query embedding of the next layer of the Transformer decoder, and the steps of generating the initial set of Bezier control points for the first query embedding of a single layer of the Transformer decoder through a multilayer perceptron and a sigmoid function are repeated to obtain the final set of Bezier control points generated by the last layer of the Transformer decoder.

[0116] It should be noted that the road topology understanding system provided in the above embodiments is only illustrated by the division of the above functional modules when executing the road topology understanding method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the road topology understanding system and the road topology understanding method embodiment provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiment, which will not be repeated here.

[0117] This application discloses a computer program product, including a computer program that, when executed by a processor, implements a road topology understanding method as described in the above embodiments.

[0118] This application also discloses a computer-readable storage medium, which stores a computer program that, when executed by a processor, implements a road topology understanding method as described in the above embodiments.

[0119] The computer program can be stored in a computer-readable medium. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or certain middleware. The computer-readable medium includes any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the computer-readable medium includes, but is not limited to, the above-mentioned components.

[0120] The road topology understanding method of the above embodiment is stored in the computer-readable storage medium and loaded and executed on the processor to facilitate the storage and application of the above method.

[0121] This application also discloses an electronic device in which a computer program is stored in a computer-readable storage medium. When the computer program is loaded and executed by a processor, it implements the above-mentioned road topology understanding method.

[0122] The electronic device can be a desktop computer, a laptop computer, or a cloud server, and includes, but is not limited to, a processor and a memory. For example, the electronic device may also include input / output devices, network access devices, and buses.

[0123] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it.

[0124] The memory can be an internal storage unit of an electronic device, such as a hard disk or RAM, or an external storage device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) equipped on the electronic device. Furthermore, the memory can be a combination of an internal storage unit and an external storage device. The memory is used to store computer programs and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.

[0125] In this electronic device, a road topology understanding method according to the above embodiment is stored in the memory of the electronic device and loaded and executed on the processor of the electronic device for convenient use.

[0126] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for understanding road topology, characterized in that, The method includes: Obtain a traffic scene map of the target car during its driving process; Based on the traffic scene map, the corresponding multi-scale prior features are determined, including: extracting initial image features from the traffic scene map through a preset convolutional layer, and mapping the initial image features to a pre-trained VQVAE codebook to obtain a single-scale mapping result; Input the mapping results of all current scales into the preset vector autoregressive model to obtain the mapping results of the next scale, and repeat the step of inputting the mapping results of all current scales into the preset vector autoregressive model to obtain the mapping results at a preset number of scales. Decode all the mapping results to obtain the multi-scale prior features corresponding to the traffic scene map. The multi-scale prior features are image features of different scales aligned to a clean image. The multi-scale prior features and the degradation features in the traffic scene map are fused to obtain fused features, including: inputting the traffic scene map into a preset residual block to obtain degradation features; The degraded features are input into the weight predictor to obtain the processed features, and the processed features are mapped through a preset projection convolutional layer to obtain the mapped weights. The multi-scale prior features are weighted using the mapped weights and then transformed using a preset lightweight projection head to obtain the processed prior features. The result of the previous fusion and optimization of the degradation features and the multi-scale prior features in the traffic scene map is determined as the previous optimization result. The processed prior features and the previous optimization result are substituted into the preset weight calculation formula to obtain the first weight and the second weight. Based on the first weight and the second weight, the processed prior features and the previous optimization result are weighted and summed to obtain the fused features; The fusion optimization process is performed on the fused features to obtain the optimized scene map corresponding to the traffic scene map, including: stitching the previous optimization result and the fused features together to obtain the stitching result; The previous optimization result is subjected to a projection transformation to obtain a first transformation result, and the splicing result is subjected to a projection transformation to obtain a second transformation result; The fused features are subjected to projection transformation to obtain a third transformation result and a fourth transformation result. Based on the first transformation result, the second transformation result, the third transformation result, and the fourth transformation result, the fused features are subjected to fusion optimization processing through a two-layer attention mechanism to obtain an optimized result. Repeat the step of inputting the traffic scene map into the preset residual block to obtain the optimized scene map corresponding to the traffic scene map. The optimized scene map is the scene map after the traffic scene map has been optimized in terms of clarity. Based on the optimized scene map, at least one lane centerline is generated through a preset Bezier deformation attention.

2. The road topology understanding method according to claim 1, characterized in that, The formula for calculating the weights is as follows: ； In the formula, concat represents the concatenation operation. Represents a series of residuals In this block, conv represents a convolutional layer, and softmax represents a softmax activation layer. Indicates the first weight. Indicates the second weight. This indicates the result of the previous optimization. This represents the prior features after processing.

3. The road topology understanding method according to claim 1, characterized in that, Based on the optimized scene map, generating at least one lane centerline using a preset Bezier deformation attention mechanism specifically includes: The optimized scene map is converted into perspective view features using a preset feature extraction function; The perspective view features are aggregated and projected onto the BEV feature map using a preset projection function to obtain the projection result. The projection result is input into a Bezier deformation-based attention system. In the decoder, the following is obtained The final set of Bezier control points generated by the last layer of the decoder; Based on the final set of Bezier control points, at least one lane centerline corresponding to the optimized scene map is generated.

4. The road topology understanding method according to claim 3, characterized in that, The projection result is then input into a Bezier deformation-based attention system. In the decoder, the following is obtained The final set of Bezier control points generated by the last layer of the decoder specifically includes: Through multi-layer perceptron and The function generates the above. The first query of the decoder layer embeds the initial set of Bessel control points; Based on the first query embedding and the The position embedding of the decoder at the same layer determines the second query embedding of the same layer for the Bezier deformation attention; Based on the projection result, the second query embedding, and the initial set of Bézier control points, the output result is obtained using Bézier deformation attention; Based on the output and the position embedding, self-attention calculation is performed to obtain the calculation result; The calculation result is used as the... The first query embedding of the next layer of the decoder is repeated, and the process is repeated through the multilayer perceptron and... The function generates the above. The first query embedding of the decoder layer, the initial set of Bezier control points, is used to obtain the... The final set of Bezier control points generated by the last layer of the decoder.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor loads and executes the computer program, it implements the method of any one of claims 1-4.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it implements the method of any one of claims 1-4.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-4.

Images