Road topology understanding method and device, storage medium and program product

Abstract

The invention relates to a road topology understanding method and device, a storage medium and a program product, and relates to the technical field of road topologies, and the method comprises the steps: obtaining a traffic scene graph of a target automobile in a driving process; based on the traffic scene graph, corresponding multi-scale prior features are determined, the multi-scale prior features are image features of different scales aligned to a clean image, and the multi-scale prior features and degradation features in the traffic scene graph are fused to obtain fused features; performing fusion optimization processing on the fused features to obtain an optimized scene graph corresponding to the traffic scene graph, the optimized scene graph being a scene graph obtained by optimizing the definition of the traffic scene graph; and based on the optimized scene graph, generating at least one corresponding lane center line through preset Bessel deformation attention. The method has the effect of improving the understanding capability of the automobile road topology.

Description

Technical Field

[0001] The present application relates to the field of road topology technology, and in particular to a road topology understanding method, device, storage medium, and program product. Background Art

[0002] Road topology refers to a topological structure that describes the structure of a road network and its interconnections. For cars, especially self-driving cars, understanding road topology means being able to grasp their position within the road network, as well as the layout and connectivity of their surroundings. This understanding is crucial for functions such as navigation, route planning, and autonomous driving. It helps cars accurately determine driving style, select optimal routes, and avoid getting lost or causing traffic accidents in complex road environments. With the continuous advancement of artificial intelligence technologies, autonomous driving technology is rapidly developing towards widespread adoption and high intelligence, and with it, the requirements for cars' road topology understanding capabilities are becoming increasingly stringent.

[0003] Currently, in the field of autonomous driving, real-time images of the vehicle's surroundings collected by the car are usually subjected to general conventional image processing, such as preprocessing, feature extraction, object detection and recognition, and scene understanding, to help the car understand road topology. Once severe weather such as rain, fog, and snow or poor lighting occurs, the image quality and clarity of the real-time images collected around the vehicle will be poor. This will reduce the car's ability to understand the surrounding road topology, resulting in poor road topology understanding. Summary of the Invention

[0004] In order to improve the road topology understanding capability of a car, the present application provides a road topology understanding method, device, system, storage medium and program product.

[0005] In a first aspect of the present application, a method for understanding road topology is provided, which specifically includes: Obtain a traffic scene graph of the target car while it is driving; Determining corresponding multi-scale prior features based on the traffic scene graph, where the multi-scale prior features are image features of different scales aligned to a clean image; fusing the multi-scale prior features with the degraded features in the traffic scene graph to obtain fused features; Performing fusion optimization processing on the fused features to obtain an optimized scene graph corresponding to the traffic scene graph, wherein the optimized scene graph is a scene graph after the traffic scene graph is optimized in terms of clarity; Based on the optimized scene graph, at least one corresponding lane centerline is generated through a preset Bezier deformation attention.

[0006] By adopting the above technical solution, after obtaining the traffic scene graph of the target car during driving, image features of different scales aligned to the clean image are extracted from the traffic scene graph, thereby obtaining high-quality prior image features corresponding to this traffic scene graph, namely multi-scale prior features, which facilitate the subsequent provision of useful prior knowledge for restoring and optimizing the image quality of the traffic scene graph. Furthermore, the multi-scale prior features are fused with the degradation features, so that under the guidance of prior knowledge, the degradation domain in the traffic scene graph is promoted to be converted to the clean domain, thereby optimizing the blurred or poor-quality areas in the traffic scene graph. The fused features are then subjected to fusion optimization processing to alleviate the problems of structural warping and texture distortion caused by the direct fusion of high-quality prior features and low-quality degradation features, thereby obtaining an optimized scene graph with better clarity optimization effect. Finally, based on this optimized scene graph, lane centerlines are more accurately generated, thereby improving the car's road topology understanding ability.

[0007] In one possible design, determining corresponding multi-scale prior features based on the traffic scene graph specifically includes: Extracting initial image features from the traffic scene graph through a preset convolutional layer, and mapping the initial image features to a pre-trained VQVAE codebook to obtain a mapping result at a single scale; Inputting the mapping results of all current scales into a preset vector autoregression model to obtain a mapping result of the next scale, and re-performing the step of inputting the mapping results of all current scales into the preset vector autoregression model to obtain mapping results at a preset number of scales; All the mapping results are decoded to obtain multi-scale prior features corresponding to the traffic scene graph.

[0008] In a possible design, the multi-scale prior features and the degraded features in the traffic scene graph are fused to obtain fused features, specifically including: Inputting the traffic scene graph into a preset residual block to obtain degradation features; Inputting the degradation features into a weight predictor to obtain processed features, and mapping the processed features through a preset projection convolution layer to obtain mapped weights; The multi-scale prior features are weighted by the mapped weights and dimensionality transformed by a preset lightweight projection head to obtain processed prior features; Determining the result of the last fusion and optimization of the degraded features in the traffic scene graph and the multi-scale prior features as the last optimization result, and substituting the processed prior features and the last optimization result into a preset weight calculation formula to obtain a first weight and a second weight; Based on the first weight and the second weight, a weighted sum is performed on the processed prior features and the previous optimization result to obtain a fused feature.

[0009] In one possible design, the weight calculation formula is: ; Where concat represents the concatenation operation, RSTBs represents a series of residual Swin-Transformer blocks, conv represents the convolutional layer, and softmax represents the softmax activation layer. represents the first weight, represents the second weight, Indicates the previous optimization result. Represents the processed prior features.

[0010] In a possible design, performing fusion optimization processing on the fused features to obtain an optimized scene graph corresponding to the traffic scene graph specifically includes: Splicing the previous optimization result and the fused features to obtain a splicing result; Performing a projection transformation on the previous optimization result to obtain a first transformation result, and performing a projection transformation on the splicing result to obtain a second transformation result; Performing a projection transformation on the fused features to obtain a third transformation result and a fourth transformation result, and performing a fusion optimization process on the fused features through a double-layer attention mechanism based on the first transformation result, the second transformation result, the third transformation result, and the fourth transformation result to obtain an optimization result; Repeat the step of inputting the traffic scene graph into a preset residual block to obtain an optimized scene graph corresponding to the traffic scene graph.

[0011] In one possible design, generating at least one corresponding lane centerline based on the optimized scene graph through a preset Bezier deformation attention specifically includes: Converting the optimized scene graph into perspective view features through a preset feature extraction function; Aggregating the perspective view features and projecting them into a BEV feature map using a preset projection function to obtain a projection result; Inputting the projection result into a Transformer decoder based on Bessel deformation attention to obtain the final Bessel control point set generated by the last layer of the Transformer decoder; At least one lane centerline corresponding to the optimized scene graph is generated based on the final Bezier control point set.

[0012] In one possible design, the projection result is input into a Transformer decoder based on Bessel deformation attention to obtain a final Bessel control point set generated by the last layer of the Transformer decoder, specifically including: Generate an initial set of Bezier control points for the first query embedding of a single layer of the Transformer decoder using a multilayer perceptron and a sigmoid function; Determining a second query embedding for the same layer of the Bessel warped attention based on the first query embedding and the position embedding of the same layer of the Transformer decoder; Obtaining an output result using Bezier deformation attention based on the projection result, the second query embedding, and the initial Bezier control point set; Performing a self-attention calculation based on the output result and the position embedding to obtain a calculation result; The calculation result is used as the first query embedding of the next layer of the Transformer decoder, and the steps of generating an initial Bezier control point set of the first query embedding of the single layer of the Transformer decoder through the multilayer perceptron and the sigmoid function are repeated to obtain a final Bezier control point set generated by the last layer of the Transformer decoder.

[0013] In a second aspect of the present application, an electronic device is provided, specifically comprising: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the processor is used to load and execute the computer program stored in the memory so that the electronic device performs the method as described in any one of the first aspects.

[0014] In a third aspect of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is loaded and executed by a processor, the method steps as described in any one of the first aspects are performed.

[0015] In a fourth aspect of the present application, a computer program product is provided, comprising a computer program, which implements the method according to any one of the first aspects when the computer program is executed by a processor.

[0016] In a fifth aspect of the present application, a road topology understanding system is provided, comprising: An information acquisition module is used to obtain a traffic scene graph of the target car during its driving process; A priori determination module is used to determine corresponding multi-scale prior features based on the traffic scene graph, where the multi-scale prior features are image features of different scales aligned to the clean image; A feature fusion module, configured to fuse the multi-scale prior features with the degraded features in the traffic scene graph to obtain fused features; An image optimization module is used to perform fusion optimization processing on the fused features to obtain an optimized scene graph corresponding to the traffic scene graph, wherein the optimized scene graph is a scene graph after the clarity of the traffic scene graph is optimized; A topology understanding module is used to generate at least one corresponding lane centerline based on the optimized scene graph through a preset Bezier deformation attention.

[0017] By adopting the above technical solution, after the information acquisition module obtains the traffic scene graph of the target car during driving, the prior determination module determines the corresponding multi-scale prior features based on the traffic scene graph. Then, the feature fusion module fuses the multi-scale prior features with the degraded features in the traffic scene graph to obtain the fused features. Then, the image optimization module determines the optimized scene graph corresponding to the traffic scene graph. Finally, the topology understanding module generates at least one corresponding lane centerline through the preset Bezier deformation attention.

[0018] In summary, the present application includes at least one of the following beneficial technical effects: after obtaining the traffic scene graph of the target car during driving, image features of different scales aligned to the clean image are extracted from the traffic scene graph, and then high-quality prior image features corresponding to this traffic scene graph, namely multi-scale prior features, are obtained, which facilitate the subsequent provision of useful prior knowledge for restoring and optimizing the image quality of the traffic scene graph. Furthermore, the multi-scale prior features are fused with the degradation features, so that under the guidance of the prior knowledge, the degradation domain in the traffic scene graph is promoted to be converted to the clean domain, thereby optimizing the blurred or poor-quality areas in the traffic scene graph. The fused features are then subjected to fusion optimization processing, thereby alleviating the problems of structural warping and texture distortion caused by the direct fusion of high-quality prior features and low-quality degradation features, thereby obtaining an optimized scene graph with better clarity optimization effect, and finally, based on this optimized scene graph, the lane centerline is generated more accurately, thereby improving the road topology understanding ability of the car. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a road topology understanding method provided in an embodiment of the present application; Figure 2 This is a structural diagram of a road topology understanding system provided in an embodiment of the present application.

[0020] Explanation of the accompanying symbols: 11. Information acquisition module; 12. Prior determination module; 13. Feature fusion module; 14. Image optimization module; 15. Topology understanding module. DETAILED DESCRIPTION

[0021] In order to enable people skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0022] In the description of the embodiments of this application, words such as "exemplarily," "for example," or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily," "for example," or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0023] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0024] See also Figure 1 The present application discloses a flowchart of a road topology understanding method, which can be implemented using a computer program or run on a road topology understanding system based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone tool application, specifically including: S101: Obtain a traffic scene image of a target vehicle during its driving process.

[0025] Specifically, in the embodiment of the present application, the target vehicle is a vehicle in an autonomous driving state traveling on the road. In other embodiments, it may also be a vehicle in an intelligent assisted driving state traveling on the road. The traffic scene graph can be understood as an image reflecting the road conditions around the target vehicle. It should be noted that the vehicle in the autonomous driving state needs to capture real-time images of the surrounding roads in order to perceive or understand the topology of the surrounding road conditions, for example, to identify the centerline of the lane in the road, so as to better autonomously plan the driving path and make reasonable driving decisions.

[0026] In the embodiments of the present application, a method for understanding road topology is implemented by a target vehicle itself. The target vehicle is equipped with a camera that captures real-time video of the road conditions surrounding the target vehicle while the vehicle is currently traveling. The camera then captures a current frame of video, i.e., a traffic scene map of the target vehicle as it is currently traveling. However, the target vehicle may be affected by environmental factors such as rain, fog, and poor lighting, resulting in a blurred and poorly-quality traffic scene map. This can lead to a poor understanding of the road topology and an inability to make effective driving decisions.

[0027] S102: Determine corresponding multi-scale prior features based on the traffic scene graph.

[0028] Specifically, multi-scale prior features are image features at different scales aligned to a clean image. The clean image can be understood as an image of the traffic scene graph in its normal, clear state. These multi-scale prior features can be understood as prior knowledge corresponding to the normal, clear state of the traffic scene graph and can be used to guide subsequent clarity restoration of the traffic scene graph. After the traffic scene graph is determined, image preprocessing is required. Initial image features are extracted from the traffic scene graph through a series of convolutional layers. These initial image features are then mapped to a pretrained Vector Quantized Variational Autoencoder (VQVAE) codebook to produce a single-scale mapping result, known as a token map. The token map is a technique used in fields such as image processing, computer vision, and natural language processing for multi-scale data analysis and representation. The VQVAE codebook is a key component of the VQVAE. Mapping the initial image features to this VQVAE codebook aims to convert them into a low-dimensional discrete vector space, reducing the amount of data required. Furthermore, the initial image features are mapped to this VQVAE codebook, generating a mapping result that provides input to the subsequent Vector Autoregression (VAR) model, facilitating the subsequent scale-by-scale prediction of the traffic scene graph and determining the multi-scale prior features corresponding to the traffic scene graph. The Vector Autoregression model is a statistical model used to analyze the interdependencies between multiple time series. In a VAR model, the current value of each time series is a function of its past values and the past values of all other time series.

[0029] Furthermore, the mapping results of all current scales are input into the vector autoregression model to obtain the mapping results corresponding to the next scale. Then, the step of inputting the mapping results of all current scales into the preset vector autoregression model is repeated, and the mapping results corresponding to the next scale are continued to be determined through the vector autoregression model, and finally the mapping results under the preset number of scales are obtained. For example, the mapping result obtained based on the initial image feature mapping is r1, then r1 is input into the vector autoregression model to obtain the mapping result corresponding to the next scale, and represented by r2. Then, the mapping results corresponding to all current scales, that is, r1 and r2, are simultaneously input into the vector autoregression model to obtain the mapping result corresponding to the next scale, and so on, to obtain the mapping results corresponding to multiple scales. Further, the decoder stage is entered to decode all the mapping results, that is, the mapping results corresponding to the preset number of scales, to obtain the multi-scale prior features corresponding to the traffic scene graph, and use S v express, , kRepresents a preset number, where the multi-scale prior features include image feature representations from low resolution to high resolution, and in the above-mentioned autoregressive generation process, they have been gradually aligned to the distribution of clean images, providing useful prior knowledge for the subsequent restoration of the clarity of the traffic scene graph.

[0030] S103: Fusing the multi-scale prior features with the degraded features in the traffic scene graph to obtain fused features.

[0031] Specifically, degraded features in traffic scene graphs refer to features that have degraded quality, information loss, or reduced resolution due to various reasons. An optimized scene graph is a traffic scene graph that has been optimized for clarity. After determining the multi-scale prior features, the traffic scene graph is input into a preset residual block to extract the degraded features. A residual block is a convolutional neural network structure widely used in deep learning and has significant advantages in extracting image degradation features. Each residual block consists of two convolutional layers (for feature extraction) and a residual connection (to address the vanishing and exploding gradient problems). Furthermore, the degraded features are input into a preset weight predictor, where they are processed using a self-attention mechanism and a feedforward neural network to produce processed features. A weight predictor is a tool or algorithm used in machine learning and data modeling to estimate and predict weight parameters.

[0032] The processed features are mapped through the preset projection convolution layer to obtain the corresponding mapped weights, and there are multiple mapped weights. Then the multi-scale prior features are weighted by the mapped weights, that is, Sv middle The weighted features are weighted and summed to obtain the weighted prior features. Finally, the lightweight projection head is used to transform the dimension of the weighted prior features to obtain the processed prior features, which are expressed as ,in, represents a lightweight projection head, j represents the jth scale, represents the prior features corresponding to the j-th scale, Represents the weight after mapping.

[0033] Furthermore, the processed prior features and the previous optimization results are substituted into the preset weight calculation formula to obtain the first weight and the second weight. In the embodiment of the present application, the process of restoring the clarity of the traffic scene graph is to fuse the degraded features in the traffic scene graph with the multi-scale prior features. After fusion, in order to alleviate the problems of structural warping and texture distortion caused by the direct fusion of high-quality prior features and low-quality degraded features corresponding to the traffic scene graph, it is also necessary to perform fusion optimization processing on the fused features, and repeat the feature fusion and post-fusion fusion optimization process multiple times. Therefore, the previous optimization result is the result of the last fusion and optimization of the degraded features and multi-scale prior features in the traffic scene graph. In addition, the weight calculation formula is: ; Where concat represents the concatenation operation, RSTBs represents a series of residual Swin-Transformer blocks, conv represents the convolutional layer, and softmax represents the softmax activation layer. represents the first weight, represents the second weight, Indicates the previous optimization result. Represents the prior features after processing, and i represents the i-th feature fusion.

[0034] Furthermore, the previous optimization result and the processed prior features are weighted and summed by the first weight and the second weight to obtain the fused features, which are specifically expressed as follows: , where Indicates the fused features of this feature fusion.

[0035] S104: Perform fusion optimization processing on the fused features to obtain an optimized scene graph corresponding to the traffic scene graph.

[0036] Specifically, after determining the fused features, it is necessary to perform fusion optimization on the fused features to alleviate the problems of structural warping and texture distortion caused by the direct fusion of high-quality prior features and low-quality degraded features corresponding to the traffic scene graph, so as to optimize and restore the clarity of the traffic scene graph. In the embodiment of the present application, a feasible processing method is to splice the previous optimization result and the fused features to obtain a spliced result, that is, Then, the previous optimization result is projected and transformed to obtain the first transformation result, that is, Projection to ; Perform projection transformation on the splicing result to obtain the second transformation result, that is, Projection to ; Perform projection transformation on the fused features to obtain the third transformation result and the fourth transformation result, that is, Projection to K= and Among them, W q 、W m 、W k 、W v Finally, based on the determined first, second, third, and fourth transformation results, the fused features are fused and optimized through a double-layer attention mechanism to obtain the optimized result. The specific process is described as follows: Where Q represents the first transformation result, M represents the second transformation result, K represents the third transformation result, V represents the fourth transformation result, T represents the transpose, and d represents the scalable training factor. The optimization result is used as the previous optimization result for the next feature fusion and fusion optimization process.

[0037] Finally, the step of inputting the traffic scene graph into the preset residual block in step S103 is repeated, and the degradation features are extracted again. Based on the mapped weights after the degradation feature mapping, the multi-scale prior features are weighted and dimensionally transformed to obtain the processed prior features. The processed prior features and the previous optimization results are weighted and summed, and the multi-scale prior features and the degradation features in the traffic scene graph are once again fused to obtain the fused features. Finally, the fusion optimization process is performed based on the fused features and the previous optimization result. The preset number of times is repeated to obtain the final optimization result. The final optimization result is passed through a 3x3 convolution layer to obtain an optimized scene graph with better clarity optimization effect, that is, a scene graph after the traffic scene graph is optimized in clarity.

[0038] In other embodiments, based on the historical accident records of the current road, a first number of occurrences of road areas on the current road where lane-to-lane vehicle accidents have historically occurred is counted, and a first number of road areas are selected from each road area in descending order of the first number of occurrences to determine as target road areas, i.e., areas prone to lane-to-lane vehicle accidents. Historical time periods in which lane-to-lane vehicle accidents have occurred in a single target road area are obtained, and a second number of occurrences of each historical time period is counted, and a second number of historical time periods are selected from each historical time period in descending order of the second number of occurrences to determine as target time periods corresponding to the single target road area, i.e., time periods prone to lane-to-lane vehicle accidents. The historical accident records include the road areas on the current road where lane-to-lane vehicle accidents have historically occurred and the corresponding time periods.

[0039] Furthermore, a first weight is determined for each target road area. The first weight is the ratio of the first occurrence count of each target road area to the sum of the first occurrence counts of all target road areas. A second weight is then determined for each target time period corresponding to the target road area. The second weight is the ratio of the second occurrence count of a single target time period corresponding to the target road area to the sum of the second occurrence counts of all corresponding target time periods. Furthermore, the actual area currently located by the target vehicle is obtained. If the actual area is the target road area, the product of the first weight of this actual area and the second weight of the corresponding key time period is calculated. The larger the product, the greater the likelihood that the target vehicle will currently be involved in an inter-lane vehicle accident. The key time period is the target time period at the current time. Finally, a preset number corresponding to this product is determined from a preset number matching table. The larger the product and the preset number, the more accurate the generated lane centerline, the more likely it is to avoid inter-lane vehicle accidents, and thus improve the driving safety of the target vehicle. The number matching table includes different products and corresponding preset numbers, all of which are set based on human experience.

[0040] S105: Based on the optimized scene graph, generate at least one corresponding lane centerline through a preset Bezier deformation attention.

[0041] Specifically, Bézier Deformable Attention combines the properties of Bézier curves with the self-attention mechanism. Specifically, it uses Bézier curves to define the distribution or deformation of attention, allowing the attention mechanism to more flexibly adapt to the geometric shape or structural characteristics of the input data. After the optimized scene graph is determined, it is converted into perspective view features using a preset feature extraction function, using the SIFT function. The perspective view features are then aggregated and projected into a BEV feature map using a preset projection function, using LSS (Lift-Splat-Shoot). It should be noted that the BEV feature map is a widely used representation in the fields of autonomous driving and computer vision. It generates a feature map by projecting object information in three-dimensional space onto a bird's-eye view (BEV), facilitating subsequent processing and analysis.

[0042] Furthermore, the projection result is input into the Transformer decoder based on Bezier deformation attention to determine the final Bezier control point set generated by the last layer in the Transformer decoder. The final Bezier control point set is a set composed of multiple Bezier control points, and finally at least one lane centerline is generated based on the multiple Bezier control points. Among them, the Bezier control points are key parameters for defining and adjusting the shape of the Bezier curve. The direction, curvature and smoothness of the curve can be accurately controlled by the position and number of the control points. In the embodiment of the present application, the lane centerline can be regarded as a curve defined by Bezier control points. Based on the Bezier control points in the final Bezier control point set, the shape and curvature of the lane centerline can be accurately simulated, which helps to improve the road topology understanding ability of the target car. It should be noted that the lane centerline can be generated based on the Bezier control points in the final Bezier control point set using the preset Adobe Illustrator tool to help the target car understand the road topology. In other embodiments, the lane centerline can also be generated using the preset CorelDRAW tool.

[0043] Furthermore, a feasible way to determine the final Bezier control point set is to generate an initial Bezier control point set for the first query embedding of a single layer of the Transformer decoder through a multilayer perceptron and a sigmoid function. This initial Bezier control point set is the set of Bezier control points that define the shape of the lane centerline. For example, the initial Bezier control point set for the first query embedding of the first layer of the Transformer decoder can be generated by the following process: , The first query embedding represents the first layer. Query embedding is a technique that converts query information into a low-dimensional vector representation, used to describe and locate desired features. A multilayer perceptron (MLP) is a feedforward neural network model consisting of an input layer, an output layer, and at least one hidden layer.

[0044] Next, based on this first query embedding and the position embedding of the same layer of the Transformer decoder, the second query embedding of the same layer of the Bessel deformation attention is determined. The specific process is expressed as: , represents the first query embedding of the Transformer decoder layer l, represents the position embedding of the lth layer of the Transformer decoder, represents the second query embedding of the first layer of Bezier deformable attention. Next, based on the projection result, the second query embedding, and the initial set of Bezier control points, Bezier deformable attention is used to guide attention using the Bezier control points that define the shape of the lane centerline to obtain an output result, thereby accurately capturing the position and shape of the lane centerline involved in the traffic scene graph. Then, based on the output result and the position embedding, self-attention calculation is performed to obtain the calculation result, thereby optimizing the feature representation. The specific process is expressed as follows: , Represents the output result of the lth layer.

[0045] Finally, the calculation result of the current layer is used as the first query embedding of the next layer of the Transformer decoder, and the steps of generating the initial Bezier control point set of the first query embedding of the single layer of the Transformer decoder through the multi-layer perceptron and sigmoid function are repeated until the final Bezier control point set generated by the last layer of the Transformer decoder is obtained. It should be noted that when repeated, the calculation process of the Bezier control point set of the first query embedding of the next layer is: , where represents the set of Bezier control points of the first query embedding at layer l, represents the set of Bezier control points of the first query embedding of the layer above layer l.

[0046] The implementation principle of the road topology understanding method in the embodiment of the present application is as follows: after obtaining the traffic scene graph of the target car during driving, image features of different scales aligned to the clean image are extracted from the traffic scene graph, and then high-quality prior image features corresponding to this traffic scene graph are obtained, namely multi-scale prior features, which facilitate the subsequent provision of useful prior knowledge for restoring and optimizing the image quality of the traffic scene graph. Furthermore, the multi-scale prior features are fused with the degradation features, so that under the guidance of the prior knowledge, the degradation domain in the traffic scene graph is promoted to be converted to the clean domain, thereby optimizing the blurred and poor-quality areas in the traffic scene graph. The fused features are then subjected to fusion optimization processing, thereby alleviating the problems of structural warping and texture distortion caused by the direct fusion of high-quality prior features and low-quality degradation features, thereby obtaining an optimized scene graph with better clarity optimization effect, and finally, based on this optimized scene graph, the lane centerline is generated more accurately, thereby improving the road topology understanding ability of the car.

[0047] The following are system embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the system embodiments of the present application, please refer to the method embodiments of the present application.

[0048] See Figure 2, which is a schematic diagram of the structure of the road topology understanding system provided in an embodiment of the present application. This road topology understanding system can be implemented as all or part of the system through software, hardware, or a combination of both. The system includes an information acquisition module 11, a priori determination module 12, a feature fusion module 13, an image optimization module 14, and a topology understanding module 15.

[0049] The information acquisition module 11 is used to obtain a traffic scene diagram of the target vehicle during its driving process; A priori determination module 12 is used to determine corresponding multi-scale prior features based on the traffic scene graph, where the multi-scale prior features are image features of different scales aligned to the clean image; A feature fusion module 13 is used to fuse the multi-scale prior features and the degraded features in the traffic scene graph to obtain fused features; An image optimization module 14 is configured to perform fusion optimization processing on the fused features to obtain an optimized scene graph corresponding to the traffic scene graph, wherein the optimized scene graph is a scene graph after the clarity of the traffic scene graph is optimized; The topology understanding module 15 is configured to generate at least one corresponding lane centerline based on the optimized scene graph through a preset Bezier deformation attention.

[0050] Optionally, the priori determination module 12 is specifically configured to: The initial image features are extracted from the traffic scene graph through the preset convolutional layer and mapped to the pre-trained VQVAE codebook to obtain the mapping results of a single scale; Inputting the mapping results of all current scales into a preset vector autoregression model to obtain the mapping results of the next scale, and re-performing the step of inputting the mapping results of all current scales into the preset vector autoregression model to obtain the mapping results at a preset number of scales; All mapping results are decoded to obtain multi-scale prior features corresponding to the traffic scene graph.

[0051] Optionally, the feature fusion module 13 is specifically configured to: Input the traffic scene graph into the preset residual block to obtain the degradation features; The degraded features are input into the weight predictor to obtain the processed features, and the processed features are mapped through the preset projection convolution layer to obtain the mapped weights; Through the post-mapping weights, the multi-scale prior features are weighted and the dimension is transformed through a preset lightweight projection head to obtain the processed prior features; The result of the last fusion and optimization of the degraded features and the multi-scale prior features in the traffic scene graph is determined as the previous optimization result, and the processed prior features and the previous optimization result are substituted into the preset weight calculation formula to obtain the first weight and the second weight; Based on the first weight and the second weight, a weighted sum is performed on the processed prior features and the previous optimization result to obtain the fused features.

[0052] Optionally, the image optimization module 14 is specifically configured to: Splice the previous optimization result and the fused features to obtain the splicing result; Performing a projection transformation on the previous optimization result to obtain a first transformation result, and performing a projection transformation on the splicing result to obtain a second transformation result; Performing projection transformation on the fused features to obtain a third transformation result and a fourth transformation result, and performing fusion optimization processing on the fused features through a double-layer attention mechanism based on the first transformation result, the second transformation result, the third transformation result and the fourth transformation result to obtain an optimization result; Repeat the step of inputting the traffic scene graph into the preset residual block to obtain an optimized scene graph corresponding to the traffic scene graph.

[0053] Optionally, the topology understanding module 15 is specifically configured to: Convert the optimized scene graph into perspective view features through a preset feature extraction function; Through the preset projection function, the perspective view features are aggregated and projected into the BEV feature map to obtain the projection result; The projection result is input into the Transformer decoder based on Bessel deformation attention to obtain the final Bessel control point set generated by the last layer of the Transformer decoder; Based on the final set of Bezier control points, at least one lane centerline corresponding to the optimized scene graph is generated.

[0054] Optionally, the topology understanding module 15 is specifically configured to: Generate the initial set of Bezier control points for the first query embedding of a single layer of the Transformer decoder through a multilayer perceptron and a sigmoid function; Based on the first query embedding and the position embedding of the same layer of the Transformer decoder, determine the second query embedding of the same layer of the Bessel deformation attention; Based on the projection result, the second query embedding and the initial Bezier control point set, the output result is obtained by using Bezier deformation attention; Based on the output results and position embedding, self-attention calculation is performed to obtain the calculation results; The calculation result is used as the first query embedding of the next layer of the Transformer decoder, and the steps of generating the initial set of Bezier control points of the first query embedding of a single layer of the Transformer decoder through the multilayer perceptron and the sigmoid function are repeated to obtain the final set of Bezier control points generated by the last layer of the Transformer decoder.

[0055] It should be noted that the road topology understanding system provided in the above embodiment, when executing the road topology understanding method, is merely illustrated by the division of the aforementioned functional modules. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the road topology understanding system provided in the above embodiment and the road topology understanding method embodiment are based on the same concept. The implementation process is detailed in the method embodiment and will not be repeated here.

[0056] An embodiment of the present application discloses a computer program product, including a computer program. When the computer program is executed by a processor, a road topology understanding method of the above embodiment is implemented.

[0057] An embodiment of the present application further discloses a computer-readable storage medium, and the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, a road topology understanding method of the above embodiment is implemented.

[0058] Among them, the computer program can be stored in a computer-readable medium, the computer program includes computer program code, the computer program code can be in the form of source code, object code, executable file or certain middleware, etc. The computer-readable medium includes any entity or device that can carry computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that computer-readable medium includes but is not limited to the above-mentioned components.

[0059] Among them, through this computer-readable storage medium, a road topology understanding method of the above embodiment is stored in a computer-readable storage medium, and is loaded and executed on a processor to facilitate the storage and application of the above method.

[0060] An embodiment of the present application further discloses an electronic device, wherein a computer program is stored in a computer-readable storage medium. When the computer program is loaded and executed by a processor, the above-mentioned road topology understanding method is implemented.

[0061] Among them, the electronic device can be an electronic device such as a desktop computer, a laptop computer or a cloud server, and the electronic device includes but is not limited to a processor and a memory. For example, the electronic device can also include input and output devices, network access devices and buses, etc.

[0062] Among them, the processor can adopt a central processing unit (CPU). Of course, according to actual usage, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be adopted. The general-purpose processor can adopt a microprocessor or any conventional processor, etc., and this application does not impose any restrictions on this.

[0063] Among them, the memory can be an internal storage unit of the electronic device, such as the hard disk or memory of the electronic device, or it can be an external storage device of the electronic device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD) or flash memory card (FC) equipped on the electronic device. In addition, the memory can also be a combination of an internal storage unit and an external storage device of the electronic device. The memory is used to store computer programs and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or is to be output. This application does not impose any restrictions on this.

[0064] Among them, through this electronic device, a road topology understanding method of the above embodiment is stored in the memory of the electronic device, and is loaded and executed on the processor of the electronic device for easy use.

[0065] The above description is merely an exemplary embodiment of the present disclosure and is not intended to limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A road topology understanding method, characterized in that: The method comprises: Obtain a traffic scene graph of the target car while it is driving; Determining corresponding multi-scale prior features based on the traffic scene graph, where the multi-scale prior features are image features of different scales aligned to a clean image; fusing the multi-scale prior features with the degraded features in the traffic scene graph to obtain fused features; Performing fusion optimization processing on the fused features to obtain an optimized scene graph corresponding to the traffic scene graph, wherein the optimized scene graph is a scene graph after the traffic scene graph is optimized in terms of clarity; Based on the optimized scene graph, at least one corresponding lane centerline is generated through a preset Bezier deformation attention.

2. The road topology understanding method according to claim 1, characterized in that: The determining of corresponding multi-scale prior features based on the traffic scene graph specifically includes: Extracting initial image features from the traffic scene graph through a preset convolutional layer, and mapping the initial image features to a pre-trained VQVAE codebook to obtain a mapping result at a single scale; Inputting the mapping results of all current scales into a preset vector autoregression model to obtain a mapping result of the next scale, and re-performing the step of inputting the mapping results of all current scales into the preset vector autoregression model to obtain mapping results at a preset number of scales; All the mapping results are decoded to obtain multi-scale prior features corresponding to the traffic scene graph.

3. The road topology understanding method according to claim 1, characterized in that: The step of fusing the multi-scale prior features with the degraded features in the traffic scene graph to obtain fused features specifically includes: Inputting the traffic scene graph into a preset residual block to obtain degradation features; Inputting the degradation features into a weight predictor to obtain processed features, and mapping the processed features through a preset projection convolution layer to obtain mapped weights; The multi-scale prior features are weighted by the mapped weights and dimensionality transformed by a preset lightweight projection head to obtain processed prior features; Determining the result of the last fusion and optimization of the degraded features in the traffic scene graph and the multi-scale prior features as the last optimization result, and substituting the processed prior features and the last optimization result into a preset weight calculation formula to obtain a first weight and a second weight; Based on the first weight and the second weight, a weighted sum is performed on the processed prior features and the previous optimization result to obtain a fused feature.

4. The road topology understanding method according to claim 3, characterized in that: The weight calculation formula is: ； Where concat represents the concatenation operation, RSTBs represents a series of residual Swin-Transformer blocks, conv represents the convolutional layer, and softmax represents the softmax activation layer. represents the first weight, represents the second weight, Indicates the previous optimization result. Represents the processed prior features.

5. The road topology understanding method according to claim 3, characterized in that: The performing fusion optimization processing on the fused features to obtain an optimized scene graph corresponding to the traffic scene graph specifically includes: Splicing the previous optimization result and the fused features to obtain a splicing result; Performing a projection transformation on the previous optimization result to obtain a first transformation result, and performing a projection transformation on the splicing result to obtain a second transformation result; Performing a projection transformation on the fused features to obtain a third transformation result and a fourth transformation result, and performing a fusion optimization process on the fused features through a double-layer attention mechanism based on the first transformation result, the second transformation result, the third transformation result, and the fourth transformation result to obtain an optimization result; Repeat the step of inputting the traffic scene graph into a preset residual block to obtain an optimized scene graph corresponding to the traffic scene graph.

6. The road topology understanding method according to claim 1, characterized in that: The step of generating at least one corresponding lane centerline based on the optimized scene graph by using a preset Bezier deformation attention method specifically includes: Converting the optimized scene graph into perspective view features through a preset feature extraction function; Aggregating the perspective view features and projecting them into a BEV feature map using a preset projection function to obtain a projection result; Inputting the projection result into a Transformer decoder based on Bessel deformation attention to obtain the final Bessel control point set generated by the last layer of the Transformer decoder; At least one lane centerline corresponding to the optimized scene graph is generated based on the final Bezier control point set.

7. The road topology understanding method according to claim 6, characterized in that: Inputting the projection result into the Transformer decoder based on Bessel deformation attention to obtain the final Bessel control point set generated by the last layer of the Transformer decoder specifically includes: Generate an initial set of Bezier control points for the first query embedding of a single layer of the Transformer decoder using a multilayer perceptron and a sigmoid function; Determining a second query embedding for the same layer of the Bessel deformed attention based on the first query embedding and the position embedding of the same layer of the Transformer decoder; Obtaining an output result using Bezier deformation attention based on the projection result, the second query embedding, and the initial Bezier control point set; Performing a self-attention calculation based on the output result and the position embedding to obtain a calculation result; The calculation result is used as the first query embedding of the next layer of the Transformer decoder, and the steps of generating an initial Bezier control point set of the first query embedding of the single layer of the Transformer decoder through the multilayer perceptron and the sigmoid function are repeated to obtain a final Bezier control point set generated by the last layer of the Transformer decoder.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor loads and executes the computer program, the method according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium storing a computer program, wherein: When the computer program is loaded and executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images