Traffic map generation method and device, equipment, storage medium and program product

Abstract

The application provides a traffic map generation method, device and equipment, a storage medium and a program product, and is applied to the technical field of image processing. The method comprises the following steps: acquiring image features of a traffic scene image, wherein the image features are multi-scale feature maps; determining ground element information and a traffic sign bounding box in the traffic scene image according to the image features, and determining component information in the traffic sign bounding box; determining relationship attributes between elements in the ground element information and the component information; and generating a traffic map based on the ground element information, the component information and the relationship attributes.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a traffic map generation method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the rapid development of technology, intelligent transportation systems are playing an increasingly important role. Through real-time data analysis and intelligent management, intelligent transportation systems can optimize traffic flow, provide real-time traffic information, thereby improving traffic efficiency, reducing traffic accidents, saving energy, and improving the quality of life for urban residents. Environmental perception is a fundamental and crucial technology in intelligent transportation systems, providing important information for subsequent intelligent decision-making and planning control. Summary of the Invention

[0003] This invention provides a traffic map generation method, apparatus, device, storage medium, and program product to solve the problem of environmental perception in intelligent transportation systems.

[0004] This invention provides a traffic map generation method, comprising: acquiring image features of a traffic scene image, wherein the image features are multi-scale feature maps; determining ground element information and traffic sign bounding boxes in the traffic scene image based on the image features, and determining component information within the traffic sign bounding boxes; determining the relationship attributes between the elements in the ground element information and the component information; and generating a traffic map based on the ground element information, the component information, and the relationship attributes.

[0005] According to a traffic map generation method provided by the present invention, the step of determining ground element information and traffic sign bounding boxes in the traffic scene image based on the image features, and determining component information within the traffic sign bounding boxes, includes: inputting the multi-scale feature map into a macroscopic perception module to obtain the ground element information and the traffic sign bounding boxes, wherein the ground element information includes road masks and lane masks; inputting the traffic sign bounding boxes into a microscopic perception module to obtain the component information, wherein the component information includes component category and component location; wherein both the macroscopic perception module and the microscopic perception module employ a Transformer decoder structure.

[0006] According to a traffic map generation method provided by the present invention, the decoder of the macro-perception module is used to: perform feature update on the macro query through a multi-head self-attention mechanism; transform the updated macro query through a feedforward network to obtain a target macro query; and perform mask prediction based on the target macro query.

[0007] According to a traffic map generation method provided by the present invention, each query target in the decoder of the micro-perception module corresponds to an anchor box, and the anchor box is embedded as a position feature through sinusoidal position encoding to assist the decoder of the micro-perception module in performing self-attention and cross-attention fusion.

[0008] According to a traffic map generation method provided by the present invention, the relational attributes include at least one of the following: complementary relationship between traffic signs, associative relationship between traffic sign components, directional relationship between traffic sign components, correspondence between traffic signs and ground elements, and correspondence between traffic sign components and ground elements.

[0009] The present invention also provides a traffic map generation device, comprising the following modules: an acquisition module and a processing module; the acquisition module is used to acquire image features of a traffic scene image, wherein the image features are multi-scale feature maps; the processing module is used to determine ground element information and traffic sign bounding boxes in the traffic scene image based on the image features, and determine component information within the traffic sign bounding boxes; determine the relationship attributes between the elements in the ground element information and the component information; and generate a traffic map based on the ground element information, the component information, and the relationship attributes.

[0010] According to a traffic map generation device provided by the present invention, the processing module is used to input the multi-scale feature map into a macroscopic perception module to obtain the ground element information and the traffic sign bounding box, wherein the ground element information includes road mask and lane mask; and input the traffic sign bounding box into a microscopic perception module to obtain the component information, wherein the component information includes component category and component location; wherein both the macroscopic perception module and the microscopic perception module adopt a Transformer decoder structure.

[0011] According to a traffic map generation device provided by the present invention, the decoder of the macroscopic perception module is used to: perform feature update on the macro query through a multi-head self-attention mechanism; transform the updated macro query through a feedforward network to obtain a target macro query; and perform mask prediction based on the target macro query.

[0012] According to a traffic map generation device provided by the present invention, each query target in the decoder of the micro-sensing module corresponds to an anchor box, and the anchor box is embedded as a position feature through sinusoidal position encoding to assist the decoder of the micro-sensing module in performing self-attention and cross-attention fusion.

[0013] According to a traffic map generation device provided by the present invention, the relational attributes include at least one of the following: complementary relationship between traffic signs, associated relationship between traffic sign components, directional relationship between traffic sign components, correspondence between traffic signs and ground elements, and correspondence between traffic sign components and ground elements.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the traffic map generation method as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the traffic map generation method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the traffic map generation method as described above.

[0017] The traffic map generation method, apparatus, device, storage medium, and program product provided by this invention can acquire image features of traffic scene images, wherein the image features are multi-scale feature maps; determine ground element information and traffic sign bounding boxes in the traffic scene image based on the image features, and determine component information within the traffic sign bounding boxes; determine the relationship attributes between the elements in the ground element information and the component information; and generate a traffic map based on the ground element information, the component information, and the relationship attributes. This scheme allows for the analysis of traffic scene images, extraction of information from traffic signs and ground elements, and the organization of traffic maps by combining the relationships between the elements. Since the traffic map contains key element features of the traffic scene image, it enables the extraction of image information, thus finding wide application in fields such as intelligent navigation, positioning assistance, and map correction. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the traffic map generation method provided by the present invention;

[0020] Figure 2 This is a schematic diagram of the processing flow of the macroscopic sensing module provided by the present invention;

[0021] Figure 3 This is a schematic diagram of the processing flow of the microscopic sensing module provided by the present invention;

[0022] Figure 4 This is a schematic diagram of the processing flow of the relational reasoning module provided by the present invention;

[0023] Figure 5 This is a schematic diagram of the traffic map structure provided by the present invention;

[0024] Figure 6 This is a schematic diagram of the traffic map generation device provided by the present invention;

[0025] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0028] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0029] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0030] This application describes some exemplary embodiments for illustrative purposes. It should be understood that this application may be implemented in other ways not specifically shown in the accompanying drawings.

[0031] like Figure 1 As shown, this application provides a traffic map generation method, which can be applied to a traffic map generation device. The traffic map generation method may include steps S101-S104:

[0032] S101, The traffic map generation device acquires image features of traffic scene images.

[0033] Specifically, the traffic map generation device can first acquire traffic scene images captured by vehicle-mounted cameras or handheld cameras, and then extract image features from the traffic scene images through a feature extraction network.

[0034] Optionally, the aforementioned feature extraction network employs a network structure combining Convolutional Neural Networks (CNN) and Transformer. The CNN utilizes the ResNet-50 model, while the Transformer functions similarly to a feature pyramid network, extracting multi-scale feature maps composed of features at four scales, denoted as... These correspond to 1 / 32, 1 / 16, 1 / 8, and 1 / 4 of the resolution of traffic scene images, respectively.

[0035] S102. The traffic map generation device determines the ground element information and traffic sign bounding boxes in the traffic scene image based on the image features, and determines the component information in the traffic sign bounding boxes.

[0036] Optionally, determining the ground element information and traffic sign bounding boxes in the traffic scene image based on the image features, and determining the component information within the traffic sign bounding boxes, includes: inputting the multi-scale feature map into a macroscopic perception module to obtain the ground element information and the traffic sign bounding boxes, wherein the ground element information includes road masks and lane masks; inputting the traffic sign bounding boxes into a microscopic perception module to obtain the component information, wherein the component information includes component category and component location; wherein both the macroscopic perception module and the microscopic perception module employ a Transformer decoder structure.

[0037] The macroscopic perception module will be explained in detail below:

[0038] The macroscopic perception module is used to segment ground elements (i.e., roads and lanes) and detect traffic signs. It is a module capable of simultaneously performing object detection and segmentation. The macroscopic perception module includes a Transformer decoder, which can be used to: perform feature updates on macro queries through a multi-head self-attention mechanism; transform the updated macro queries through a feedforward network to obtain the target macro query; and perform mask prediction based on the target macro query.

[0039] Specifically, such as Figure 2 As shown, the macroscopic detection module is a Transformer decoder, which consists of N decoder layers with identical structures. The macro query of the macroscopic detection module can be represented as... ,in, 'D' represents the total number of queries, and 'D' represents the dimension. The macroscopic detection module uses the aforementioned multi-scale feature maps. As input, in order to utilize multi-scale features and achieve more refined segmentation results, the Transformer decoder can progressively update features, from small-scale, semantically rich features... It begins by extending to larger-scale, more detailed features through its decoder layer. That is, the input of the first layer is The second layer input is The third layer input is The Transformer decoder's first The input of the layer is ,in, for The remainder when divided by 3, for example, when When it is 5 o'clock, The value is 2, and the input of the fifth layer is... For the Transformer decoder's... Layer, input features It is used to perform multi-head cross attention (MHCA) with macro queries:

[0040] ;

[0041] in, It is the first Output query of layer decoder It is a macro query The predicted attention mask enables each query to extract local features within the foreground region of the predicted mask, rather than focusing on the entire feature map.

[0042] Subsequently, the Transformer decoder uses a multi-head self-attention (MHSA) mechanism to perform feature updates on the macro query:

[0043] ;

[0044] Updated macro query Then through the feedforward network Perform the transformation:

[0045] ;

[0046] Finally, the mask to be predicted is obtained through macro queries. and input features Make a prediction:

[0047] ;

[0048] in, , and These represent the height and width of the input image, respectively, and the categories are determined by the fully connected layer. It is confirmed that the bounding box is composed of a fully connected layer. Sure, The number of categories is as follows: there are 8 categories in total, namely current road, left road, right road, road ahead, lane, circular traffic sign, triangular traffic sign, and rectangular traffic sign.

[0049] During training, the macroscopic perception module is used to segment and regress all roads, lanes, and traffic signs. The output of each layer is first binary-matched with the ground truth to calculate the loss for classification, regression, and segmentation. During testing, this application only retains the road masks, lane masks, and traffic sign bounding boxes.

[0050] The microscopic sensing module will be described in detail below:

[0051] The micro-perception module is used to locate the quadrilateral bounding boxes of components (text, symbols, and arrowheads) inside the bounding box of a traffic sign. The micro-perception module includes a Transformer decoder, in which each query target corresponds to an anchor box. These anchor boxes are embedded as positional features using sinusoidal position encoding to assist the decoder in fusing self-attention and cross-attention.

[0052] Specifically, such as Figure 3 As shown, the microscopic detection module is a Transformer decoder, which is also composed of N decoder layers with the same structure. The macro query of the microscopic detection module can be represented as... ,in, This represents the total number of queries, and D represents the dimension. The micro-detection module uses RoIAlign technology to... The internal features of the traffic sign bounding boxes are extracted, ensuring that each micro-query interacts only with features within the sign's bounding box. All sign features are scaled to a fixed size and represented as... All in All micro-queries are initialized with the same learnable parameters. Meanwhile, to leverage explicit location priors to improve the similarity of query features, the micro-awareness module sets a quadrilateral frame as an anchor frame for each query target. , among which, the An anchor box can be represented as:

[0053] ;

[0054] Subsequently, these anchor boxes are embedded as positional features using sinusoidal positional encoding, thereby assisting in the fusion of self-attention and cross-attention for micro-queries. For the first... Layer decoder, this process is represented as:

[0055] ;

[0056] ;

[0057] ;

[0058] in, , and Represents respectively in Micro-query representations at different processing stages within the layer.

[0059] Ultimately, all anchor boxes will be based on microqueries. Update the predicted offset:

[0060] ;

[0061] in, .

[0062] The output anchor box of the last layer of the microscopic perception module decoder is the component location. The component category comes from the classification result of a fully connected layer classifier. In this application, the component categories include 65 types.

[0063] S103, The traffic map generation device determines the relationship attributes between the elements in the ground element information and the component information.

[0064] Optionally, the relational attributes include at least one of the following: complementary relationships between traffic signs, associative relationships between traffic sign components, directional relationships between traffic sign components, correspondence between traffic signs and ground elements, and correspondence between traffic sign components and ground elements.

[0065] Specifically, the complementary relationship between traffic signs can be represented as SS, indicating that the two traffic signs complement each other; the associative relationship between traffic sign components and the directional relationship between traffic sign components can be represented as CC, where the associative relationship indicates that the two components refer to the same place, and the directional relationship indicates that the place referred to by the components is located in the direction indicated by a certain arrow; the correspondence between traffic signs and ground elements can be represented as ST, and the correspondence between traffic sign components and ground elements can be represented as CT.

[0066] Optionally, the traffic map generation device can predict the relationships between multiple elements and text attributes through a relationship reasoning module. Text attributes are text content labels that support visual traffic map generation, including five categories: road code, road name, place name, direction, and description.

[0067] Specifically, such as Figure 4As shown, in the input graph structure, macro queries, micro queries, and edge queries are represented by squares, triangles, and circles, respectively. Dashed arrows represent SC links. Bold arrows with black fill represent dense ST links, while bold arrows with white fill represent sparse CT links. To achieve comprehensive context awareness and fusion, all elements and relationships are modeled as an input graph and updated through a Transformer decoder. In the input graph, all elements are treated as nodes, and the relationships between elements are treated as edges. Their features are represented as node queries. And edge query Thanks to the query prediction mechanism, the relational reasoning module can directly use the updated macro-queries and micro-queries as initial node queries. Because they already contain the category and location information of the corresponding elements. Therefore, the number of node queries It is the sum of effective macro queries and micro queries. For nodes and The links between them, and the edge query, is their sum vector:

[0068] .

[0069] For complex traffic scenarios with a large number of elements, constructing a fully connected graph would result in a significant computational burden, and some links would be meaningless and unnecessary. To address this issue, this module employs a hierarchical graph structure. For example... Figure 4 As shown, the diagram includes three subgraphs: the sign subgraph (S-subgraph), the component subgraph (C-subgraph), and the traffic subgraph (T-subgraph). Furthermore, three cross-subgraph links (SC, ST, CT) are created to facilitate communication between subgraphs. The relationships to be predicted include sign-to-sign (SS) relationships within the S-subgraph, component-to-component (CC) relationships within the C-subgraph, traffic sign-to-ground element (ST) relationships represented by the ST link, and component-to-ground element (CT) relationships represented by the CT link.

[0070] SS links in the S-subgraph are determined heuristically based on the relative distances and alignments between traffic signs. For example, signs that are far apart or spatially misaligned have no links. For the C-subgraph, all components of a single sign or all components of two signs with SS links are fully connected to infer CC relationships (relationships between components) within or between different signs. Thus, the S-subgraph and C-subgraph are constructed as sparse graphs. Since SS and CC relationships are undirected, they are represented by two opposite edges.

[0071] The T-subgraph contains only ground elements, namely roads and lanes. This subgraph uses dense connections to predict the membership and relative positional relationships between ground elements to assist in reasoning ST and CT relationships.

[0072] The SC link flows from the traffic sign to its components, indicated by a dashed line. Figure 4 The diagram provides a high-level understanding. ST links, indicated by bold arrows filled in black, are dense connections from each traffic sign to each ground element to determine ST relationships. CT links, indicated by bold arrows filled in white, are sparse connections that exist only between arrow elements (arrow symbols and arrowheads) and ground elements to determine CT relationships.

[0073] In the relational reasoning module, a message-passing mechanism is designed for the three subgraphs and their cross-graph links to improve reasoning efficiency. The relational reasoning module contains multiple network layers with identical structures. For the first... layer( ), the input node query is represented as The query is represented as ,in, and These are initial features.

[0074] The relation reasoning module first applies a self-attention fusion mechanism to S, C, and T-subgraphs. Since edge information is crucial for self-attention fusion between nodes, this module modifies the self-attention mechanism to fuse edge features. (Self-attention coefficient matrix) It can be obtained through the following equation:

[0075] ⊙ ;

[0076] ;

[0077] ;

[0078] in, and It is a fully connected layer. Through Edge features obtained from edge queries:

[0079] ;

[0080] Node queries are obtained through attention matrix fusion and a feedforward network:

[0081] ;

[0082] ;

[0083] in, .

[0084] The query is performed as follows:

[0085] ( ⊙ ) ;

[0086] Then, information is passed from the S-subgraph to the C-subgraph via the SC link. For the source node... and target node The information transmission process can be represented as follows:

[0087] ;

[0088] in, .

[0089] Finally, by concatenating and mapping the ST and CT links, the matching relationship features can be obtained:

[0090] ;

[0091] in, ; This represents the tensor concatenation operation.

[0092] The relational reasoning module uses a fully connected layer. Obtain the updated edge query representation of the relationship. This represents the number of relation categories. All node queries representing text are fed into another fully connected layer. To obtain the text properties.

[0093] S104. The traffic map generation device generates a traffic map based on the ground element information, the component information, and the relational attributes.

[0094] Specifically, such as Figure 5 As shown, the traffic map is composed of roads as the root node ( The traffic map generation device consists of a tree structure with other roads, lanes, and various attributes as leaf nodes. For each road, the traffic map generation device can extract information such as road name, direction of travel, destination, lane, and other descriptions. Based on the output results of the above modules, it uses the road as the root node and organizes all the extracted information into a traffic map according to the corresponding relationships.

[0095] In this embodiment, traffic scene images can be analyzed to extract information on traffic signs and ground elements, and a traffic map can be generated by combining the relationships between the elements. Since the traffic map contains key element features in the traffic scene image, image information can be extracted, which can then be widely used in fields such as intelligent navigation, positioning assistance, and map correction.

[0096] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0097] The traffic map generation method provided in this application can be executed by a traffic map generation device or a control module for traffic map generation within that device. This application uses the example of a traffic map generation device executing the traffic map generation method to illustrate the traffic map generation device provided in this application.

[0098] It should be noted that the embodiments of this application can divide the traffic map generation device into functional modules according to the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. Optionally, the module division in the embodiments of this application is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0099] like Figure 6 As shown in the figure, this application embodiment provides a traffic map generation device 600. The traffic map generation device 600 includes: an acquisition module 601 and a processing module 602. The acquisition module 601 is used to acquire image features of a traffic scene image, wherein the image features are multi-scale feature maps; the processing module 602 is used to determine ground element information and traffic sign bounding boxes in the traffic scene image based on the image features, and determine component information within the traffic sign bounding boxes; determine the relationship attributes between elements in the ground element information and the component information; and generate a traffic map based on the ground element information, the component information, and the relationship attributes.

[0100] Optionally, the processing module 602 is used to input the multi-scale feature map into the macroscopic perception module to obtain the ground element information and the traffic sign bounding box, wherein the ground element information includes road mask and lane mask; and input the traffic sign bounding box into the microscopic perception module to obtain the component information, wherein the component information includes component category and component location; wherein both the macroscopic perception module and the microscopic perception module adopt a Transformer decoder structure.

[0101] Optionally, the decoder of the macroscopic perception module is used to: perform feature updates on the macro query through a multi-head self-attention mechanism; transform the updated macro query through a feedforward network to obtain a target macro query; and perform mask prediction based on the target macro query.

[0102] Optionally, each query target in the decoder of the micro-perception module corresponds to an anchor box, which is embedded as a position feature through sinusoidal position encoding to assist the decoder of the micro-perception module in performing the fusion of self-attention and cross-attention.

[0103] Optionally, the relational attributes include at least one of the following: complementary relationships between traffic signs, associative relationships between traffic sign components, directional relationships between traffic sign components, correspondence between traffic signs and ground elements, and correspondence between traffic sign components and ground elements.

[0104] In this embodiment, traffic scene images can be analyzed to extract information on traffic signs and ground elements, and a traffic map can be generated by combining the relationships between the elements. Since the traffic map contains key element features in the traffic scene image, image information can be extracted, which can then be widely used in fields such as intelligent navigation, positioning assistance, and map correction.

[0105] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a traffic map generation method. This method includes: acquiring image features of a traffic scene image, wherein the image features are multi-scale feature maps; determining ground element information and traffic sign bounding boxes in the traffic scene image based on the image features, and determining component information within the traffic sign bounding boxes; determining the relationship attributes between the elements in the ground element information and the component information; and generating a traffic map based on the ground element information, the component information, and the relationship attributes.

[0106] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the traffic map generation method provided by the above methods. The method includes: acquiring image features of a traffic scene image, wherein the image features are multi-scale feature maps; determining ground element information and traffic sign bounding boxes in the traffic scene image based on the image features, and determining component information in the traffic sign bounding boxes; determining the relationship attributes between the elements in the ground element information and the component information; and generating a traffic map based on the ground element information, the component information, and the relationship attributes.

[0108] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the traffic map generation method provided by the above methods. The method includes: acquiring image features of a traffic scene image, wherein the image features are multi-scale feature maps; determining ground element information and traffic sign bounding boxes in the traffic scene image based on the image features, and determining component information within the traffic sign bounding boxes; determining the relationship attributes between the elements in the ground element information and the component information; and generating a traffic map based on the ground element information, the component information, and the relationship attributes.

[0109] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0110] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating traffic maps, characterized in that, include: Acquire image features of a traffic scene image, wherein the image features are multi-scale feature maps; Based on the image features, determine the ground element information and traffic sign bounding boxes in the traffic scene image, and determine the component information within the traffic sign bounding boxes; Determine the relationship attributes between the elements in the ground element information and the component information; A traffic map is generated based on the ground element information, the component information, and the relationship attributes; The step of determining the ground element information and traffic sign bounding boxes in the traffic scene image based on the image features, and determining the component information within the traffic sign bounding boxes, includes: The multi-scale feature map is input into the macroscopic perception module to obtain the ground element information and the traffic sign bounding box. The ground element information includes road mask and lane mask. The traffic sign bounding box is input into the microscopic perception module to obtain the component information, which includes the component category and component location. Both the macroscopic sensing module and the microscopic sensing module adopt a Transformer decoder structure. The decoder of the macroscopic sensing module is used for: Macro queries are updated using a multi-head self-attention mechanism; The updated macro query is transformed using a feedforward network to obtain the target macro query; Perform mask prediction based on the target macro query; Each query target in the decoder of the micro-perception module corresponds to an anchor box, which is embedded as a position feature through sinusoidal position encoding to assist the decoder of the micro-perception module in performing the fusion of self-attention and cross-attention.

2. The traffic map generation method according to claim 1, characterized in that, The relational attributes include at least one of the following: complementary relationships between traffic signs, associative relationships between traffic sign components, directional relationships between traffic sign components, correspondence between traffic signs and ground elements, and correspondence between traffic sign components and ground elements.

3. A traffic map generation device, characterized in that, include: Acquisition module and processing module; The acquisition module is used to acquire image features of traffic scene images, wherein the image features are multi-scale feature maps; The processing module is used to determine the ground element information and traffic sign bounding boxes in the traffic scene image based on the image features, and to determine the component information in the traffic sign bounding boxes; Determine the relationship attributes between the elements in the ground element information and the component information; A traffic map is generated based on the ground element information, the component information, and the relationship attributes; The processing module is used to input the multi-scale feature map into the macroscopic perception module to obtain the ground element information and the traffic sign bounding box, wherein the ground element information includes road mask and lane mask; and input the traffic sign bounding box into the microscopic perception module to obtain the component information, wherein the component information includes component category and component location; wherein both the macroscopic perception module and the microscopic perception module adopt the Transformer decoder structure. The decoder of the macroscopic sensing module is used for: Macro queries are updated using a multi-head self-attention mechanism; The updated macro query is transformed using a feedforward network to obtain the target macro query; Perform mask prediction based on the target macro query; Each query target in the decoder of the micro-perception module corresponds to an anchor box, which is embedded as a position feature through sinusoidal position encoding to assist the decoder of the micro-perception module in performing the fusion of self-attention and cross-attention.

4. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the traffic map generation method as described in any one of claims 1 to 2.

5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the traffic map generation method as described in any one of claims 1 to 2.

6. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the traffic map generation method as described in any one of claims 1 to 2.

Images