Traffic map generation method and device, equipment, storage medium and program product

Abstract

The invention provides a traffic map generation method and device, equipment, a storage medium and a program product, and is applied to the technical field of image processing. The method comprises the following steps: acquiring image features of a traffic scene image, wherein the image features are multi-scale feature maps; determining ground element information and a traffic sign bounding box in the traffic scene image according to the image features, and determining component information in the traffic sign bounding box; determining relation attributes among elements in the ground element information and the component information; and generating a traffic map based on the ground element information, the component information and the relationship attribute.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a traffic map generation method, device, equipment, storage medium and program product. Background Art

[0002] With the rapid development of science and technology, intelligent transportation systems are playing an increasingly important role. Through real-time data analysis and intelligent management, intelligent transportation systems can optimize traffic flow and provide real-time traffic information, thereby improving traffic efficiency, reducing traffic accidents, saving energy, and improving the quality of life of urban residents. In intelligent transportation systems, environmental perception is a basic and key technology that can provide important information for subsequent intelligent decision-making and planning control. Summary of the invention

[0003] The present invention provides a traffic map generation method, device, equipment, storage medium and program product to solve the problem of environmental perception in an intelligent transportation system.

[0004] The present invention provides a method for generating a traffic map, comprising: obtaining image features of a traffic scene image, wherein the image features are multi-scale feature maps; determining ground element information and a traffic sign boundary box in the traffic scene image according to the image features, and determining component information in the traffic sign boundary box; determining the ground element information and the relationship attributes between the elements in the component information; and generating a traffic map based on the ground element information, the component information and the relationship attributes.

[0005] According to a traffic map generation method provided by the present invention, the method of determining the ground element information and the traffic sign bounding box in the traffic scene image according to the image features, and determining the component information in the traffic sign bounding box, includes: inputting the multi-scale feature map into a macro perception module to obtain the ground element information and the traffic sign bounding box, the ground element information includes a road mask and a lane mask; inputting the traffic sign bounding box into a micro perception module to obtain the component information, the component information includes a component category and a component position; wherein the macro perception module and the micro perception module both adopt a Transformer decoder structure.

[0006] According to a traffic map generation method provided by the present invention, the decoder of the macro perception module is used to: perform feature update on the macro query through a multi-head self-attention mechanism; transform the updated macro query through a feedforward network to obtain a target macro query; and perform mask prediction based on the target macro query.

[0007] According to a traffic map generation method provided by the present invention, each query target in the decoder of the micro perception module corresponds to an anchor frame, and the anchor frame is embedded as a position feature through sinusoidal position encoding, which is used to assist the decoder of the micro perception module to fuse self-attention and cross-attention.

[0008] According to a traffic map generation method provided by the present invention, the relationship attributes include at least one of the following: a complementary relationship between traffic signs, an association relationship between traffic sign components, a directional relationship between traffic sign components, a corresponding relationship between traffic signs and ground elements, and a corresponding relationship between traffic sign components and ground elements.

[0009] The present invention also provides a traffic map generating device, comprising the following modules: an acquisition module and a processing module; the acquisition module is used to acquire image features of a traffic scene image, wherein the image features are multi-scale feature maps; the processing module is used to determine ground element information and a traffic sign boundary box in the traffic scene image according to the image features, and determine component information in the traffic sign boundary box; determine the ground element information and the relationship attributes between the elements in the component information; and generate a traffic map based on the ground element information, the component information and the relationship attributes.

[0010] According to a traffic map generation device provided by the present invention, the processing module is used to input the multi-scale feature map into a macro perception module to obtain the ground element information and the traffic sign boundary box, and the ground element information includes a road mask and a lane mask; the traffic sign boundary box is input into a micro perception module to obtain the component information, and the component information includes a component category and a component position; wherein the macro perception module and the micro perception module both adopt a Transformer decoder structure.

[0011] According to a traffic map generation device provided by the present invention, the decoder of the macro perception module is used to: perform feature update on the macro query through a multi-head self-attention mechanism; transform the updated macro query through a feedforward network to obtain a target macro query; and perform mask prediction based on the target macro query.

[0012] According to a traffic map generation device provided by the present invention, each query target in the decoder of the micro perception module corresponds to an anchor frame, and the anchor frame is embedded as a position feature through sinusoidal position encoding, so as to assist the decoder of the micro perception module in fusing self-attention and cross-attention.

[0013] According to a traffic map generating device provided by the present invention, the relationship attributes include at least one of the following: a complementary relationship between traffic signs, an association relationship between traffic sign components, a directional relationship between traffic sign components, a corresponding relationship between traffic signs and ground elements, and a corresponding relationship between traffic sign components and ground elements.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a traffic map generating method as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the traffic map generation method as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the traffic map generation method as described in any one of the above is implemented.

[0017] The traffic map generation method, device, equipment, storage medium and program product provided by the present invention can obtain the image features of the traffic scene image, wherein the image features are multi-scale feature maps; determine the ground element information and the traffic sign boundary box in the traffic scene image according to the image features, and determine the component information in the traffic sign boundary box; determine the relationship attributes between the ground element information and the elements in the component information; and generate a traffic map based on the ground element information, the component information and the relationship attributes. Through this scheme, the traffic scene image can be analyzed, the information of traffic signs and ground elements can be extracted, and the traffic map can be generated by combining the relationship between the elements. Since the traffic map contains the key element features in the traffic scene image, the image information can be refined, so that it can be widely used in the fields of intelligent navigation, positioning assistance and map correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a schematic diagram of the flow of the traffic map generation method provided by the present invention; Figure 2 It is a schematic diagram of the processing flow of the macro perception module provided by the present invention; Figure 3 It is a schematic diagram of the processing flow of the microscopic perception module provided by the present invention; Figure 4 It is a processing flow diagram of the relational reasoning module provided by the present invention; Figure 5 It is a structural schematic diagram of the traffic map provided by the present invention; Figure 6 It is a structural schematic diagram of the traffic map generating device provided by the present invention; Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0022] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0023] In order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and order of execution.

[0024] The embodiments of the present application describe some exemplary embodiments for the purpose of explanation. It should be understood that the present application can be implemented in other ways that are not specifically shown in the drawings.

[0025] like Figure 1 As shown, the embodiment of the present application provides a method for generating a traffic map, which can be applied to a traffic map generating device. The method for generating a traffic map may include S101-S104: S101. A traffic map generating device obtains image features of a traffic scene image.

[0026] Specifically, the traffic map generating device can first obtain a traffic scene image captured by a vehicle-mounted camera or a handheld camera, and then extract image features from the traffic scene image through a feature extraction network.

[0027] Optionally, the feature extraction network adopts a network structure combining convolutional neural networks (CNN) and transformers. The convolutional neural network adopts the ResNet-50 model, and the function of transformer is similar to the feature pyramid network, which is used to extract multi-scale feature maps. The multi-scale feature maps are composed of features of four scales, which are recorded as , corresponding to 1 / 32, 1 / 16, 1 / 8 and 1 / 4 of the traffic scene image resolution respectively.

[0028] S102. The traffic map generating device determines ground element information and a traffic sign boundary box in the traffic scene image according to the image features, and determines component information in the traffic sign boundary box.

[0029] Optionally, determining the ground element information and the traffic sign bounding box in the traffic scene image according to the image features, and determining the component information in the traffic sign bounding box, includes: inputting the multi-scale feature map into a macro perception module to obtain the ground element information and the traffic sign bounding box, the ground element information including a road mask and a lane mask; inputting the traffic sign bounding box into a micro perception module to obtain the component information, the component information including a component category and a component position; wherein the macro perception module and the micro perception module both adopt a Transformer decoder structure.

[0030] The following is a detailed description of the macro perception module: The macro perception module is used to segment ground elements (i.e., roads and lanes) and detect traffic signs. It is a module that can perform both target detection and target segmentation. The macro perception module includes a Transformer decoder, which can be used to: perform feature updates on macro queries through a multi-head self-attention mechanism; transform the updated macro queries through a feedforward network to obtain target macro queries; and perform mask prediction based on the target macro queries.

[0031] Specifically, Figure 2 As shown in Figure 2, the macro detection module is a Transformer decoder, which consists of N decoder layers with the same structure. The macro query of the macro detection module can be expressed as ,in, represents the total number of queries, and D represents the dimension. The macro detection module uses the multi-scale feature map As input, in order to utilize multi-scale features and achieve more refined segmentation results, the Transformer decoder can gradually update the features, from small-scale, semantically rich features It starts by scaling to larger and more detailed features through its decoder layers , that is, the first layer input is , the second layer input is , the third layer input is , the first The input of the layer is ,in, for The remainder when divided by 3, for example, It is 5 o'clock. is 2, and the fifth layer input is For the Transformer decoder Layer, input features Exploited to perform Multi-Head Cross Attention (MHCA) with macro queries: ; in, It is The output query of the layer decoder, Is queried by macro The predicted attention mask enables each query to extract local features within the foreground region of the predicted mask instead of focusing on the entire feature map.

[0032] Afterwards, the Transformer decoder performs feature updates on the macro query using the Multi-Head Self Attention (MHSA) mechanism: ; Updated macro query Then through the feedforward network To transform: ; Finally, the mask to be predicted is queried through the macro and input features Make predictions: ; in, , and Represents the height and width of the input image respectively, and the category is determined by the fully connected layer Determine the bounding box by the fully connected layer Sure, is the number of categories, including 8 categories in total, namely current road, left road, right road, road ahead, lane, circular traffic sign, triangular traffic sign and rectangular traffic sign, a total of 8 categories.

[0033] During training, the macro perception module is used to segment and regress all roads, lanes, and traffic signs. The output of each layer is first binary matched with the true value to calculate the classification, regression, and segmentation losses. During testing, this application only retains the road mask, lane mask, and traffic sign bounding box.

[0034] The following is a detailed description of the micro-perception module: The micro-perception module is used to locate the quadrilateral bounding box of the components (text, symbols, and arrowheads) inside the traffic sign bounding box. The micro-perception module includes a Transformer decoder, in which each query target corresponds to an anchor box, and the anchor box is embedded as a position feature through sinusoidal position encoding to assist the decoder of the micro-perception module in fusing self-attention and cross-attention.

[0035] Specifically, Figure 3 As shown in the figure, the micro-detection module is a Transformer decoder, which is also composed of N decoder layers with the same structure. The macro query of the micro-detection module can be expressed as ,in, represents the total number of queries, and D represents the dimension. The microscopic detection module uses RoIAlign technology to The internal features of the traffic sign bounding box are extracted so that each micro-query only interacts with the features within the sign area. All sign features are scaled to a fixed size and represented as All in The micro-queries in are initialized with the same learnable parameters. At the same time, in order to use explicit position priors to improve the similarity of query features, the micro-perception module sets a quadrilateral box as an anchor box for each query target. , among which, The anchor boxes can be represented as: ; These anchor boxes are then embedded as position features via sinusoidal position encoding to assist the self-attention and cross-attention fusion of micro-queries. Layer decoder, this process is expressed as: ; ; ; in, , and Respectively represent Micro-query representation at different processing stages in a layer.

[0036] Finally, all anchor boxes will be The predicted offset is updated: ; in, .

[0037] The output anchor box of the last layer of the micro-perception module decoder is the component position, and the component category comes from the classification result of a fully connected layer classifier. In this application, there are 65 component categories.

[0038] S103: The traffic map generating device determines the relationship attributes between the ground element information and the elements in the component information.

[0039] Optionally, the relationship attributes include at least one of the following: a complementary relationship between traffic signs, an association relationship between traffic sign components, a directional relationship between traffic sign components, a corresponding relationship between traffic signs and ground elements, and a corresponding relationship between traffic sign components and ground elements.

[0040] Specifically, the complementary relationship between traffic signs can be expressed as SS, which means that the two traffic signs complement each other; the association relationship between traffic sign components and the directional relationship between traffic sign components can be expressed as CC, where the association relationship indicates that the two components refer to the same place, and the directional relationship indicates that the place referred to by the components is located in the direction indicated by an arrowhead; the correspondence between traffic signs and ground elements can be expressed as ST, and the correspondence between traffic sign components and ground elements can be expressed as CT.

[0041] Optionally, the traffic map generation device can predict the relationship between multiple elements and text attributes through a relationship reasoning module. Text attributes are text content labels that support the generation of visual traffic maps, including five categories: road code, road name, place name, direction and description.

[0042] Specifically, Figure 4 As shown in the figure, in the input graph structure, macro queries, micro queries, and edge queries are represented by squares, triangles, and circles, respectively. Dashed arrows represent SC links. Black-filled bold arrows represent dense ST links, while white-filled bold arrows represent sparse CT links. In order to achieve comprehensive context awareness and fusion, all elements and relations are modeled into an input graph and updated through the Transformer decoder. In the input graph, all elements are regarded as nodes and the relations between elements are regarded as edges. Their features are represented as node queries, and edge query Thanks to the query prediction mechanism, the relational reasoning module can directly use the updated macro query and micro query as the initial node query. , because they already contain the category and position information of the corresponding element. Therefore, the number of node queries is the sum of effective macro queries and micro queries. and The links between them, the edge query is their sum vector: .

[0043] For complex traffic scenes with a large number of elements, building a fully connected graph will result in a large amount of computation, and some links have no practical significance or necessity. To solve this problem, this module adopts a hierarchical graph structure. Figure 4As shown, it includes three subgraphs: sign subgraph (S-subgraph), component subgraph (C-subgraph) and traffic subgraph (T-subgraph). In addition, three cross-subgraph links (SC, ST, CT) are created to facilitate communication between subgraphs. The relationships that need to be predicted include the sign-to-sign (SS) relationship within the S-subgraph, the component-to-component (CC) relationship within the C-subgraph, the traffic sign-to-ground element (ST) relationship represented by the ST link, and the component-to-ground element (CT) relationship represented by the CT link.

[0044] SS links in the S-subgraph are determined by a heuristic method based on the relative distance and alignment between traffic signs. For example, there are no links between signs that are far away or spatially misaligned. For the C-subgraph, all components of a single sign or all components of two signs with SS links are fully connected to infer CC relations (relationships between components) within a sign or between different signs. Therefore, the S-subgraph and C-subgraph are constructed as sparse graphs. Since SS and CC relations are undirected, they are represented by two opposite edges.

[0045] The T-subgraph contains only ground elements, i.e., roads and lanes. This subgraph predicts the affiliation and relative position relations between ground elements with dense connections to assist in reasoning about ST and CT relations.

[0046] SC links flow from traffic signs to their components, indicated by dashed lines in Figure 4 A high-level information is provided in Figure 1. ST links, represented by bold arrows filled with black, are dense connections from each traffic sign to each ground element to determine the ST relationship. CT links, represented by bold arrows filled with white, are sparse connections that exist only between arrow elements, i.e., arrow signs and arrowheads, and ground elements to determine the CT relationship.

[0047] In the relational reasoning module, this module designs a message passing mechanism for the three subgraphs and their cross-graph links to improve the reasoning efficiency. There are multiple network layers with the same structure in the relational reasoning module. layer( ), the input node query is expressed as , the edge query is expressed as ,in, and is the initial feature. ; The relational reasoning module first applies the self-attention fusion mechanism to the S, C, and T-subgraphs. Since edge information is crucial for the self-attention fusion between nodes, this module modifies the self-attention mechanism to fuse edge features. It can be obtained by the following equation: ⊙ ; ; ; in, and is a fully connected layer, is through Edge features converted from edge queries: ; Node queries are obtained through attention matrix fusion and feed-forward network: ; ; in, .

[0048] The edge query is obtained as follows: ( ⊙ ) ; The information is then passed from the S-subgraph to the C-subgraph via SC links. and the target node , the information transmission process can be expressed as follows: ; in, .

[0049] Finally, by concatenating and mapping ST and CT links, the matching relationship features can be obtained: ; in, ; Represents a tensor concatenation operation.

[0050] The relational reasoning module passes through the fully connected layer Get the relationship represented by the updated edge query, is the number of relation categories. All node queries representing text are fed into another fully connected layer to get the text attributes.

[0051] S104. A traffic map generating device generates a traffic map based on the ground element information, the component information and the relationship attributes.

[0052] Specifically, Figure 5 As shown in the figure, the traffic map consists of roads as root nodes ( ), with other roads, lanes and multiple attributes as leaf nodes. For each road, the traffic map generation device can extract information such as the road name, driving direction, destination, lane and other descriptions, and based on the output results of the above multiple modules, with the road as the root node, organize all the extracted information into a traffic map according to the corresponding relationship.

[0053] In an embodiment of the present application, traffic scene images can be analyzed to extract information about traffic signs and ground elements, and a traffic map can be generated by combining the relationships between the elements. Since the traffic map contains key element features in the traffic scene image, the image information can be refined, thereby being widely used in the fields of intelligent navigation, positioning assistance, and map correction.

[0054] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed herein, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0055] The traffic map generation method provided in the embodiment of the present application can be executed by a traffic map generation device or a control module for traffic map generation in the traffic map generation device. In the embodiment of the present application, the traffic map generation device provided in the embodiment of the present application is described by taking the traffic map generation method executed by the traffic map generation device as an example.

[0056] It should be noted that the embodiment of the present application can divide the functional modules of the traffic map generation device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. Optionally, the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0057] like Figure 6As shown, an embodiment of the present application provides a traffic map generating device 600. The traffic map generating device 600 includes: an acquisition module 601 and a processing module 602. The acquisition module 601 can be used to acquire image features of a traffic scene image, wherein the image features are multi-scale feature maps; the processing module 602 can be used to determine ground element information and a traffic sign boundary box in the traffic scene image according to the image features, and determine component information in the traffic sign boundary box; determine the ground element information and the relationship attributes between the elements in the component information; and generate a traffic map based on the ground element information, the component information and the relationship attributes.

[0058] Optionally, the processing module 602 is used to input the multi-scale feature map into a macro perception module to obtain the ground element information and the traffic sign bounding box, wherein the ground element information includes a road mask and a lane mask; input the traffic sign bounding box into a micro perception module to obtain the component information, wherein the component information includes a component category and a component position; wherein both the macro perception module and the micro perception module adopt a Transformer decoder structure.

[0059] Optionally, the decoder of the macro perception module is used to: perform feature update on the macro query through a multi-head self-attention mechanism; transform the updated macro query through a feedforward network to obtain a target macro query; and perform mask prediction based on the target macro query.

[0060] Optionally, each query target in the decoder of the micro perception module corresponds to an anchor frame, and the anchor frame is embedded as a position feature through sinusoidal position encoding to assist the decoder of the micro perception module in fusing self-attention and cross-attention.

[0061] Optionally, the relationship attributes include at least one of the following: a complementary relationship between traffic signs, an association relationship between traffic sign components, a directional relationship between traffic sign components, a corresponding relationship between traffic signs and ground elements, and a corresponding relationship between traffic sign components and ground elements.

[0062] In an embodiment of the present application, traffic scene images can be analyzed to extract information about traffic signs and ground elements, and a traffic map can be generated by combining the relationships between the elements. Since the traffic map contains key element features in the traffic scene image, the image information can be refined, thereby being widely used in the fields of intelligent navigation, positioning assistance, and map correction.

[0063] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the traffic map generation method, which includes: obtaining image features of a traffic scene image, wherein the image features are multi-scale feature maps; determining ground element information and a traffic sign boundary box in the traffic scene image according to the image features, and determining component information in the traffic sign boundary box; determining the relationship attributes between the ground element information and each element in the component information; and generating a traffic map based on the ground element information, the component information and the relationship attributes.

[0064] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0065] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the traffic map generation method provided by the above-mentioned methods, which includes: obtaining image features of a traffic scene image, wherein the image features are multi-scale feature maps; determining ground element information and a traffic sign bounding box in the traffic scene image based on the image features, and determining component information in the traffic sign bounding box; determining the relationship attributes between the ground element information and the elements in the component information; and generating a traffic map based on the ground element information, the component information and the relationship attributes.

[0066] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the traffic map generation method provided by the above-mentioned methods, the method comprising: obtaining image features of a traffic scene image, wherein the image features are multi-scale feature maps; determining ground element information and a traffic sign bounding box in the traffic scene image according to the image features, and determining component information in the traffic sign bounding box; determining the relationship attributes between the ground element information and the elements in the component information; and generating a traffic map based on the ground element information, the component information and the relationship attributes.

[0067] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0068] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a traffic map, characterized in that: include: Acquire image features of a traffic scene image, wherein the image features are multi-scale feature maps; Determining ground element information and a traffic sign boundary frame in the traffic scene image according to the image features, and determining component information in the traffic sign boundary frame; Determine the relationship attributes between the ground element information and the elements in the component information; A traffic map is generated based on the ground element information, the component information and the relationship attributes.

2. The method for generating a traffic map according to claim 1, characterized in that: The determining, according to the image features, ground element information and a traffic sign boundary frame in the traffic scene image, and determining component information in the traffic sign boundary frame, comprises: Inputting the multi-scale feature map into a macro perception module to obtain the ground element information and the traffic sign bounding box, wherein the ground element information includes a road mask and a lane mask; Inputting the traffic sign boundary box into a microscopic perception module to obtain the component information, wherein the component information includes a component category and a component position; Among them, the macro perception module and the micro perception module both adopt the Transformer decoder structure.

3. The traffic map generation method according to claim 2, characterized in that: The decoder of the macro perception module is used to: Perform feature updates on macro queries via a multi-head self-attention mechanism; Transform the updated macro query through a feedforward network to obtain a target macro query; Mask prediction is performed based on the target macro query.

4. The method for generating a traffic map according to claim 2, characterized in that: Each query target in the decoder of the micro perception module corresponds to an anchor frame, and the anchor frame is embedded as a position feature through sinusoidal position encoding to assist the decoder of the micro perception module in fusing self-attention and cross-attention.

5. The method for generating a traffic map according to claim 1, characterized in that: The relationship attributes include at least one of the following: a complementary relationship between traffic signs, an association relationship between traffic sign components, a directional relationship between traffic sign components, a corresponding relationship between traffic signs and ground elements, and a corresponding relationship between traffic sign components and ground elements.

6. A traffic map generating device, characterized in that: include: Acquisition module and processing module; The acquisition module is used to acquire image features of the traffic scene image, wherein the image features are multi-scale feature maps; The processing module is used to determine ground element information and a traffic sign boundary frame in the traffic scene image according to the image features, and determine component information in the traffic sign boundary frame; Determine the relationship attributes between the ground element information and the elements in the component information; A traffic map is generated based on the ground element information, the component information and the relationship attributes.

7. The traffic map generating device according to claim 6, characterized in that: The processing module is used to input the multi-scale feature map into a macro perception module to obtain the ground element information and the traffic sign boundary box, wherein the ground element information includes a road mask and a lane mask; input the traffic sign boundary box into a micro perception module to obtain the component information, wherein the component information includes a component category and a component position; wherein both the macro perception module and the micro perception module adopt a Transformer decoder structure.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the traffic map generation method as described in any one of claims 1 to 5 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the traffic map generation method as described in any one of claims 1 to 5 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the traffic map generation method as described in any one of claims 1 to 5 is implemented.

Images