Map construction method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a map construction method, electronic equipment and a computer readable storage medium. The map construction method comprises the steps of obtaining image features of a target aerial view image; based on a closed graph detection model, detecting and identifying a closed graph in the image features of the target aerial view image to obtain closed graph parameters; on the basis of a non-closed graph detection model, non-closed graphs in the image features of the target aerial view image are detected and recognized, and non-closed graph parameters are obtained; based on the closed graph parameters, the non-closed graph parameters and the image features of the target aerial view image, constructing a map for identifying the closed graph and the non-closed graph; the parameter category of the closed graph parameter is different from the parameter category of the non-closed graph parameter. Based on the above mode, the accuracy of map identification can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a map construction method, an electronic device, and a computer-readable storage medium. Background Art

[0002] In the prior art, it is usually necessary to detect and identify the categories of various graphics in the obtained bird's-eye view image, mark them in the bird's-eye view image, and thus obtain a map. Only then can navigation, search, and other operations be performed based on the obtained map.

[0003] The defect of the prior art is that the existing detection and recognition methods for bird's-eye view images usually result in a poor identification effect for one of the closed figures and the non-closed figures relative to the other. For example, using points to identify closed figures can easily lead to difficulties in identification convergence, and using segmentation to identify non-closed figures can easily lead to poor identification, thereby making the accuracy of existing map identification poor. Summary of the invention

[0004] The main technical problem solved by this application is how to improve the accuracy of map marking.

[0005] In order to solve the above technical problems, the first technical solution adopted in the present application is: a map construction method, including: obtaining image features of a target bird's-eye view image; based on a closed graphic detection model, detecting and identifying closed graphics in the image features of the target bird's-eye view image to obtain closed graphic parameters; based on a non-closed graphic detection model, detecting and identifying non-closed graphics in the image features of the target bird's-eye view image to obtain non-closed graphic parameters; based on closed graphic parameters, non-closed graphic parameters and image features of the target bird's-eye view image, constructing a map for identifying closed graphics and non-closed graphics; the parameter category of the closed graphic parameters is different from the parameter category of the non-closed graphic parameters.

[0006] Among them, the closed figure parameters include at least one of the size information of the closed figure, the position information of the closed figure, the angle information of the closed figure and the category information of the closed figure; the non-closed figure parameters include at least one of the category information of the non-closed figure and the point information of the non-closed figure.

[0007] Among them, the closed figure detection model includes at least one closed detection layer group and at least one convolutional neural network layer connected in sequence, the input end of the first closed detection layer group is the input end of the closed figure detection model, the output end of the last closed detection layer group is connected to the input end of the convolutional neural network layer, and the output end of the convolutional neural network is the output end of the closed figure detection model; the closed detection layer group includes a self-attention layer, a cross-attention layer and a feedforward neural network layer connected in sequence, the self-attention layer is the input end of the closed detection layer group, and the feedforward neural network layer is the output end of the closed detection layer group; the input end of the first closed detection layer group is used to receive image features of the target bird's-eye view image, and the output end of the last closed detection layer group is used to output closed figure parameters.

[0008] Among them, the non-closed graphic detection model includes at least one non-closed detection layer group connected in sequence, the input end of the first non-closed detection layer group is the input end of the non-closed graphic detection model, and the output end of the last non-closed detection layer group is the output end of the non-closed graphic detection model; the non-closed detection layer group includes a multi-head self-attention layer, a first normalization layer, a deformable cross attention layer, a second normalization layer, a non-closed graphic parameter self-attention layer and a third normalization layer connected in sequence, the input end of the multi-head self-attention layer is the input end of the non-closed detection layer group, and the output end of the third normalization layer is the output end of the non-closed detection layer group end; the multi-head self-attention layer in the first non-closed detection layer group is used to receive the image features of the target bird's-eye view image, the multi-head self-attention layers in other non-closed detection layer groups except the first non-closed detection layer group are used to receive the results output by the previous multi-head self-attention layer, the deformable cross attention layer is also used to receive the image features of the target bird's-eye view image, the second normalization layer is also used to receive the image features of the target bird's-eye view image and the results output by the first normalization layer, the third normalization layer is also used to receive the results output by the second normalization layer, and the third normalization layer in the last non-closed detection layer group is used to output non-closed graphic parameters.

[0009] Among them, the non-closed graphic parameters include category information of the non-closed graphic and point information of the non-closed graphic; the non-closed graphic parameter self-attention layer includes a category self-attention layer and a point self-attention layer, the third normalization layer includes a first sub-normalization layer and a second sub-normalization layer, the category self-attention layer, the first sub-normalization layer, the point self-attention layer and the second sub-normalization layer are connected in sequence, the first sub-normalization layer and the second sub-normalization layer are also used to receive the results output by the second normalization layer respectively, the input end of the category self-attention layer is the input end of the non-closed graphic parameter self-attention layer, and the output end of the second sub-normalization layer is the output end of the third normalization layer; the output end of the second sub-normalization layer in the last non-closed detection layer group is used to output the category information of the non-closed graphic and the point information of the non-closed graphic.

[0010] Among them, obtaining the image features of the target bird's-eye view image includes: obtaining multi-view images, and obtaining image bird's-eye view features based on the multi-view images; obtaining point cloud data, and obtaining radar bird's-eye view features based on the point cloud data; performing feature fusion based on the image bird's-eye view features and the radar bird's-eye view features to obtain the image features of the target bird's-eye view image.

[0011] Among them, based on the multi-view image, the image bird's-eye view features are obtained, including: inputting the multi-view image into the image backbone network model to obtain the image features of the multi-view image; inputting the image features of the multi-view image into the perspective converter to obtain the image bird's-eye view features; and / or, based on the point cloud data, the radar bird's-eye view features are obtained, including: inputting the point cloud data into the radar backbone network model to obtain the radar bird's-eye view features.

[0012] Wherein, before detecting and identifying closed graphics in the image features of the target bird's-eye view image based on the closed graphics detection model and obtaining closed graphics parameters, the map construction method also includes: obtaining multiple closed training samples; the closed training samples include target bird's-eye view image samples or image feature samples of the target bird's-eye view image, and closed graphics parameter samples; based on the closed training samples, the closed graphics detection model is trained; and / or, before detecting and identifying non-closed graphics in the image features of the target bird's-eye view image based on the non-closed graphics detection model and obtaining non-closed graphics parameters, the map construction method also includes: obtaining multiple non-closed training samples; the non-closed training samples include target bird's-eye view image samples or image feature samples of the target bird's-eye view image, and non-closed graphics parameter samples; based on the non-closed training samples, the non-closed graphics detection model is trained.

[0013] In order to solve the above technical problems, the second technical solution adopted in the present application is: an electronic device, comprising: a memory and a processor; the memory is used to store program instructions, and the processor is used to execute the program instructions to implement the above method.

[0014] In order to solve the above technical problems, the third technical solution adopted in the present application is: a computer-readable storage medium, which stores program instructions, and the above method is implemented when the program instructions are executed by a processor.

[0015] The beneficial effect of the present application lies in that, different from the prior art, in the technical solution of the present application, image features of a target bird's-eye view image are acquired, closed graphics in the image features of the target bird's-eye view image are detected and identified based on a closed graphics detection model to obtain closed graphics parameters, non-closed graphics in the image features of the target bird's-eye view image are detected and identified based on a non-closed graphics detection model to obtain non-closed graphics parameters, and a map for identifying the closed graphics and the non-closed graphics is constructed based on the closed graphics parameters, the non-closed graphics parameters and the image features of the target bird's-eye view image, wherein the parameter category of the closed graphics parameters is different from the parameter category of the non-closed graphics parameters. Based on the above method, different models can be used to detect and identify closed figures and non-closed figures respectively, that is, the closed figure detection model is used to detect and identify the closed figure and output the corresponding closed figure parameters suitable for identifying the closed figure, and the non-closed figure detection model is used to detect and identify the non-closed figure and output the corresponding non-closed figure parameters suitable for identifying the non-closed figure. The closed figure parameters are different from the non-closed figure parameters, so that in the subsequent steps, the closed figure can be identified based on the closed figure parameters adapted to the closed figure, and the non-closed figure can be identified based on the non-closed figure parameters adapted to the non-closed figure, so as to construct a map in which both closed and non-closed figures are reasonably identified, thereby improving the accuracy of map identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 It is a flowchart of an embodiment of the map construction method of the present application;

[0018] Figure 2 is a flowchart of another embodiment of the map construction method of the present application;

[0019] Figure 3 It is a structural schematic diagram of an embodiment of the closed figure detection model of the present application;

[0020] Figure 4 It is one of the structural schematic diagrams of an embodiment of the non-closed graphic detection model of the present application;

[0021] Figure 5 This is the second structural schematic diagram of an embodiment of the non-closed graphic detection model of the present application;

[0022] Figure 6It is a schematic diagram of the structure of an embodiment of the electronic device of the present application;

[0023] Figure 7 It is a structural schematic diagram of an embodiment of the computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0024] The present application is further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present application, but are not intended to limit the scope of the present application. Similarly, the following examples are only some embodiments of the present application rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of the present application.

[0025] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0026] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "install", "set", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or connected through an intermediate medium. For ordinary technicians in this field, the above-mentioned specific meanings belonging to this application can be connected according to specific circumstances.

[0027] This application first proposes a map construction method, see Figure 1 , Figure 1 is a flowchart of an embodiment of the map construction method of the present application. Figure 1 As shown, the map construction method includes:

[0028] Step S11: Acquire image features of the target bird's-eye view image.

[0029] Among them, the target bird's-eye view image can specifically be a top-view image of the area where a map needs to be constructed. For example, it can be a multi-perspective image first acquired by an image acquisition module (such as a camera), and / or point cloud data acquired by a radar module, and then corresponding processing is performed based on the multi-perspective image and / or the point cloud data to obtain corresponding bird's-eye view features, and corresponding feature fusion is performed to obtain the image features of the target bird's-eye view image.

[0030] Step S12: Based on the closed figure detection model, the closed figures in the image features of the target bird's-eye view image are detected and identified to obtain closed figure parameters.

[0031] Step S13: Based on the non-closed figure detection model, the non-closed figures in the image features of the target bird's-eye view image are detected and identified to obtain non-closed figure parameters.

[0032] It should be noted that step S12 and step S13 can be parallel steps, or steps executed in any order, or two steps of other types executed after step S11. The specific steps can be determined according to needs and are not limited here.

[0033] In one example, the closed figure parameters may include at least one of the size information of the closed figure, the position information of the closed figure, the angle information of the closed figure, and the category information of the closed figure. The non-closed figure parameters may include at least one of the category information of the non-closed figure and the point information of the non-closed figure.

[0034] The size information may be the area size, the location information may be the coordinates, the angle information may refer to the angle between the extension direction of a closed figure such as a road and a fixed direction (such as due north or other directions), the point information may be the related information of the point set and point coordinates used to represent the figure, and the category information may be the category of the object corresponding to the figure.

[0035] In step S12, a pre-trained closed figure detection model can be used to perform a first type of detection and identification on closed figures in the image features of the target bird's-eye view image to identify closed figure parameters suitable for characterizing the closed figures. For example, the closed figure parameters can specifically be at least one of the size information, position information, angle information and category information of the corresponding closed figure. Compared with using point information to characterize the closed image in a point set manner, using at least one of the above-mentioned size information, position information, angle information and category information can more accurately and conveniently identify the closed figure.

[0036] In step S13, a second type of detection and identification can be performed on the non-closed graphics in the image features of the target bird's-eye view image through a pre-trained non-closed graphics detection model to identify non-closed graphics parameters suitable for characterizing the non-closed graphics. For example, the non-closed graphics parameters can specifically be at least one of the point information and category information of the corresponding non-closed graphics. Compared with using at least one of size information, position information, angle information and category information to characterize the non-closed image, using at least one of the above-mentioned point information and category information can more accurately and conveniently realize the identification of the non-closed graphics.

[0037] Step S14: constructing a map for identifying closed graphics and non-closed graphics based on closed graphics parameters, non-closed graphics parameters and image features of the target bird's-eye view image.

[0038] Among them, the parameter category of the closed graphic parameters is different from the parameter category of the non-closed graphic parameters.

[0039] By adopting parameters of parameter categories that are respectively adapted to closed figures and non-closed figures, closed figures and non-closed figures on the map are adaptively marked, which can solve the problem that closed figures in the map are difficult to detect and mark. While achieving normal detection and marking of non-closed figures, it can also improve the reliability of detection and marking of closed figures, so as to construct a map in which both closed figures and non-closed figures are reasonably marked, thereby improving the accuracy of map marking.

[0040] Different from the prior art, in the technical solution of the present application, image features of a target bird's-eye view image are acquired, closed graphics in the image features of the target bird's-eye view image are detected and identified based on a closed graphics detection model to obtain closed graphics parameters, non-closed graphics in the image features of the target bird's-eye view image are detected and identified based on a non-closed graphics detection model to obtain non-closed graphics parameters, and a map for identifying the closed graphics and the non-closed graphics is constructed based on the closed graphics parameters, the non-closed graphics parameters and the image features of the target bird's-eye view image, wherein a parameter category of the closed graphics parameters is different from a parameter category of the non-closed graphics parameters. Based on the above method, different models can be used to detect and identify closed figures and non-closed figures respectively, that is, the closed figure detection model is used to detect and identify the closed figure and output the corresponding closed figure parameters suitable for identifying the closed figure, and the non-closed figure detection model is used to detect and identify the non-closed figure and output the corresponding non-closed figure parameters suitable for identifying the non-closed figure. The closed figure parameters are different from the non-closed figure parameters, so that in the subsequent steps, the closed figure can be identified based on the closed figure parameters adapted to the closed figure, and the non-closed figure can be identified based on the non-closed figure parameters adapted to the non-closed figure, so as to construct a map in which both closed and non-closed figures are reasonably identified, thereby improving the accuracy of map identification.

[0041] In one embodiment, step S11 may specifically include:

[0042] A multi-view image is acquired, and based on the multi-view image, a bird's-eye view feature of the image is obtained.

[0043] The point cloud data is acquired, and based on the point cloud data, the radar bird's-eye view features are obtained.

[0044] Based on the image bird's-eye view features and the radar bird's-eye view features, feature fusion is performed to obtain the image features of the target bird's-eye view image.

[0045] Specifically, based on the multi-view images, the image bird's-eye view features are obtained, which may specifically include:

[0046] The multi-view images are input into the image backbone network model to obtain the image features of the multi-view images.

[0047] The image features of the multi-view images are input into the view converter to obtain the image bird's-eye view features.

[0048] And / or, based on the point cloud data, obtaining radar bird's-eye view features, which may specifically include:

[0049] The point cloud data is input into the radar backbone network model to obtain the radar bird's-eye view features.

[0050] For an example, see Figure 2 , Figure 2 is a flowchart of another embodiment of the map construction method of the present application. Figure 2 As shown, first, multiple image acquisition modules (such as cameras) can be used to capture images of the area where a map needs to be constructed from multiple perspectives to obtain multi-perspective images, and the multi-perspective images can be input into the image backbone network model to be downsampled by a preset multiple to obtain image features of the multi-perspective images, and the image features of the multi-perspective images can be input into the perspective converter, and the image features of the multi-perspective images can be converted into image bird's-eye view features using the internal and external parameters of the image acquisition module. Among them, the image backbone network model can be a model with a ResNet50[] structure.

[0051] Second, the radar module can be used to obtain point cloud data of the area where the map needs to be constructed, and the point cloud data can be input into the radar backbone network model. The radar backbone network model may include a voxelization layer, an intermediate feature extraction layer, a backbone layer, and a neck network layer. The voxelization layer is used to rasterize the point cloud data to obtain regular voxel features. The intermediate feature extraction layer is used to extract features by continuous three-dimensional sparse convolution. The backbone layer adopts a SECOND structure to extract feature maps of two different scales through two channels. The neck network layer is used to use two-dimensional deconvolution to upsample the features obtained by the above method to perform feature fusion, realize the fusion of deep features and shallow features, and obtain radar bird's-eye view features. Among them, deep features usually have a smaller feature map scale and a wider overall convolution field of view, while shallow features usually have a larger scale and can better reflect local features. Through fusion, the radar bird's-eye view features can have the characteristics of wide field of view and detailed local features at the same time.

[0052] Third, the obtained image bird's-eye view features and radar bird's-eye view features can be input into the fusion network model for feature fusion to obtain the image features of the target bird's-eye view image that have the characteristics of both the image bird's-eye view features and the radar bird's-eye view features. In addition, the image features of the target bird's-eye view image can be subsequently input into the closed figure detection model and the non-closed figure detection model respectively to execute the above steps S11-S14. By adopting parameters of parameter categories that are respectively adapted to closed figures and non-closed figures, the closed figures and non-closed figures on the map are adaptively marked, which can solve the problem that closed figures in the map are difficult to detect and mark. While realizing the normal detection and marking of non-closed figures, the reliability of the detection and marking of closed figures can also be improved, so as to construct a map in which both closed figures and non-closed figures are reasonably marked, thereby improving the accuracy of map marking.

[0053] It should be noted that the closed figure detection model can specifically be a model built based on the DETR (detection transformer) detection framework, and the closed figures can specifically be figures corresponding to crosswalks, stop lines and ground signs. For example, the closed figures can be identified by size information, position information, angle information and category information. The size information can be the area size, the position information can be the coordinates, the angle information can refer to the angle between the extension direction of the road-type closed figure and a fixed direction (such as due north or other directions), and the category information can be the category of the thing corresponding to the figure.

[0054] The non-closed figure detection model may be a model built based on a Transformer structure. The non-closed figure may specifically be a curb or a lane line. For example, the non-closed figure may be identified by a number of points.

[0055] In one embodiment, the closed figure detection model includes at least one closed detection layer group and at least one convolutional neural network layer connected in sequence, the input end of the first closed detection layer group is the input end of the closed figure detection model, the output end of the last closed detection layer group is connected to the input end of the convolutional neural network layer, and the output end of the convolutional neural network is the output end of the closed figure detection model.

[0056] The closed detection layer group includes a self-attention layer, a cross-attention layer and a feedforward neural network layer connected in sequence, the self-attention layer is the input end of the closed detection layer group, and the feedforward neural network layer is the output end of the closed detection layer group.

[0057] The input end of the first closed detection layer group is used to receive the image features of the target bird's-eye view image, and the output end of the last closed detection layer group is used to output closed graphic parameters.

[0058] Specifically, see Figure 3 , Figure 3is a structural diagram of an embodiment of the closed figure detection model of the present application, such as Figure 3 As shown, the image features of the target bird's-eye view image can be input into the first closed detection layer group and processed sequentially by the self-attention layer, the cross-attention layer and the feedforward neural network layer, and then the results of the sequential processing output are input into the next closed detection layer group and processed sequentially by the self-attention layer, the cross-attention layer and the feedforward neural network layer, and so on, until the output result of the above-mentioned sequential processing corresponding to the last closed detection layer group, and the output result of the last closed detection layer group is respectively input into 4 convolutional neural network layers, and each convolutional neural network layer corresponds to outputting the size information of the closed figure, the position information of the closed figure, the angle information of the closed figure and the category information of the closed figure, respectively. The 4 convolutional neural network layers can also be increased or decreased to output more or less information, which is not limited here.

[0059] The number of at least one closed detection layer group connected in sequence may be 1, 6, 9 or other numbers, which is not limited here.

[0060] Based on the above method, the information of the input features themselves can be fused through the self-attention layer to ensure that each detection and recognition represents a different object. The image features of the target bird's-eye view image are introduced into the detection and recognition through the cross-attention layer. The corresponding class information is output respectively through different convolutional neural network layers. This information can be set as information suitable for identifying closed figures. Finally, a closed figure detection model suitable for detection and recognition of closed figures is constructed, which can also be called a closed figure detection head, thereby improving the accuracy of map identification.

[0061] In one embodiment, see Figure 4 , Figure 4 is one of the structural schematic diagrams of an embodiment of the non-closed graphic detection model of the present application, such as Figure 4 As shown, the non-closed graphic detection model includes at least one non-closed detection layer group connected in sequence, the input end of the first non-closed detection layer group is the input end of the non-closed graphic detection model, and the output end of the last non-closed detection layer group is the output end of the non-closed graphic detection model.

[0062] The non-closed detection layer group includes a multi-head self-attention layer, a first normalization layer, a deformable cross attention layer, a second normalization layer, a non-closed graphic parameter self-attention layer and a third normalization layer connected in sequence. The input end of the multi-head self-attention layer is the input end of the non-closed detection layer group, and the output end of the third normalization layer is the output end of the non-closed detection layer group.

[0063] Among them, the multi-head self-attention layer in the first non-closed detection layer group is used to receive non-closed graphic features in the image features of the target bird's-eye view image, the multi-head self-attention layers in other non-closed detection layer groups except the first non-closed detection layer group are used to receive the results output by the previous multi-head self-attention layer, the deformable cross attention layer is also used to receive the image features of the target bird's-eye view image, the second normalization layer is also used to receive the image features of the target bird's-eye view image and the results output by the first normalization layer, the third normalization layer is also used to receive the results output by the second normalization layer, and the third normalization layer in the last non-closed detection layer group is used to output non-closed graphic parameters.

[0064] Specifically, the number of at least one non-enclosed detection layer group connected in sequence may be 1, 6, 9 or other numbers, which is not limited here.

[0065] The non-closed graphic feature in the image feature of the target bird's-eye view image may specifically be an initial value of the non-closed graphic feature obtained by preprocessing the image feature of the target bird's-eye view image, that is, an initial value of the non-closed graphic parameter.

[0066] See also Figure 5 , Figure 5 This is a second structural diagram of an embodiment of the non-closed graphic detection model of the present application. Figure 5 As shown, the non-closed figure parameters include the category information of the non-closed figure and the point information of the non-closed figure.

[0067] The non-closed graphic parameter self-attention layer includes a category self-attention layer and a point self-attention layer, the third normalization layer includes a first sub-normalization layer and a second sub-normalization layer, the category self-attention layer, the first sub-normalization layer, the point self-attention layer and the second sub-normalization layer are connected in sequence, the first sub-normalization layer and the second sub-normalization layer are also used to receive the results output by the second normalization layer respectively, the input end of the category self-attention layer is the input end of the non-closed graphic parameter self-attention layer, and the output end of the second sub-normalization layer is the output end of the third normalization layer.

[0068] The output end of the second sub-normalization layer in the last non-closed detection layer group is used to output the category information of the non-closed graphics and the point information of the non-closed graphics.

[0069] It should be noted that the normalization layer can be used to add and normalize the received data, and then output it to the corresponding layer. For example, after the non-closed graphic features in the image features of the target bird's-eye view image are input into the multi-head self-attention layer, the multi-head self-attention layer can output the results of non-closed graphic detection and recognition in multiple target bird's-eye view images respectively, which can be added and normalized through the first normalization layer to improve the stability, accuracy and reliability of the output results.

[0070] In addition, each deformable cross attention layer is used to receive the image features of the target bird's-eye view image, so as to incorporate the original image features into the associated considerations of detection and recognition, so that the deformable cross attention layer can associate the non-closed graphics with the image features of the target bird's-eye view image, thereby better realizing the detection and recognition of non-closed graphics.

[0071] The category information of the non-closed figure can be used to represent the category of the object corresponding to the non-closed figure, and the point information of the non-closed figure can be used to represent the points and the positions of the points in the non-closed figure, that is, the point set information.

[0072] Based on the above method, the information of the input features themselves can be fused through the multi-head self-attention layer to ensure that each detection and recognition represents a different object. The image features of the target bird's-eye view image are introduced into the detection and recognition for association through the deformable cross-attention layer. Finally, a non-closed graphic detection model suitable for detection and recognition of non-closed graphics is constructed, which can also be called a non-closed graphic detection head, thereby improving the accuracy of map marking.

[0073] In one embodiment, before step S12, the map construction method further includes:

[0074] A plurality of closed training samples are obtained, wherein the closed training samples include target bird's-eye view image samples or image feature samples of the target bird's-eye view image, and closed graphic parameter samples.

[0075] Based on closed training samples, the closed figure detection model is trained.

[0076] And / or, before step S13, the map construction method further includes:

[0077] A plurality of non-closed training samples are obtained, wherein the non-closed training samples include target bird's-eye view image samples or image feature samples of the target bird's-eye view image, and non-closed graphic parameter samples.

[0078] Based on the non-closed training samples, the non-closed figure detection model is trained.

[0079] Specifically, the target bird's-eye view image samples representing closed graphics and / or non-closed graphics may be combined with the training samples constructed by the corresponding closed graphics parameter samples and / or non-closed graphics parameter samples to obtain the following: Figure 2 The total model consisting of all models of the volume in the process shown is trained.

[0080] The closed figure detection model and the non-closed figure detection model can also be trained based on the image feature samples of the target bird's-eye view image representing closed figures and / or non-closed figures, combined with the training samples constructed by the corresponding closed figure parameter samples and / or non-closed figure parameter samples.

[0081] Based on the above method, the closed figure detection model and the non-closed figure detection model can be trained based on the training samples generated by manual annotation or other annotation methods, and then steps S11 to S14 are executed based on the trained closed figure detection model and the non-closed figure detection model to improve the accuracy of map marking.

[0082] This application also proposes an electronic device, see Figure 6 , Figure 6 is a schematic diagram of the structure of an embodiment of the electronic device of the present application, such as Figure 6 As shown, the electronic device 20 includes a processor 21 , a memory 22 and a bus 23 .

[0083] The processor 21 and the memory 22 are connected to the bus 23 respectively. The memory 22 stores program instructions. The processor 21 is used to execute the program instructions to implement the map construction method in the above embodiment.

[0084] In this embodiment, the processor 21 may also be referred to as a CPU (Central Processing Unit). The processor 21 may be an integrated circuit chip having signal processing capabilities. The processor 21 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor 21 may also be any conventional processor, etc.

[0085] Different from the prior art, in the technical solution of the present application, image features of a target bird's-eye view image are acquired, closed graphics in the image features of the target bird's-eye view image are detected and identified based on a closed graphics detection model to obtain closed graphics parameters, non-closed graphics in the image features of the target bird's-eye view image are detected and identified based on a non-closed graphics detection model to obtain non-closed graphics parameters, and a map for identifying the closed graphics and the non-closed graphics is constructed based on the closed graphics parameters, the non-closed graphics parameters and the image features of the target bird's-eye view image, wherein a parameter category of the closed graphics parameters is different from a parameter category of the non-closed graphics parameters. Based on the above method, different models can be used to detect and identify closed figures and non-closed figures respectively, that is, the closed figure detection model is used to detect and identify the closed figure and output the corresponding closed figure parameters suitable for identifying the closed figure, and the non-closed figure detection model is used to detect and identify the non-closed figure and output the corresponding non-closed figure parameters suitable for identifying the non-closed figure. The closed figure parameters are different from the non-closed figure parameters, so that in the subsequent steps, the closed figure can be identified based on the closed figure parameters adapted to the closed figure, and the non-closed figure can be identified based on the non-closed figure parameters adapted to the non-closed figure, so as to construct a map in which both closed and non-closed figures are reasonably identified, thereby improving the accuracy of map identification.

[0086] This application also proposes a computer-readable storage medium, see Figure 7 , Figure 7 is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of the present application. Figure 7 As shown, the computer-readable storage medium 30 stores program instructions 31 thereon, and when the program instructions 31 are executed by a processor (not shown), the map construction method in the above embodiment is implemented.

[0087] The computer-readable storage medium 30 of this embodiment can be, but is not limited to, a USB flash drive, an SD card, a PD optical drive, a mobile hard disk, a large-capacity floppy drive, a flash memory, a multimedia memory card, a server, a storage unit in an FPGA or an ASIC, etc.

[0088] Different from the prior art, in the technical solution of the present application, image features of a target bird's-eye view image are acquired, closed graphics in the image features of the target bird's-eye view image are detected and identified based on a closed graphics detection model to obtain closed graphics parameters, non-closed graphics in the image features of the target bird's-eye view image are detected and identified based on a non-closed graphics detection model to obtain non-closed graphics parameters, and a map for identifying the closed graphics and the non-closed graphics is constructed based on the closed graphics parameters, the non-closed graphics parameters and the image features of the target bird's-eye view image, wherein a parameter category of the closed graphics parameters is different from a parameter category of the non-closed graphics parameters. Based on the above method, different models can be used to detect and identify closed figures and non-closed figures respectively, that is, the closed figure detection model is used to detect and identify the closed figure and output the corresponding closed figure parameters suitable for identifying the closed figure, and the non-closed figure detection model is used to detect and identify the non-closed figure and output the corresponding non-closed figure parameters suitable for identifying the non-closed figure. The closed figure parameters are different from the non-closed figure parameters, so that in the subsequent steps, the closed figure can be identified based on the closed figure parameters adapted to the closed figure, and the non-closed figure can be identified based on the non-closed figure parameters adapted to the non-closed figure, so as to construct a map in which both closed and non-closed figures are reasonably identified, thereby improving the accuracy of map identification.

[0089] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradicting each other.

[0090] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0091] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0092] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (which can be a personal computer, server, network device or other system that can fetch instructions from the instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0093] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A map construction method, characterized in that: include: Obtain image features of a target bird's-eye view image; Based on a closed figure detection model, closed figures in the image features of the target bird's-eye view image are detected and identified to obtain closed figure parameters; Based on a non-closed figure detection model, detecting and identifying non-closed figures in the image features of the target bird's-eye view image to obtain non-closed figure parameters; Based on the closed figure parameters, the non-closed figure parameters and the image features of the target bird's-eye view image, constructing a map for marking the closed figure and the non-closed figure; The parameter category of the closed graphic parameter is different from the parameter category of the non-closed graphic parameter.

2. The map construction method according to claim 1, characterized in that: The closed figure parameter includes at least one of size information of the closed figure, position information of the closed figure, angle information of the closed figure, and category information of the closed figure; The non-closed figure parameter includes at least one of category information of the non-closed figure and point information of the non-closed figure.

3. The map construction method according to claim 1 or 2, characterized in that: The closed figure detection model comprises at least one closed detection layer group and at least one convolutional neural network layer connected in sequence, the input end of the first closed detection layer group is the input end of the closed figure detection model, the output end of the last closed detection layer group is connected to the input end of the convolutional neural network layer, and the output end of the convolutional neural network is the output end of the closed figure detection model; The closed detection layer group includes a self-attention layer, a cross-attention layer and a feedforward neural network layer connected in sequence, the self-attention layer is the input end of the closed detection layer group, and the feedforward neural network layer is the output end of the closed detection layer group; The input end of the first closed detection layer group is used to receive the image features of the target bird's-eye view image, and the output end of the last closed detection layer group is used to output the closed graphic parameters.

4. The map construction method according to claim 1 or 2, characterized in that: The non-closed figure detection model comprises at least one non-closed detection layer group connected in sequence, the input end of the first non-closed detection layer group is the input end of the non-closed figure detection model, and the output end of the last non-closed detection layer group is the output end of the non-closed figure detection model; The non-closed detection layer group includes a multi-head self-attention layer, a first normalization layer, a deformable cross attention layer, a second normalization layer, a non-closed graphic parameter self-attention layer and a third normalization layer connected in sequence, the input end of the multi-head self-attention layer is the input end of the non-closed detection layer group, and the output end of the third normalization layer is the output end of the non-closed detection layer group; Among them, the multi-head self-attention layer in the first non-closed detection layer group is used to receive closed graphic features in the image features of the target bird's-eye view image, the multi-head self-attention layers in the other non-closed detection layer groups except the first non-closed detection layer group are used to receive the results output by the previous multi-head self-attention layer, the deformable cross attention layer is also used to receive the image features of the target bird's-eye view image, the second normalization layer is also used to receive the image features of the target bird's-eye view image and the results output by the first normalization layer, the third normalization layer is also used to receive the results output by the second normalization layer, and the third normalization layer in the last non-closed detection layer group is used to output the non-closed graphic parameters.

5. The map construction method according to claim 4, characterized in that: The non-closed figure parameters include category information of the non-closed figure and point information of the non-closed figure; The non-closed graphic parameter self-attention layer includes a category self-attention layer and a point self-attention layer, the third normalization layer includes a first sub-normalization layer and a second sub-normalization layer, the category self-attention layer, the first sub-normalization layer, the point self-attention layer and the second sub-normalization layer are connected in sequence, the first sub-normalization layer and the second sub-normalization layer are also used to receive the result output by the second normalization layer, the input end of the category self-attention layer is the input end of the non-closed graphic parameter self-attention layer, and the output end of the second sub-normalization layer is the output end of the third normalization layer; The output end of the second sub-normalization layer in the last non-closed detection layer group is used to output the category information of the non-closed figure and the point information of the non-closed figure.

6. The map construction method according to claim 1 or 2, characterized in that: The step of acquiring image features of the target bird's-eye view image includes: Acquire multi-view images, and obtain bird's-eye view features of the images based on the multi-view images; Acquire point cloud data, and obtain radar bird's-eye view features based on the point cloud data; Feature fusion is performed based on the image bird's-eye view features and the radar bird's-eye view features to obtain image features of the target bird's-eye view image.

7. The map construction method according to claim 6, characterized in that: The method of obtaining the bird's-eye view features of the image based on the multi-view image includes: Inputting the multi-view image into an image backbone network model to obtain image features of the multi-view image; Inputting the image features of the multi-view image into a view converter to obtain the image bird's-eye view features; And / or, obtaining radar bird's-eye view features based on the point cloud data includes: The point cloud data is input into a radar backbone network model to obtain the radar bird's-eye view features.

8. The map construction method according to claim 1 or 2, characterized in that: Before detecting and identifying the closed graphics in the image features of the target bird's-eye view image based on the closed graphics detection model to obtain the closed graphics parameters, the map construction method further includes: Acquire a plurality of closed training samples; wherein the closed training samples include target bird's-eye view image samples or image feature samples of the target bird's-eye view image, and closed graphic parameter samples; Based on the closed training samples, training the closed figure detection model; And / or, before detecting and identifying the non-closed graphics in the image features of the target bird's-eye view image based on the non-closed graphics detection model to obtain the non-closed graphics parameters, the map construction method further includes: Acquire a plurality of non-closed training samples; wherein the non-closed training samples include target bird's-eye view image samples or image feature samples of the target bird's-eye view image, and non-closed graphic parameter samples; The non-closed figure detection model is trained based on the non-closed training samples.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store program instructions, and the processor is used to execute the program instructions to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images