Map element positioning identification method and device, equipment, medium and product

Abstract

The invention provides a map element positioning identification method, device, equipment, medium and product, and is applied to the field of high-precision maps, and the method comprises the steps: determining geometric feature information used for describing the spatial information of each map element in a road image based on a map element detection result of road image data and a size corresponding to the road image data; based on a road type detection result of the road image data, determining type feature information used for representing a road type; inputting the geometric feature information and the type feature information into a preset map element positioning recognition model to obtain a map element positioning recognition result corresponding to the road image data; wherein the map element positioning identification model is a model used for identifying the position of the map element on the road. According to the scheme, the positioning identification precision of the map elements can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of high-precision map technology, and in particular to a method, device, equipment, medium and product for positioning and identifying map elements. Background Art

[0002] With the continuous development of science and technology, the demand for intelligent road construction is becoming increasingly strong. How to efficiently and quickly build high-precision maps has become an important research object.

[0003] In the process of building high-precision maps, it involves the extraction of various map elements, such as traffic lights and vehicle signals in the air, and speed limit signs and point speed limits in the ground. In the process of extracting the corresponding map elements, it is necessary to accurately determine the location information of the elements on the road. At present, the road image is mainly identified and located through predefined rules (such as standardized conditions in terms of element shape, size, relative position, direction, etc.).

[0004] However, in scenarios with diverse and complex road structures, the accuracy of feature positioning and recognition is difficult to guarantee due to the lack of flexibility and adaptability of these rules. Therefore, there is an urgent need to provide a technical solution that can effectively improve the accuracy of map feature positioning and recognition. Summary of the invention

[0005] The present application provides a map element positioning and identification method, device, equipment, medium and product to solve at least one of the above-mentioned technical problems.

[0006] According to one aspect of the present application, a method for positioning and identifying map elements is provided, comprising:

[0007] Determining geometric feature information for describing spatial information of each map element in the road image based on the map element detection result of the road image data and the size corresponding to the road image data;

[0008] Determining type feature information for characterizing the road type based on the road type detection result of the road image data;

[0009] Inputting the geometric feature information and the type feature information into a preset map element positioning and recognition model to obtain a map element positioning and recognition result corresponding to the road image data, wherein the map element positioning and recognition result is used to indicate the position of the map element on the road;

[0010] The map element positioning and recognition model is a model used to recognize the position of map elements on the road.

[0011] In a possible implementation, the map element detection result includes the coordinates of the element detection frame and the element category; the map element detection result based on the road image data and the size corresponding to the road image data determines the geometric feature information used to describe the spatial information of each map element in the road image, including:

[0012] Determine the size and size ratio of each map element according to the element detection frame coordinates of each map element;

[0013] Determining the pixel distance between each map element and the road image data about the image boundary according to the size of each map element and the size corresponding to the road image data;

[0014] Determining the pixel ratio of each map element to the road image data with respect to the image boundary according to the pixel distance corresponding to each map element and the size corresponding to the road image data;

[0015] The geometric feature information is obtained based on the pixel distance and pixel ratio corresponding to each map element, the element size and size ratio, the element category and the size corresponding to the road image data.

[0016] In a possible implementation manner, the road image data includes a road video; and the method for obtaining the map element detection result and / or the road type detection result includes:

[0017] Extracting at least one video frame of the road video;

[0018] Inputting the video frame into a preset feature detection model and / or a preset road type classification model; wherein the feature detection model is a model obtained by training based on historical road videos and used to detect map feature features, and the road type classification model is a model obtained by training based on historical road videos and used to detect road type features;

[0019] The video frame is processed based on the feature detection model to obtain the map feature detection result, and / or the video frame is processed based on the road type classification model to obtain the road type detection result.

[0020] In a possible implementation, the road type classification model is trained by:

[0021] Collecting historical road video data, and extracting multiple video frames of the historical video data to obtain an initial video frame set;

[0022] Performing deduplication processing on the video frames in the initial video frame set to obtain a first video frame set;

[0023] Based on the first video frame set and its corresponding road type classification annotation information, an initial road type model is trained to obtain the road type classification model.

[0024] In a possible implementation, the training method of the element detection model includes:

[0025] Based on the road type classification model, deduplication is performed on the first video frame set to select video frames with different road features from the first video frame set to obtain a second video frame set;

[0026] Based on the second video frame set and its corresponding element annotation information, an initial element detection model is trained to obtain the element detection model.

[0027] In a possible implementation, the road type detection model includes an input end for inputting a video frame, a backbone network for processing the video frame, and an output end for outputting a road type detection result; wherein,

[0028] The input end includes a spatial transformation network module for performing jitter elimination on an input video frame, and the output end includes a first fully connected layer for predicting a road scene and a second fully connected layer for identifying a road type.

[0029] In a possible implementation manner, the road type detection result includes a multi-dimensional feature output from a road detection model for characterizing the road type;

[0030] The road type detection result based on the road image data determines the type feature information used to characterize the road type, including:

[0031] According to the principal component analysis technique, the multi-dimensional features are reduced to low-dimensional features of a preset number of dimensions; and

[0032] Based on the low-dimensional features, the type feature information is obtained.

[0033] In a possible implementation, the training method of the map element location recognition model includes:

[0034] Extracting historical geometric feature information and historical type feature information of the historical road image data from the feature library to construct a training data set; wherein the historical geometric feature information is determined based on the feature detection result, and the historical type feature information is determined based on the road type detection result;

[0035] Based on the training data set and the element positioning and recognition results corresponding to each training data, an initial map element positioning and recognition model is trained to obtain the map element positioning and recognition model.

[0036] According to a second aspect of the present application, a map element positioning and identification device is provided, comprising:

[0037] A geometric feature determination module, which is configured to determine geometric feature information for describing spatial information of each map element in the road image based on a map element detection result of the road image data and a size corresponding to the road image data;

[0038] A type feature determination module, configured to determine type feature information for characterizing a road type based on a road type detection result of the road image data;

[0039] a model processing module, which is configured to input the geometric feature information and the type feature information into a preset map element positioning and recognition model to obtain a map element positioning and recognition result corresponding to the road image data, wherein the map element positioning and recognition result is used to indicate the position of the map element in the road;

[0040] The map element positioning and recognition model is a model used to recognize the position of map elements on the road.

[0041] In a possible implementation, the map element detection result includes element detection frame coordinates and element categories; the geometric feature determination module includes:

[0042] A first determining unit, configured to determine the size and size ratio of each map element according to the element detection frame coordinates of each map element;

[0043] A second determining unit, configured to determine a pixel distance between each map element and the road image data about an image boundary according to a size of each map element and a size corresponding to the road image data;

[0044] A third determining unit is configured to determine a pixel ratio of an image boundary between each map element and the road image data according to a pixel distance corresponding to each map element and a size corresponding to the road image data;

[0045] The geometric feature acquisition unit is configured to obtain the geometric feature information based on the pixel distance and pixel ratio corresponding to each map element, the element size and size ratio, the element category and the size corresponding to the road image data.

[0046] In a possible implementation manner, the road image data includes a road video; the device further includes a detection module, and the detection module includes:

[0047] an extraction unit, configured to extract at least one video frame of the road video;

[0048] A video frame input unit, configured to input the video frame into a preset feature detection model and / or a preset road type classification model; wherein the feature detection model is a model obtained by training based on historical road videos and used to detect map feature features, and the road type classification model is a model obtained by training based on historical road videos and used to detect road type features;

[0049] A video frame processing unit is configured to process the video frame based on the feature detection model to obtain the map feature detection result, and / or to process the video frame based on the road type classification model to obtain the road type detection result.

[0050] In a possible implementation, the device further includes a first model training module, and the first model training module includes:

[0051] A data acquisition unit, which is configured to acquire historical road video data and extract multiple video frames of the historical video data to obtain an initial video frame set;

[0052] A first deduplication unit, configured to perform deduplication processing on the video frames in the initial video frame set to obtain a first video frame set;

[0053] The first training unit is configured to train an initial road type model based on the first video frame set and its corresponding road type classification annotation information to obtain the road type classification model.

[0054] In a possible implementation, the device further includes a second model training module, and the second model training module includes:

[0055] A second deduplication unit is configured to deduplicate the first video frame set based on the road type classification model, so as to select video frames with different road features from the first video frame set to obtain a second video frame set;

[0056] The second training unit is configured to train an initial feature detection model based on the second video frame set and its corresponding feature annotation information to obtain the feature detection model.

[0057] In a possible implementation, the road type detection model includes an input end for inputting a video frame, a backbone network for processing the video frame, and an output end for outputting a road type detection result; wherein,

[0058] The input end includes a spatial transformation network module for performing jitter elimination on an input video frame, and the output end includes a first fully connected layer for predicting a road scene and a second fully connected layer for identifying a road type.

[0059] In a possible implementation, the road type detection result includes multidimensional features output from the road detection model for characterizing the road type; the type feature determination module is specifically configured to reduce the multidimensional features to low-dimensional features of a preset number of dimensions based on principal component analysis technology; and, based on the low-dimensional features, obtain the type feature information.

[0060] In a possible implementation, the device further includes a third model training module, and the third model training module includes:

[0061] A training set construction unit, which is configured to extract historical geometric feature information and historical type feature information of the historical road image data from a feature library to construct a training data set; wherein the historical geometric feature information is determined based on a feature detection result, and the historical type feature information is determined based on a road type detection result;

[0062] The third training unit is configured to train an initial map element positioning and recognition model based on the training data set and the element positioning and recognition results corresponding to each training data to obtain the map element positioning and recognition model.

[0063] According to the third aspect of the present application, an electronic device is provided, comprising: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the map element positioning and identification method provided in any one of the above-mentioned first aspects.

[0064] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the map element positioning and identification method as described in any one of the above-mentioned first aspects.

[0065] According to a fifth aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, which, when executed by a processor, implements a method for positioning and identifying map elements as provided in any one of the first aspects above.

[0066] The map element positioning and identification method, device, equipment, medium and product provided in the present application jointly consider the element geometric characteristics and road type characteristics of the road image data, and the original size information of the road image data is also considered in the process of determining the element geometric characteristics, so that the constructed features are richer. The map element positioning and identification model is used to locate and identify elements based on element geometric characteristics and road type characteristics, which can effectively deal with the complexity and diversity of road scenes and effectively improve the positioning and identification accuracy of map elements. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0068] Figure 1 A flowchart of a method for locating and identifying map elements provided in an embodiment of the present application;

[0069] Figure 2 A schematic diagram of a flow chart of another method for positioning and identifying map elements provided in an embodiment of the present application;

[0070] Figure 3 This is an example diagram of the MAE model in the embodiment of the present application;

[0071] Figure 4 This is an example diagram of a road type detection model in an embodiment of the present application;

[0072] Figure 5 A schematic diagram of a flow chart of another method for positioning and identifying map elements provided in an embodiment of the present application;

[0073] Figure 6 A possible scenario diagram provided for an embodiment of the present application;

[0074] Figure 7 A schematic diagram of the structure of a map element positioning and identification device provided in an embodiment of the present application;

[0075] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0076] Fig. 9 It is a block diagram of a terminal device provided by an exemplary embodiment of the present application.

[0077] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0078] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0079] It can be understood that in the embodiments of the present application, the term "and / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0080] The following is an explanation of the embodiments of the present application in conjunction with the application scenarios. The map element positioning and identification method provided in the embodiments of the present application can be applied to the application scenarios of intelligent driving, more specifically, it can be applied to the application scenarios of autonomous driving based on vehicle cloud computing. For example, the execution subject of the method provided in the embodiments of the present application can be a server or an embedded device, more specifically, for example, a server of a high-precision map producer, based on the road image data (video or image) obtained by the road test equipment (or other equipment such as satellite equipment, etc.), a high-precision map is constructed. This embodiment introduces the server as the execution subject of the method provided in the embodiments of the present application. In some application scenarios, it can also be applied to various types of vehicle cameras to collect road videos for element detection, and obtain the detection results to disk to construct map traffic information, and facilitate the relevant algorithm development and test personnel to review the map element positioning information, etc. This embodiment does not specifically limit the application scenarios of the map element positioning and identification solution.

[0081] In the process of building high-precision maps, related technologies use traditional image feature extraction methods to extract feature target frames from video frames and judge the position of features on the road based on some predefined rules. When the road is complex and the acquisition scenes are diverse, it is easy to fail in unforeseen scenes, making it difficult to ensure the accuracy of position judgment. In addition to identifying map elements based on predefined rules, manual review methods are also used to identify map elements required for building high-precision maps and the positions of corresponding elements on the road from the returned video frame sequence. Although map elements can be positioned and identified more accurately, it requires a lot of manpower and material resources, the labor cost is too high, and the degree of automation is low. In other related technologies, deep neural network algorithms are used to extract feature target frames, and the position prediction is output directly based on the input of the original image. For the above-mentioned scheme, the algorithm is automated to a certain extent, but because the input is only based on the original image without considering other effective road information, it is difficult to converge and generalize in complex and diverse road scenes, so the recognition accuracy is still not ideal.

[0082] To this end, the map element positioning and recognition scheme proposed in the embodiment of the present application determines the geometric feature information used to describe the spatial information of each map element in the road image based on the map element detection result of the road image data and the size corresponding to the road image data, determines the type feature information used to characterize the road type based on the road type detection result of the road image data, inputs the geometric feature information and the type feature information into the preset map element positioning and recognition model, obtains the map element positioning and recognition result corresponding to the road image data, and the map element positioning and recognition result is used to indicate the position of the map element in the road, and the map element positioning and recognition model is a model for identifying the position of the map element on the road. In this process, the element geometric features and the road type features are jointly considered, and the element geometric features not only consider the element detection results but also consider the original size information of the road image data, and its features are richer. Using the map element positioning and recognition model to locate and recognize the elements according to the element geometric features and road type features can effectively solve the problem of difficulty in accurately identifying map elements in scenes such as complex roads.

[0083] The following is a detailed description of the technical solution of the present application and how the technical solution of the present application solves the above technical problems in conjunction with the accompanying drawings and specific embodiments. It should be noted that these specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0084] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0085] Figure 1 A method for positioning and identifying map elements provided in an embodiment of the present application may be executed by a server, such as Figure 1 As shown, the method may include steps S101-S103:

[0086] Step S101: Determine geometric feature information for describing spatial information of each map element in the road image based on the map element detection result of the road image data and the size corresponding to the road image data.

[0087] In this embodiment, the road image data can be a single-frame road image or a road video. In the single-frame road image scenario, after the server obtains the image, it can directly process the image (such as map feature detection, road type detection, etc.). In the road video scenario, the extracted video frames can be processed by extracting frames. This embodiment takes the extraction of video frames as an example for explanation.

[0088] In this embodiment, the map element detection result can be obtained by server detection (for example, by using a map element detection model to identify the road image data), or it can be detected by other devices and transmitted to the server for direct use. The map element detection result is the map element detection result for the extracted video frame image, and the size corresponding to the road image data is the same. Optionally, the size corresponding to the road image data is the original size of the road image data, that is, the size of the extracted video frame image, such as the height and width of the video frame image.

[0089] In this embodiment, the spatial information of each map element in the road image, that is, the size, position, shape and other geometric features of each map element in the image space, such as the pixel distance and pixel ratio corresponding to each map element, the element size and size ratio, the element category and the size corresponding to the road image data, etc.

[0090] In a possible implementation, the map element detection result may include the element detection frame coordinates and the element category. The above step S101 determines the geometric feature information used to describe the spatial information of each map element in the road image based on the map element detection result of the road image data and the size corresponding to the road image data, and may adopt the following method:

[0091] Determine the size and size ratio of each map element according to the element detection frame coordinates of each map element;

[0092] Determining the pixel distance between each map element and the road image data about the image boundary according to the size of each map element and the size corresponding to the road image data;

[0093] Determining the pixel ratio of each map element to the road image data with respect to the image boundary according to the pixel distance corresponding to each map element and the size corresponding to the road image data;

[0094] The geometric feature information is obtained based on the pixel distance and pixel ratio corresponding to each map element, the element size and size ratio, the element category and the size corresponding to the road image data.

[0095] Exemplarily, the map element detection result may be the result output by the element detection model (in some instances, the map element detection result may also be a detection result obtained based on rules or other detection methods), and the element detection result output by the element detection model includes the element detection frame coordinates (it can be understood that the element detection frame is used to identify the location of the map element, and the element detection frame coordinates are the coordinates of each vertex of the detection frame. After the road image is processed by the map element detection model, the model can directly output the element detection frame and the corresponding coordinates, element type and other information. In other embodiments, based on the detection results obtained by other detection methods, the element detection frame can be annotated with an annotation tool, and the element detection frame coordinates can be obtained through a positioning algorithm), wherein the element detection frame coordinates such as the upper left corner and lower right corner coordinates of the element frame are (lx, ly) and (rx, ry), respectively, and the element detection result includes the element category id_j, the size corresponding to the road image data, that is, the input video frame width and height of the model are w_o and h_o, respectively. Among them, the calculation of the size and size ratio of each map element (taking the calculation of one of the elements as an example) is:

[0096] The width of the element w_j = rx–lx;

[0097] The height of the element h_j = ry – rx;

[0098] The aspect ratio of the element: ration_wh = w_j / h_j;

[0099] Next, using the size of the map feature and the size of the video frame image, calculate the pixel distance of the feature from the edge of the image, as well as the ratio of the edge pixel distance to the corresponding image size:

[0100] The pixel distance between the left edge of the feature frame and the left edge of the video frame image is diff_lx=lx;

[0101] The pixel distance between the upper boundary of the feature frame and the upper boundary of the video frame image is diff_ly=ly;

[0102] The pixel distance between the right edge of the feature frame and the right edge of the video frame image is diff_rx=w_o–rx;

[0103] The pixel distance between the lower boundary of the feature frame and the lower boundary of the video frame image is diff_ry=h_o–ry;

[0104] The ratio of the pixel distance from the left edge of the feature to the image width ratio_diff_lx = diff_lx / w_o;

[0105] The ratio of the pixel distance on the feature edge to the image height ratio_diff_ly=diff_ly / h_o;

[0106] The ratio of the pixel distance of the right edge of the feature to the image width ratio_diff_rx=diff_rx / w_o;

[0107] The ratio of the pixel distance of the lower edge of the feature to the image height is ratio_diff_ry=diff_ry / h_o.

[0108] It can be seen that after the above processing, the geometric feature information of the map element includes at least 14 types of features, namely, feature width w_j, feature height h_j, feature aspect ratio ration_wh, feature type id_j, video frame width w_o, video frame height h_o, left border pixel distance diff_lx, upper border pixel distance diff_ly, right border pixel distance diff_rx, lower border pixel distance diff_ry, left border pixel ratio ratio_diff_lx, upper border pixel ratio ratio_diff_ly, right border pixel ratio ratio_diff_rx and lower border pixel ratio ratio_diff_ry. The geometric feature information of the element obtained by combining any of the above features has richer feature details than the detection information containing only the element itself, which provides data support for subsequent feature positioning and identification.

[0109] In some embodiments, in addition to obtaining the geometric feature information of the elements in the above manner, the map element detection results of the road image data and the corresponding size of the road image data can also be directly used to determine the geometric feature information used to describe the spatial information of each map element in the road image. For example, the geometric feature information may only include the element size and element type corresponding to the element detection result, and the original size of the road image data, etc.

[0110] Step S102: Determine type feature information for characterizing the road type based on the road type detection result of the road image data.

[0111] In the related art, in the process of locating and identifying road elements, the type of road is rarely considered. However, as roads become more diversified and complicated, it may be difficult for the recognition algorithm to accurately locate and identify map elements. This embodiment takes into account that different types of roads (such as highways, urban streets, rural roads, etc.) have differences in width, number of lanes, signs and markings, curb design, etc. These feature differences will affect the positioning and recognition accuracy of the elements. For example, the lane lines and signs and markings of highways and urban streets may be significantly different. Therefore, in the process of locating and identifying map elements, in addition to considering the geometric feature information of map elements, this embodiment also considers the road type information corresponding to the road image data, so as to combine the feature geometric feature information and road type feature information in the subsequent process to achieve the positioning and recognition of map elements.

[0112] Similar to the map element detection result, the road type detection result in this embodiment may be obtained through model output, or may be obtained based on rules or other detection algorithms, which is not particularly limited in this embodiment.

[0113] In a possible implementation, the road type detection result includes multidimensional features output from the road detection model for characterizing the road type. The above step S102 determines the type feature information for characterizing the road type based on the road type detection result of the road image data. The following method can be used: according to the principal component analysis technology, the multidimensional features are reduced to low-dimensional features of a preset number of dimensions; and, based on the low-dimensional features, the type feature information is obtained.

[0114] Principal Component Analysis (PCA) is a statistical technique used to simplify the dimensions of a data set while retaining the most important variation information in the data. PCA transforms the original data into a new coordinate system so that the maximum variance of the data is projected on the first few axes of the new coordinate system.

[0115] It can be understood that for road images, multidimensional features for characterizing road types are extracted from the road detection model, which usually include road width, number of lanes, type and position of signs and markings, curb features, etc. The detection results contain a lot of information. Usually, the more feature dimensions there are in the detection results, the richer the information. For example, the road type detection model can be designed to output 512-dimensional features, but directly using the output multidimensional features for subsequent processing will result in excessive computing power requirements. This embodiment uses PCA technology to reduce the dimensionality of the 512-dimensional features, for example, to 10-dimensional features. The 10-dimensional features are used to characterize the corresponding road type id_r, which can not only retain rich road feature details, but also effectively reduce the impact of computing power, thereby improving the efficiency of feature positioning and recognition.

[0116] In some possible implementations, in addition to determining the road type characteristic information in the above-mentioned manner, the road type detection results can also be directly used as the road type characteristic information, or the detection results can be processed in other ways to determine the road type characteristic information, such as by using feature selection technology, clustering technology, etc., so as to provide type characteristic information with rich details and less redundant data.

[0117] Step S103: input the geometric feature information and type feature information into a preset map element positioning and recognition model to obtain a map element positioning and recognition result corresponding to the road image data, wherein the map element positioning and recognition result is used to indicate the position of the map element on the road, wherein the map element positioning and recognition model is a model for identifying the position of the map element on the road.

[0118] After obtaining the geometric feature information and road type feature information of the element, the feature data are jointly input into a preset map element positioning and recognition model. The map element positioning and recognition model can be trained in the server using the historical geometric feature information and historical type feature information of historical road image data, or it can be obtained from other servers or devices. This embodiment does not specifically limit the specific training subject of the map element positioning and recognition model.

[0119] Compared with the positioning and identification methods of map elements in related technologies, the road image data is usually detected directly using the map element detection model, and the road image is identified and located in combination with predefined rules (such as standardized conditions in terms of element shape, size, relative position, direction, etc.). When new scenes appear in scenes such as diverse road structures and complex environments, it will be difficult to accurately locate map elements due to the lack of flexibility and adaptability of these rules. This embodiment combines the geometric feature information of the elements and the feature information of the road type, and uses the map element positioning and identification model to locate and identify the elements. It can efficiently locate and identify map elements in scenes such as diversified and complex roads, thereby improving the accuracy of high-precision map construction.

[0120] Figure 2 This is a flowchart of another method for positioning and identifying map elements provided by an embodiment of the present application. Based on the above embodiment, this embodiment illustrates a method for obtaining map element detection results and road type detection results on the server side. By extracting frames from road videos and inputting them into corresponding detection models for feature monitoring, map element detection results and road type detection results are obtained, effectively improving feature extraction accuracy and enriching feature information of map elements and road types. Specifically, in addition to the above steps S101-S103, this embodiment also includes a method for obtaining map element detection results and / or road type detection results before step S201, specifically, including the following steps S201-S203:

[0121] Step S201: After acquiring road image data, the road image data includes a road video, and extracting at least one video frame of the road video.

[0122] In this embodiment, one or more video frames in the road video may be randomly extracted, or frames may be extracted according to specific events or conditions. For example, when a specific object (such as a map element) is detected in the video, extraction is performed. This embodiment does not specifically limit the method of extracting video frames.

[0123] The road image data may be acquired in real time by a collection vehicle, or may be existing road image data acquired from a database.

[0124] Step S202: input the video frame into a preset feature detection model and / or a preset road type classification model; wherein the feature detection model is a model obtained by training based on historical road videos and used to detect map feature features, and the road type classification model is a model obtained by training based on historical road videos and used to detect road type features.

[0125] Exemplarily, the feature detection model can be obtained by training with a lightweight yolov10 detection network, or other detection networks such as a Deform-DETR network, a Centernet network, etc. can be used; the road type classification model can be obtained by training with a relatively lightweight classification network resnet, such as a resnet18 network, or lightweight models such as squeezenet use other classification networks, which are not particularly limited in this embodiment.

[0126] It should be noted that the various models involved in this implementation can be trained by a server, or can be trained by other devices and stored or transmitted to the server for direct use. This embodiment does not specifically limit this, and this embodiment only takes training on a server as an example for explanation.

[0127] Step S203: Process the video frame based on the feature detection model to obtain the map feature detection result, and / or process the video frame based on the road type classification model to obtain the road type detection result.

[0128] In some examples, when multiple video frames are extracted, each video frame can be input into the feature detection model, and the feature detection model can be used to process a single frame image, and the processing results of each frame image can be combined to obtain the map feature detection result and the road type detection result. For example, the processing results are fused on the time axis, such as using weighted average, voting mechanism or Bayesian method to combine the detection results of multiple frames, which can reduce the noise and error in the single frame detection and improve the robustness of the overall detection.

[0129] It should be noted that the above steps S201-S203 include at least three example methods, such as obtaining map element detection results based only on the above methods, or obtaining road type detection results based only on the above methods, or obtaining map element detection results and road type detection results based on the above methods.

[0130] By extracting frames and inputting the video frames into the relevant model for processing, an automated and efficient detection process of map key features and road type features can be achieved, thereby obtaining rich feature information, which is convenient for the subsequent calculation or determination of geometric feature information and type feature information.

[0131] Next, this embodiment further introduces the training method of the road type classification model:

[0132] Collecting historical road video data, and extracting multiple video frames of the historical video data to obtain an initial video frame set;

[0133] Performing deduplication processing on the video frames in the initial video frame set to obtain a first video frame set;

[0134] Based on the first video frame set and its corresponding road type classification annotation information, an initial road type model is trained to obtain the road type classification model.

[0135] In this embodiment, the historical video data may be road video data collected and transmitted back by a collection vehicle, and the video data may be extracted to obtain corresponding video frames. Frame extraction may be performed at fixed time intervals. Frame extraction may effectively reduce redundant information and the amount of data. Video data may be managed and processed more effectively to ensure that the model is prevented from overfitting during model training.

[0136] For the initial video frame set after frame extraction, features can be extracted through a self-supervised model (Masked Autoencode, MAE) to obtain image features of the video frames and remove duplicates. Specifically, the size of the video frames in the video frame set after frame extraction can be scaled to 256*256, and the patch_size in the MAE model can be set to 16*16. The video frames are deduplicated according to the 256-dimensional features output by the encoder module in the MAE model. During the deduplication process, the differentiated video frames can be filtered and retained based on the distance threshold hyperparameter pre-set by the MAE model to retain the differentiated video frames and obtain the first video frame set. The MAE model example is as follows: Figure 3As shown. It can be understood that the distance threshold hyperparameter combines clustering or similarity measurement, and the distance threshold can be used to determine which data points are considered similar or belong to the same class. Those skilled in the art can adaptively adjust the distance threshold hyperparameter in combination with practical applications or prior data. In some embodiments, in addition to using the MAE model to deduplicate the initial video frame set, other methods can also be used for deduplication, such as calculating the pixel difference between adjacent frames to determine the degree of change in the frame. If the difference is less than a certain threshold, the frame is considered to be redundant and is removed, etc.

[0137] After deduplication to obtain the first video frame set, the first video frame set is subjected to road type classification annotation. The road type classification annotation can be annotated by an annotation tool or manually annotated. The annotation can include category labels, such as "highway", "urban road", "rural road", etc., which are not particularly limited in this embodiment. As mentioned above, the initial road type classification model can adopt the resnet18 network, use the first video frame set and its corresponding road type classification annotation information, iteratively train the initial road type model, optimize the parameters and weights in the initial road type model, and obtain the road type classification model. During the training process, a loss function suitable for the classification task can be selected, such as cross entropy loss, and an optimization algorithm can be selected, such as SGD (stochastic gradient descent), Adam, etc., and hyperparameters such as learning rate can be set. Then, the model parameters are iteratively updated using the training data (video frames and their annotations). In addition, the data can be divided into a training set and a validation set to monitor the performance of the model and prevent overfitting.

[0138] Through the above technical solution, the redundancy of the extracted video frames is removed based on the joint self-supervised image features, which reduces the repeated retention and annotation number of video frames and improves the training efficiency and accuracy of the road type model.

[0139] In a possible implementation, in order to improve the accuracy of the feature detection model, this embodiment uses the above-mentioned road type classification model to perform secondary deduplication on the first video frame set used to train the feature detection model, and uses the secondary deduplication data set to train the feature detection model. Specifically, the training method of the feature detection model can be as follows:

[0140] Based on the road type classification model, deduplication is performed on the first video frame set to select video frames with different road features from the first video frame set to obtain a second video frame set;

[0141] Based on the second video frame set and its corresponding element annotation information, an initial element detection model is trained to obtain the element detection model.

[0142] In this embodiment, in order to further reduce the data redundancy in the training data set, the above-mentioned trained road type classification model is used to infer the video frames after the first round of deduplication and extract the 512-dimensional features to calculate the cosine distance for secondary screening. Specifically, for each video frame in the first video frame set, the trained road type classification model is used to extract features. The model infers each video frame to generate a 512-dimensional feature vector. These feature vectors are high-dimensional numerical representations that capture the key features and information of each video frame, and select video frames that meet the preset road conditions. Then, for each pair of feature vectors of video frames, the cosine distance between them is calculated. The cosine distance is an indicator that measures the similarity between two vectors. The smaller the value, the more similar the two vectors are, and the larger the value, the less similar they are. The cosine distance is used for secondary screening, and the goal is to remove those video frames whose feature vectors are too similar to reduce data redundancy. A cosine distance threshold can be set according to actual applications or prior data, and frame pairs with cosine distances less than the threshold are removed, or the most representative one in each group of similar frames is selected to complete the second deduplication. Finally, video frames with various road types or overall backgrounds are obtained, which are used to train feature detection models, so that the feature detection models can perform better in complex road structures.

[0143] Furthermore, the road type detection model of this embodiment includes an input end for inputting video frames, a backbone network for processing video frames, and an output end for outputting road type detection results; wherein, the input end includes a spatial transformation network module for eliminating jitter on the input video frames, and the output end includes a first fully connected layer for predicting road scenes and a second fully connected layer for identifying road types.

[0144] For example, the structural diagram of the road type detection model is as follows: Figure 4 As shown, the spatial transformation network module (STN) module is a differentiable module that can perform spatial transformation on input data without explicit annotation, thereby improving the robustness and performance of the model. In this embodiment, the STN module is inserted into the input end of the road type detection model to dynamically adjust the geometric properties of the input video frame (such as rotation, scaling, translation, etc.), so that the model can better adapt to different input changes, thereby reducing the impact of camera jitter during data acquisition.

[0145] In addition to inserting the STN module at the input end to reduce data jitter images, this embodiment constructs two parallel fully connected layers at the output end, which are used to predict road scene categories (for example: highways, intersections, urban main roads, expressways, rural roads, etc.) and identify road types (one-way streets, two-way streets, straight roads, curves, etc.), so that the model can handle different tasks, improve the richness of road type features through branch prediction, and decouple tasks based on the design of parallel fully connected layers. The model can be optimized independently on each task, which effectively reduces the competition between categories at the output end and improves the richness of feature expression.

[0146] It can be understood that the road type feature information in this embodiment may include the road scene features and road type features output by the above-mentioned two fully connected layers. For example, during the training process of the above-mentioned feature detection model, the road type detection model is used to perform secondary deduplication on the first video frame to obtain video frame images with diverse road types or diverse overall road scenes.

[0147] Figure 5 is a flowchart of another method for locating and identifying map elements provided by an embodiment of the present application. Based on the above embodiment, this embodiment trains a map element locating and identifying model on the server side, and trains the model using historical geometric feature information determined by the feature detection results and historical type feature information determined by the road type detection results. Specifically, in addition to the above steps S101-S103, this embodiment also includes a training method for the map element locating and identifying model, such as Figure 5 As shown, the following steps S501 and S502 are also included:

[0148] Step S501, extracting historical geometric feature information and historical type feature information of the historical road image data from a feature library to construct a training data set; wherein the historical geometric feature information is determined based on feature detection results, and the historical type feature information is determined based on road type detection results.

[0149] In this embodiment, the feature library is used to collect and store historical geometric feature information and historical type feature information, wherein the specific determination process of the historical geometric feature information and the historical type feature information is the same as the determination process of the geometric feature information and the type feature information in the above embodiment, and the relevant description will not be repeated here.

[0150] Step S502: Based on the training data set and the element positioning and recognition results corresponding to each training data, an initial map element positioning and recognition model is trained to obtain the map element positioning and recognition model.

[0151] In this embodiment, the initial map element location and recognition model can adopt a machine learning model, such as a machine learning algorithm based on gradient boosting (eXtreme Gradient Boosting, XGBoost). This model is efficient, flexible and accurate, and can effectively improve the machine learning tasks of map element location and recognition. In some embodiments, in addition to the XGBoost model, other machine learning models can also be used, such as SVM, logistic regression, etc., which are not particularly limited in this embodiment.

[0152] It can be understood that the data set used to train the map feature location and recognition model is an existing data set, and its feature location and recognition results are existing data.

[0153] Compared with the related art, in which historical road image data is directly used to train the map element positioning and recognition model, this embodiment uses the historical geometric feature information and historical type feature information of the historical road image data to jointly train the map element positioning and recognition model, which can effectively improve the training accuracy and efficiency of the model, and the features are intuitive and highly interpretable, and the pressure and time cost of manual review are greatly reduced.

[0154] Figure 6 This is a possible application scenario diagram provided by the embodiment of the present application, such as Figure 6As shown, the collection vehicle collects road videos and transmits the road videos back to the video library; the server obtains video data from the video library and extracts frames of the video data, and then uses the video frame deduplication model (such as the MAE model) to extract features to obtain image features of the video frames and perform the first deduplication; the road types are classified and labeled through the video frames after the first deduplication, and the road types are used as training data to train the road type classification model; then, the video frame deduplication model is returned (the trained road type classification model is used at this time) to perform a second deduplication on the first deduplication video frames to obtain video frames with diverse backgrounds and road types; the map elements in the video frames after the second deduplication are labeled with detection boxes, and the labeled video frames and labels are used to train the feature detection model; according to the elements in the labeled video frames Detection box information, calculate the size of the feature, aspect ratio, pixel distance from the edge of the image, the ratio of the edge pixel distance to the corresponding size of the image, image size and other 14 feature information, and the features extracted by the road type classification model are combined into 24 feature information. At the same time, the feature positioning information is annotated and stored in the feature library for training the xgboost feature positioning and recognition model. During inference, the data is first processed through the trained feature detection model to obtain the corresponding video frame detection results, and the road type classification model is used to extract the road type features. Based on the video frame detection results, 14 feature feature information such as pixel size are calculated, and combined with the road type information into 24 features and input into the trained xgboost model, the positioning and recognition results of the feature can be predicted.

[0155] During the whole process, the geometric features of the elements and the road type features are combined to locate and identify the map elements. First, the feature detection and road type classification model is used to obtain the feature information related to the elements and road scenes. Then, xgboost is used to comprehensively summarize and judge the extracted features, avoiding the black box related to the deep learning algorithm. At the same time, the scene road information is integrated to improve the interpretability and accuracy of positioning. Among them, STN can be introduced into the road type classification model to reduce the errors caused by rotation, translation, etc., and the richness of road type features can be improved through multi-branch prediction.

[0156] It can be seen that the embodiment of the present application comprehensively considers the feature features and road features, and introduces a cascade deduplication module while extracting video frames, and adopts a lightweight detection and classification module in the feature processing stage. It uses xgboost to improve the interpretability of feature discrimination, improve the accuracy of the algorithm, and reduce the cost of manual labeling, thereby achieving the purpose of reducing costs and increasing efficiency. The overall process can be easily migrated to other map feature positioning scenarios. At the same time, due to the lightweight of the overall process, it can be deployed on the cloud and embedded devices.

[0157] Figure 7 is a schematic diagram of the structure of a map element positioning and identification device provided in an embodiment of the present application, such as Figure 7 As shown, the device 700 may include:

[0158] A geometric feature determination module 701, which is configured to determine geometric feature information for describing spatial information of each map element in the road image based on the map element detection result of the road image data and the size corresponding to the road image data;

[0159] A type feature determination module 702, which is configured to determine type feature information for characterizing a road type based on a road type detection result of the road image data;

[0160] A model processing module 703, which is configured to input the geometric feature information and the type feature information into a preset map element positioning and recognition model to obtain a map element positioning and recognition result corresponding to the road image data, wherein the map element positioning and recognition result is used to indicate the position of the map element in the road;

[0161] The map element positioning and recognition model is a model used to recognize the position of map elements on the road.

[0162] In a possible implementation, the map element detection result includes element detection frame coordinates and element categories; the geometric feature determination module 701 includes:

[0163] A first determining unit, configured to determine the size and size ratio of each map element according to the element detection frame coordinates of each map element;

[0164] A second determining unit, configured to determine a pixel distance between each map element and the road image data about an image boundary according to a size of each map element and a size corresponding to the road image data;

[0165] A third determining unit is configured to determine a pixel ratio of an image boundary between each map element and the road image data according to a pixel distance corresponding to each map element and a size corresponding to the road image data;

[0166] The geometric feature acquisition unit is configured to obtain the geometric feature information based on the pixel distance and pixel ratio corresponding to each map element, the element size and size ratio, the element category and the size corresponding to the road image data.

[0167] In a possible implementation manner, the road image data includes a road video; the device further includes a detection module, and the detection module includes:

[0168] an extraction unit, configured to extract at least one video frame of the road video;

[0169] A video frame input unit, configured to input the video frame into a preset feature detection model and / or a preset road type classification model; wherein the feature detection model is a model obtained by training based on historical road videos and used to detect map feature features, and the road type classification model is a model obtained by training based on historical road videos and used to detect road type features;

[0170] A video frame processing unit is configured to process the video frame based on the feature detection model to obtain the map feature detection result, and / or to process the video frame based on the road type classification model to obtain the road type detection result.

[0171] In a possible implementation, the device further includes a first model training module, and the first model training module includes:

[0172] A data acquisition unit, which is configured to acquire historical road video data and extract multiple video frames of the historical video data to obtain an initial video frame set;

[0173] A first deduplication unit, configured to perform deduplication processing on the video frames in the initial video frame set to obtain a first video frame set;

[0174] The first training unit is configured to train an initial road type model based on the first video frame set and its corresponding road type classification annotation information to obtain the road type classification model.

[0175] In a possible implementation, the device further includes a second model training module, and the second model training module includes:

[0176] A second deduplication unit is configured to deduplicate the first video frame set based on the road type classification model, so as to select video frames with different road features from the first video frame set to obtain a second video frame set;

[0177] The second training unit is configured to train an initial feature detection model based on the second video frame set and its corresponding feature annotation information to obtain the feature detection model.

[0178] In a possible implementation, the road type detection model includes an input end for inputting a video frame, a backbone network for processing the video frame, and an output end for outputting a road type detection result; wherein,

[0179] The input end includes a spatial transformation network module for performing jitter elimination on an input video frame, and the output end includes a first fully connected layer for predicting a road scene and a second fully connected layer for identifying a road type.

[0180] In a possible implementation, the road type detection result includes multidimensional features output from the road detection model for characterizing the road type; the type feature determination module is specifically configured to reduce the multidimensional features to low-dimensional features of a preset number of dimensions based on principal component analysis technology; and, based on the low-dimensional features, obtain the type feature information.

[0181] In a possible implementation, the device further includes a third model training module, and the third model training module includes:

[0182] A training set construction unit, which is configured to extract historical geometric feature information and historical type feature information of the historical road image data from a feature library to construct a training data set; wherein the historical geometric feature information is determined based on a feature detection result, and the historical type feature information is determined based on a road type detection result;

[0183] The third training unit is configured to train an initial map element positioning and recognition model based on the training data set and the element positioning and recognition results corresponding to each training data to obtain the map element positioning and recognition model.

[0184] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0185] The present application also provides an electronic device, such as Figure 8 As shown, the electronic device includes: a processor 81, and a memory 82 communicatively connected to the processor 81;

[0186] The memory 82 stores computer-executable instructions;

[0187] The processor 81 executes the computer-executable instructions stored in the memory 82 to implement the method for positioning and identifying map elements, wherein the memory 82 and the processor 81 are connected via a bus 83 .

[0188] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0189] The embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement the map element positioning and identification method as described above.

[0190] Among them, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0191] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0192] The embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the map element positioning and identification method as described above.

[0193] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0194] The embodiment of the present application also provides a chip, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory to execute the map element positioning and identification method.

[0195] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0196] Fig. 9 It is a block diagram of a terminal device shown in an exemplary embodiment of the present application. The terminal device 800 can be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0197] The terminal device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0198] The processing component 802 generally controls the overall operation of the terminal device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0199] The memory 804 is configured to store various types of data to support operations on the terminal device 800. Examples of such data include instructions for any application or method operating on the terminal device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0200] The power supply component 806 provides power to various components of the terminal device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device 800.

[0201] The multimedia component 808 includes a screen that provides an output interface between the terminal device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the terminal device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0202] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the terminal device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0203] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0204] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the terminal device 800. For example, the sensor assembly 814 can detect the open / closed state of the terminal device 800, the relative positioning of the components, such as the display and keypad of the terminal device 800, and the sensor assembly 814 can also detect the position change of the terminal device 800 or a component of the terminal device 800, the presence or absence of contact between the user and the terminal device 800, the orientation or acceleration / deceleration of the terminal device 800, and the temperature change of the terminal device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0205] The communication component 816 is configured to facilitate wired or wireless communication between the terminal device 800 and other devices. The terminal device 800 can access a wireless network based on a communication standard, such as WiFi, 3G, 4G, 5G or other standard communication networks, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0206] In an exemplary embodiment, the terminal device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the method provided in any of the above-mentioned method embodiments of the present application.

[0207] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of a terminal device 800 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0208] An embodiment of the present application also provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by the processor of the terminal device, the terminal device 800 can execute the method provided by any one of the above-mentioned method embodiments of the present application.

[0209] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0210] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0211] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for positioning and identifying map elements, characterized in that: include: Determining geometric feature information for describing spatial information of each map element in the road image based on the map element detection result of the road image data and the size corresponding to the road image data; Determining type feature information for characterizing the road type based on the road type detection result of the road image data; Inputting the geometric feature information and the type feature information into a preset map element positioning and recognition model to obtain a map element positioning and recognition result corresponding to the road image data; The map element positioning and recognition model is a model used to recognize the position of map elements on the road.

2. The method according to claim 1, characterized in that The map element detection result includes the coordinates of the element detection frame and the element category; the map element detection result based on the road image data and the size corresponding to the road image data determines the geometric feature information used to describe the spatial information of each map element in the road image, including: Determine the size and size ratio of each map element according to the element detection frame coordinates of each map element; Determining the pixel distance between each map element and the road image data about the image boundary according to the size of each map element and the size corresponding to the road image data; Determining the pixel ratio of each map element to the road image data with respect to the image boundary according to the pixel distance corresponding to each map element and the size corresponding to the road image data; The geometric feature information is obtained based on the pixel distance and pixel ratio corresponding to each map element, the element size and size ratio, the element category and the size corresponding to the road image data.

3. The method according to claim 1 or 2, characterized in that: The road image data includes a road video; the method for obtaining the map element detection result and / or the road type detection result includes: Extracting at least one video frame of the road video; Inputting the video frame into a preset feature detection model and / or a preset road type classification model; wherein the feature detection model is a model obtained by training based on historical road videos and used to detect map feature features, and the road type classification model is a model obtained by training based on historical road videos and used to detect road type features; The video frame is processed based on the feature detection model to obtain the map feature detection result, and / or the video frame is processed based on the road type classification model to obtain the road type detection result.

4. The method according to claim 3, characterized in that The training method of the road type classification model includes: Collecting historical road video data, and extracting multiple video frames of the historical video data to obtain an initial video frame set; Performing deduplication processing on the video frames in the initial video frame set to obtain a first video frame set; Based on the first video frame set and its corresponding road type classification annotation information, an initial road type model is trained to obtain the road type classification model.

5. The method according to claim 4, characterized in that The training method of the feature detection model includes: Based on the road type classification model, deduplication is performed on the first video frame set to select video frames with different road features from the first video frame set to obtain a second video frame set; Based on the second video frame set and its corresponding element annotation information, an initial element detection model is trained to obtain the element detection model.

6. The method according to claim 3, characterized in that The road type detection model includes an input end for inputting video frames, a backbone network for processing video frames, and an output end for outputting road type detection results; wherein, The input end includes a spatial transformation network module for performing jitter elimination on an input video frame, and the output end includes a first fully connected layer for predicting a road scene and a second fully connected layer for identifying a road type.

7. The method according to any one of claims 1 to 6, characterized in that: The road type detection result includes a multi-dimensional feature output from a road detection model for characterizing the road type; The road type detection result based on the road image data determines the type feature information used to characterize the road type, including: According to the principal component analysis technology, the multi-dimensional features are reduced to low-dimensional features of a preset number of dimensions; as well as, Based on the low-dimensional features, the type feature information is obtained.

8. The method according to any one of claims 1 to 6, characterized in that: The training methods of the map feature positioning and recognition model include: Extracting historical geometric feature information and historical type feature information of the historical road image data from the feature library to construct a training data set; wherein the historical geometric feature information is determined based on the feature detection result, and the historical type feature information is determined based on the road type detection result; Based on the training data set and the element positioning and recognition results corresponding to each training data, an initial map element positioning and recognition model is trained to obtain the map element positioning and recognition model.

9. A map element positioning and identification device, characterized in that: include: A geometric feature determination module, which is configured to determine geometric feature information for describing spatial information of each map element in the road image based on a map element detection result of the road image data and a size corresponding to the road image data; A type feature determination module, configured to determine type feature information for characterizing a road type based on a road type detection result of the road image data; A model processing module, which is configured to input the geometric feature information and the type feature information into a preset map element positioning and recognition model to obtain a map element positioning and recognition result corresponding to the road image data; The map element positioning and recognition model is a model used to recognize the position of map elements on the road.

10. An electronic device / computer-readable storage medium / computer program product, characterized in that: include: Memory and processor; The memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the map element positioning and recognition method according to any one of claims 1 to 8; and / or, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method for positioning and identifying map elements according to any one of claims 1 to 8; and / or The computer program product comprises a computer program, which, when executed by a processor, implements the method for positioning and identifying map elements as described in any one of claims 1 to 8.

Images