A map updating method and device and related products

Abstract

This application discloses a map updating method, apparatus, and related products. The method involves acquiring a road image to be mined, a historical image dataset, and an initial map; obtaining a set of historical image elements corresponding to the road image to be mined from the historical image dataset, based on the road image to be mined and the historical image dataset; performing mining processing on the road image to be mined using an element detection model to obtain a set of road image elements corresponding to the road image to be mined; matching the historical image element set and the road image element set to obtain target elements; and updating the initial map based on the target elements to obtain the target map. Thus, this application can identify historical images identical to the road image to be mined and also achieve matching between the historical image element set in the historical image and the road element image set in the road image to be mined, thereby enabling better reverse mining of road elements and improving the accuracy of map updates.

Description

Technical Field

[0001] This application relates to the field of map updating technology, and in particular to a map updating method, apparatus and related products. Background Technology

[0002] The various map elements present on the map are effective physical point information on the road, which can provide important information for navigation broadcasts to ensure the efficiency and convenience of travel. However, effective physical point information may disappear with changes in traffic flow. If navigation broadcasts are then based on the original map, the navigation data may be incorrect.

[0003] Based on this, two reverse mining methods are proposed in related technologies to uncover map features that have disappeared from the map. Method 1: Directly construct an image classification model to judge the currently acquired image to determine whether there are missing features. However, due to poor image quality and imbalanced model samples, the accuracy of reverse mining is prone to low.

[0004] Method 2 for reverse mining: This method matches local information from the currently acquired image with local information from historical images to determine if any missing features exist in the currently acquired image. However, due to the limited matching range of local information, it is prone to low accuracy in reverse mining. Furthermore, updating the map based on the identified potentially missing features can easily lead to poor map update accuracy.

[0005] Therefore, improving the accuracy of map updates has become a pressing technical problem that needs to be solved in this field. Summary of the Invention

[0006] This application provides a map updating method, apparatus, and related products, aiming to improve the accuracy of map updates.

[0007] The first aspect of this application provides a map updating method, including:

[0008] Obtain images of the road to be excavated, a dataset of historical images, and an initial map;

[0009] Based on the road image to be excavated and the historical image dataset, a historical image feature set corresponding to the road image to be excavated in the historical image dataset is obtained, wherein the historical image feature set represents multiple navigation information in the initial map;

[0010] Based on the feature detection model, the road image to be mined is processed to obtain the road image feature set corresponding to the road image to be mined. The feature detection model is used to obtain image features in the image, and the road image feature set represents multiple navigation information in the initial map.

[0011] The historical image feature set and the road image feature set are matched to obtain target features, wherein the target features include image features that have disappeared from the road image feature set.

[0012] The initial map is updated based on the target features to obtain the target map.

[0013] A second aspect of this application provides a map updating apparatus, comprising:

[0014] The image data acquisition unit is used to acquire images of the road to be excavated, historical image datasets, and an initial map;

[0015] The historical element set acquisition unit is used to obtain a historical image element set of the historical image corresponding to the road image to be excavated in the historical image dataset based on the road image to be excavated and the historical image dataset, wherein the historical image element set represents multiple navigation information in the initial map;

[0016] The road element set acquisition unit is used to perform mining processing on the road image to be mined based on the element detection model to obtain the road image element set corresponding to the road image to be mined, wherein the element detection model is used to obtain image elements in the image, and the road image element set represents multiple navigation information in the initial map.

[0017] The target element acquisition unit is used to perform matching processing on the historical image element set and the road image element set to obtain target elements, wherein the target elements include image elements that have disappeared from the road image element set;

[0018] The target map acquisition unit is used to update the initial map based on the target features to obtain the target map.

[0019] A third aspect of this application provides a computer device, the device comprising a processor and a memory:

[0020] The memory is used to store computer programs and to transfer the computer programs to the processor;

[0021] The processor is configured to execute the steps of the map update method provided in the first aspect according to the instructions in the computer program.

[0022] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program that, when executed by a computer device, implements the steps of the map update method provided in the first aspect.

[0023] The fifth aspect of this application provides a computer program product, including a computer program that, when executed by a computer device, implements the steps of the map updating method provided in the first aspect.

[0024] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0025] The technical solution of this application first acquires the road image to be excavated, a historical image dataset, and an initial map. Then, based on the road image to be excavated and the historical image dataset, it obtains a historical image feature set corresponding to the road image to be excavated from the historical image dataset. Additionally, it performs excavation processing on the road image to be excavated using a feature detection model to obtain a road image feature set corresponding to the road image to be excavated. Finally, it performs matching processing on the historical image feature set and the road image feature set to obtain target features, and updates the initial map based on the target features to obtain the target map. It should be noted that the historical image feature set represents multiple navigation information in the initial map, the feature detection model is used to obtain image features in the image, the road image feature set represents multiple navigation information in the initial map, and the target features include image features that have disappeared from the road image feature set.

[0026] As can be seen, this application first identifies the historical image corresponding to the road image to be excavated and the set of historical image elements within that historical image. Then, it mines the road image element set corresponding to the road image to be excavated based on an element detection model. Subsequently, the historical image element set and the road image element set are matched to obtain the target element, thereby updating the initial map based on this target element. Thus, this application not only identifies the historical image identical to the road image to be excavated but also accurately matches the set of historical image elements within the historical image with the set of road element images in the road image to be excavated. This allows for better reverse mining of road elements, thereby improving the accuracy of map updates. Attached Figure Description

[0027] Figure 1 A scenario architecture diagram of a map update method provided in this application embodiment;

[0028] Figure 2 A flowchart illustrating a map updating method provided in this application embodiment;

[0029] Figure 3A flowchart illustrating the process of obtaining a historical image feature set in a map updating method provided in this application embodiment;

[0030] Figure 4 A schematic diagram of a historical image set in a map update method provided in an embodiment of this application;

[0031] Figure 5 A schematic diagram illustrating the determination of historical images in a map updating method provided in an embodiment of this application;

[0032] Figure 6 This application provides an architecture diagram of a feature detection model in a map update method.

[0033] Figure 7 A flowchart illustrating the process of obtaining a road image feature set in a map updating method provided in this application embodiment;

[0034] Figure 8 A schematic diagram of a region candidate box set in a map update method provided in an embodiment of this application;

[0035] Figure 9 A schematic diagram illustrating the acquisition of a road image feature set in a map updating method provided in an embodiment of this application;

[0036] Figure 10 This is a schematic diagram illustrating the application of a feature detection model in a map updating method provided in this application embodiment;

[0037] Figure 11 A schematic diagram of a feature set in a map updating method provided in an embodiment of this application;

[0038] Figure 12 A schematic diagram illustrating the acquisition of target features in a map updating method provided in an embodiment of this application;

[0039] Figure 13 This application provides a complete flowchart of a map updating method according to an embodiment of the present application.

[0040] Figure 14 This is a schematic diagram of the structure of a map updating device provided in an embodiment of this application;

[0041] Figure 15 This is a schematic diagram of the server structure in an embodiment of this application;

[0042] Figure 16 This is a schematic diagram of the structure of a terminal device in an embodiment of this application. Detailed Implementation

[0043] The embodiments of this application will now be described with reference to the accompanying drawings.

[0044] The map contains various map elements, which are effective physical point information on the road, such as speed limit signs. These elements provide important information for navigation broadcasts to ensure the efficiency and convenience of travel. However, effective physical point information may disappear with changes in traffic flow. If navigation broadcasts are then based on the original map, errors in the navigation data may occur.

[0045] Based on this, two reverse mining methods are proposed in related technologies to uncover map features that have disappeared from the map, thereby updating the original map and avoiding navigation errors. Reverse mining method one: directly constructing an image classification model to judge the currently acquired image to determine whether there are missing features (that is, extracting and classifying features from the image using an image classification model to obtain the final image recognition result, which indicates whether there are missing features in the currently acquired image). However, due to poor image quality (e.g., blurry images) and imbalanced model samples, the accuracy of reverse mining is prone to low.

[0046] Method Two for reverse mapping involves matching local information from the currently acquired image with local information from historical images to determine if any missing features exist. However, due to the limited matching range of local information, reverse mapping accuracy is prone to low accuracy. This local information can be understood as image features containing valid physical point information. Matching only local information may result in matching local information from similar historical images with local information from the currently acquired image, leading to incorrect matches. Therefore, updating the map based on these potentially missing features often results in poor map update accuracy. Improving the accuracy of map updates has thus become a pressing technical problem in this field.

[0047] In view of the above problems, this application provides a map updating method, apparatus, and related products, aiming to improve the accuracy of map updates. The technical solution provided in this application first acquires an image of the road to be excavated, a historical image dataset, and an initial map. Then, based on the image of the road to be excavated and the historical image dataset, a historical image element set corresponding to the image of the road to be excavated is obtained from the historical image dataset. Additionally, based on an element detection model, the image of the road to be excavated is processed to obtain a road image element set corresponding to the image of the road to be excavated. Finally, the historical image element set and the road image element set are matched to obtain target elements, and the initial map is updated based on the target elements to obtain the target map. The historical image element set represents multiple navigation information in the initial map, the element detection model is used to obtain image elements in the image, the road image element set represents multiple navigation information in the initial map, and the target elements include image elements that have disappeared from the road image element set.

[0048] As can be seen, this application first identifies historical images corresponding to the road image to be excavated. Based on the entire historical image, the set of historical image elements within that image is determined. Furthermore, a road image element set corresponding to the road image to be excavated can be obtained in real-time using an element detection model. Subsequently, the historical image element set and the road image element set are matched to obtain the target element, thereby updating the initial map based on this target element. Thus, this application not only identifies historical images identical to the road image to be excavated but also accurately matches the set of historical image elements within the historical image with the set of road element images in the road image to be excavated. This allows for better reverse mining of road elements, thereby improving the accuracy of map updates.

[0049] The execution subject of the map update method provided in this application embodiment can be a terminal device. For example, the terminal device acquires the image of the road to be excavated, the historical image dataset, and the initial map. As an example, the terminal device may include, but is not limited to, mobile phones, desktop computers, tablet computers, laptops, PDAs, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft. The execution subject of the map update method provided in this application embodiment can also be a server, that is, the image of the road to be excavated, the historical image dataset, and the initial map can be acquired on the server. In addition, the map update method provided in this application embodiment can also be executed collaboratively by the terminal device and the server. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited here. Therefore, the implementation subject of the technical solution of this application is not limited in this application embodiment.

[0050] Figure 1An exemplary scenario architecture diagram of a map update method is shown. The diagram includes a server and various types of terminal devices. For example, in this application, the server first obtains a set of historical image features corresponding to the road image to be updated from the historical image dataset obtained by the terminal device, based on the road image to be updated and the historical image dataset. Then, the image to be updated is processed by a feature detection model to obtain a set of road image features. Finally, the server updates the initial map based on the target features determined by the historical image feature set and the road image feature set to obtain the target map. This can significantly improve the accuracy of map updates. Figure 1 The server shown can be a standalone physical server, a server cluster consisting of multiple physical servers, or a distributed system. Additionally, the server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0051] See Figure 2 This figure is a flowchart of a map updating method provided in an embodiment of this application. Figure 2 The map update method shown includes the following steps:

[0052] S201: Obtain the road image to be mined, historical image dataset, and initial map.

[0053] In this step, the road image to be excavated can be understood as an image captured by a vehicle-mounted camera, which includes a vehicle-mounted camera (such as a dashcam). In other words, a sequence of road images of the current road can be obtained by capturing images of the current road using a vehicle-mounted camera. Then, the road image sequence is preprocessed to obtain the road image to be excavated. The preprocessing operation includes an image extraction operation, which extracts the road image with the highest clarity from the road image sequence and uses the road image with the highest clarity as the road image to be excavated.

[0054] The historical image dataset includes a set of historical images previously captured by the vehicle-mounted camera and obtained through preprocessing, and a set of image features corresponding to the historical images in this dataset. This historical image dataset is stored in a database to facilitate rapid extraction and application of historical image data in subsequent processes. The database may include Oracle or MySQL; no specific database is specified in this application. The initial map can be understood as a map displayed on the object and capable of providing navigation services to the object.

[0055] S202: Based on the road image to be excavated and the historical image dataset, obtain the historical image element set of the historical image corresponding to the road image to be excavated in the historical image dataset.

[0056] In this step, based on the road image to be excavated, the historical image corresponding to the road image in the historical image dataset and the historical image feature set of that historical image can be determined. This historical image feature set represents multiple navigation information in the initial map. Thus, in this application, based on the determination of the complete historical image, the historical image feature set of the historical image is further determined, which enables more accurate matching of the feature set in subsequent processes.

[0057] The historical image feature set includes road sign features and speed limit sign features. It should be noted that the historical image feature set in this application is merely an example, and the historical image feature set can be determined in practical applications. It is understood that a historical image feature in the historical image feature set can correspond to a navigation information point in the initial map. For example, if the historical image feature in the historical image is a speed limit sign feature (speed limit 40 km / h), then this historical image feature can be used as navigation information representing a speed limit of 40 km / h in the initial map.

[0058] The following describes the specific process of determining the historical image element set in this application, such as... Figure 3 As shown, Figure 3 This is a flowchart illustrating the process of obtaining a historical image feature set in a map updating method provided in this application embodiment. Figure 3 This includes steps S2021-S2023, which are detailed below:

[0059] S2021: The road image to be excavated and the historical image set are parsed and processed to obtain the road geographic information corresponding to the road image to be excavated and the historical geographic information set corresponding to the historical image set.

[0060] First, it should be noted that the historical image dataset includes a set of historical images and multiple sets of historical image features corresponding to those historical images. Each historical image corresponds to one set of historical image features. The historical image set includes images acquired before the current time period, and the historical image feature set includes the set of image features corresponding to images acquired before the current time period. For example... Figure 4 As shown, Figure 4 This is a schematic diagram of a historical image set in a map update method provided in an embodiment of this application. Figure 4 The historical image set includes images (a), (b), (c), and (d), which are images corresponding to different scenes. It should be noted that... Figure 4 The historical image set shown is only a partial example and is not specifically limited here.

[0061] At this point, the image of the road to be excavated can be analyzed to obtain the corresponding road geographic information, and the historical image set can be analyzed to obtain the corresponding historical geographic information set. Each historical image in the historical image set corresponds to a historical geographic information set in the historical geographic information set. The road geographic information can be understood as the GPS corresponding to the image of the road to be excavated, and the historical geographic information can be understood as the GPS corresponding to the historical image. This facilitates subsequent filtering of historical images corresponding to the image of the road to be excavated based on geographic information.

[0062] S2022: Based on the road geographic information, the historical geographic information set is filtered to obtain multiple historical geographic information sets that are identical to the road geographic information.

[0063] In this step, geographic information can be filtered from road geographic information and historical geographic information sets to obtain multiple historical geographic information sets that are identical to road geographic information. This initial screening using GPS distance can improve the accuracy of the final determination of historical images to a certain extent.

[0064] S2023: Based on the multiple historical geographic information, obtain the set of historical image elements in the historical image dataset that corresponds to the road image to be excavated.

[0065] Specifically, firstly, based on multiple historical geographic information sets, the historical images corresponding to each of these sets of historical geographic information are identified. The geographical locations when these historical images were taken are similar to the geographical locations when the images of the road to be excavated were taken. Then, trajectory points can be extracted from the images to be excavated to obtain a sequence of road trajectory points corresponding to the images, and trajectory points can be extracted from the historical images corresponding to each of the multiple sets of historical geographic information to obtain a sequence of historical trajectory points corresponding to each of the multiple historical images.

[0066] It should be noted that in this application, a ResNet17 network can be used to extract features from the image to be mined, thereby obtaining the trajectory point features θ of the image to be mined. q,k Here, q represents the q-th image, and k represents the k-th trajectory point in the q-th image. This allows us to obtain the road trajectory point sequence corresponding to the image to be mined. Furthermore, we can use a ResNet17 network to extract features from multiple historical images to obtain the trajectory point features θ corresponding to each historical image. q,kAt this point, we can obtain the historical trajectory point sequences corresponding to multiple historical images. This allows us to further determine the historical images and their feature sets corresponding to the road image to be excavated based on the trajectory point sequences in subsequent processes, thus improving the accuracy of historical image identification.

[0067] Furthermore, the process of "obtaining the set of historical image elements corresponding to the road image to be mined from the historical image dataset based on the road trajectory point sequence and multiple historical trajectory point sequences" in this application can be specifically embodied as follows:

[0068] First, the road trajectory point sequence and multiple historical trajectory point sequences can be aligned separately to obtain multiple trajectory point similarity sets. One of these sets is derived from the road trajectory point sequence and a historical trajectory point sequence. To obtain a single trajectory point similarity result, in this application, the road trajectory point sequence <θ 1,1 ,θ 1,2 ,θ 1,n A trajectory point feature in > and a historical trajectory point sequence <θ 2,1 ,θ 2,2 ,θ 2,n The similarity of a trajectory point in the set of trajectory point similarities is calculated by performing a similarity calculation (which can be an inner product calculation) to obtain the similarity score of a trajectory point in the set of trajectory point similarities.

[0069] Next, multiple trajectory point similarity sets can be combined to obtain multiple similarity results. One similarity result is obtained from a single trajectory point similarity set, representing the direct similarity between the road image to be excavated and a historical image. Subsequently, multiple similarity results are processed to obtain a trajectory point similarity matrix, and this matrix is ​​solved using dynamic programming to obtain the optimal solution. Based on this optimal solution, the historical image corresponding to the road image to be excavated can be obtained, along with its corresponding historical image feature set. Thus, in this application, based on the identified historical images similar to the road image to be excavated, dynamic programming can be used to accurately determine the corresponding historical image and its corresponding historical image feature set, thereby improving the accuracy of subsequent map updates to a certain extent.

[0070] Because dynamic programming possesses the properties of optimal substructure (the solutions to the subproblems contained in the problem are also optimal) and subproblem overlap (when solving a problem from top to bottom using a recursive algorithm, the subproblems generated each time are not always new problems, and some subproblems are repeatedly calculated), this application can find the matrix optimal solution (i.e., the alignment result of the image trajectory points with the highest similarity) in the trajectory point similarity matrix through dynamic programming. In one feasible implementation, this application can obtain the matrix optimal solution in the trajectory point similarity matrix by solving formula (1), which is specifically embodied as follows:

[0071]

[0072] Where dtw[i][j] represents the similarity matrix M of the trajectory points. 1,1 Start from the top left corner and walk to M. i,j The sum of similarity scores of the traversed paths (i.e., the optimal solution of the matrix); i represents the row of the trajectory point similarity matrix, which is determined by the number of road images to be mined; j represents the column of the trajectory point similarity matrix, which is determined by the number of historical images; dis[i][j]

[0073] Characterization of M i,j The numerical value at the location (i.e., the direct similarity between the road image to be excavated and the j-th historical image); dtw[i-1][j] represents the similarity from the trajectory point similarity matrix M. 1,1 Start by walking to M i-1,j The sum of similarity along the traversed paths; dtw[i-1][j-1] represents the similarity matrix M from the trajectory points. 1,1 Start by walking to M i-1,j-1 The sum of similarity along the traversed paths; dtw[i][j-1] represents the similarity matrix M from the trajectory points. 1,1 Start by walking to M i,j-1 The sum of the similarity of the traversed paths. Thus, in this application, a dynamic programming approach can be used to accurately determine the historical image corresponding to the road image to be mined, avoiding the reverse mining errors caused by reverse mining method one or reverse mining method two in related technologies.

[0074] like Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the determination of historical images in a map update method provided in an embodiment of this application. Figure 5 Images (a), (b), (c), and (d) are historical images in the historical image dataset that correspond to multiple historical geographic information, respectively. Image (e) is the road image to be excavated. Image (c) is the historical image that corresponds to the road image to be excavated among multiple historical images determined based on the above historical image determination process.

[0075] S203: Based on the feature detection model, perform mining processing on the road image to be mined to obtain the road image feature set corresponding to the road image to be mined.

[0076] In this step, an element detection model is used to obtain image elements in the image. This element detection model is a pre-trained model, and its model structure can be a convolutional neural network. The road image element set represents multiple navigation information in the initial map. The road image element set includes road sign elements and speed limit sign elements. It should be noted that the road image element set is only an example in this application; the historical image element set can be determined in practical applications. It is understood that a road image element in the road image element set can correspond to a navigation information in the initial map. For example, if the road image element in the road image to be excavated is a road sign element (ahead is a highway entrance), then this road image element can be represented in the initial map as navigation information indicating that there is a highway entrance ahead.

[0077] It should be noted that the feature detection model in this application includes a feature extraction layer, a region generation layer, and a feature mining layer, wherein the feature mining layer includes a pooling layer and a classification layer. For example... Figure 6 As shown, Figure 6 This is an architecture diagram of a feature detection model in a map update method provided in an embodiment of this application. Figure 6 First, the feature extraction layer in the feature detection model obtains the road feature map corresponding to the road image to be mined. Then, the region generation layer in the feature detection model obtains the candidate bounding boxes for multiple road feature features in the road feature map. Finally, the pooling and classification layers in the feature mining layer of the feature detection model obtain the road image feature set corresponding to the road image to be mined. Thus, in this application, image features can be effectively mined and obtained through the feature detection model.

[0078] like Figure 7 As shown, Figure 7 This is a flowchart illustrating the process of obtaining a road image feature set in a map updating method provided in this application embodiment. Figure 7 This includes steps S2031-S2033, which are detailed below:

[0079] S2031: Based on the feature extraction layer in the feature detection model, feature extraction is performed on the road image to be excavated to obtain the road feature map corresponding to the road image to be excavated.

[0080] In this step, the feature extraction layer includes a convolutional layer, a normalization layer, and an activation layer. The convolutional layer extracts basic features such as edge textures from the image to obtain a road feature set. The normalization layer normalizes the road feature set extracted by the convolutional layer according to a normal distribution and filters out noisy features, making the model's training and application converge faster. The activation layer performs a non-linear mapping on the features extracted by the convolutional layer to obtain multiple road element features, thereby enhancing the model's generalization ability. Thus, in this application, by using the convolutional layer, normalization layer, and activation layer in the feature extraction layer to extract features from the road image to be excavated, a road feature map corresponding to the road image to be excavated can be obtained. That is, by using multiple road element features, a road feature map corresponding to the road image to be excavated can be generated.

[0081] S2032: Based on the region generation layer in the feature detection model, region selection is performed on the road feature map to obtain a set of region candidate boxes corresponding to multiple road feature features in the road feature map.

[0082] At this point, based on the region generation layer in the feature detection model, regions can be selected from multiple road feature features in the road feature map to obtain a set of candidate bounding boxes corresponding to each of the multiple road feature features in the road feature map. Taking the set of candidate bounding boxes corresponding to a single road feature feature in the road feature map as an example, in this application, nine candidate boxes can be selected as the set of candidate bounding boxes corresponding to the road feature feature, with the road feature feature as the center point. This facilitates the accurate determination of the feature region bounding box corresponding to the road feature feature.

[0083] like Figure 8 As shown, Figure 8 This is a schematic diagram of a region candidate box set in a map update method provided in an embodiment of this application. Figure 8 The region candidate box set shown is a region candidate box set corresponding to a road feature feature. The region candidate box set includes three candidate boxes with a ratio of 1:1 and sizes of 1, 2, and 3 respectively; three candidate boxes with a ratio of 2:1 and sizes of 1, 2, and 3 respectively; and three candidate boxes with a ratio of 1:2 and sizes of 1, 2, and 3 respectively.

[0084] S2033: Based on the feature mining layer in the feature detection model, the road feature map and the region candidate box sets corresponding to the multiple road feature features are mined to obtain the road image feature set corresponding to the road image to be mined.

[0085] In this step, the pooling layer in the feature mining layer of the feature detection model can be used to map the candidate bounding boxes of multiple road feature features according to the road feature map to obtain the feature region boxes corresponding to multiple road feature features. In other words, the pooling layer can select the candidate bounding box of the most suitable road feature feature and use the candidate bounding box of the most suitable road feature feature as the feature region box.

[0086] Following this, based on the classification layer in the feature mining layer of the feature detection model, the road image to be excavated can be cropped according to the feature region bounding boxes corresponding to multiple road feature features, to obtain road feature images corresponding to multiple road feature features in the road image to be excavated. The multiple road feature features and their corresponding road feature images are then used as the road image feature set corresponding to the road image to be excavated. Thus, in this application, under the action of the feature detection model, the road image feature set of the road image to be excavated is accurately determined to support subsequent processes in achieving focused spatial domain matching.

[0087] like Figure 9 As shown, Figure 9 This is a schematic diagram illustrating the acquisition of a road image feature set in a map updating method provided in an embodiment of this application. Figure 9 Image (a) is the road image to be excavated, and image (b) is the image of the road image to be excavated marked with feature region boxes after the feature detection model. The solid-line rectangle in image (b) represents the feature region box, and the image in the feature region box represents the road feature image.

[0088] like Figure 10 As shown, Figure 10 This is a schematic diagram illustrating the application of a feature detection model in a map update method provided in this application embodiment. Figure 10First, the road image to be excavated can be input into the feature extraction layer of the feature detection model. This feature extraction layer can output the road feature map corresponding to the road image to be excavated. Then, the road feature map is input into the region generation network and the pooling layer of the feature mining layer in the feature detection model. The region generation network can output road element images corresponding to multiple road element features in the road image to be excavated, and these road element images are then input into the pooling layer of the feature mining layer. At this point, the pooling layer in the feature mining layer can obtain element region boxes corresponding to multiple road element features based on the road feature map and the set of region candidate boxes corresponding to the multiple road element features. Finally, the classification layer in the feature mining layer can obtain the road image element set corresponding to the road image to be excavated based on the element region boxes corresponding to the multiple road element features. Thus, in this application, the road image element set corresponding to the road image to be excavated can be effectively extracted based on the feature detection model.

[0089] It should also be noted that the feature detection model in this application is trained in the following way: First, a road sample image and its corresponding image feature label set can be obtained, where the line feature labels in the image feature label set represent the image features that should exist in the road sample image. Then, the road sample image can be input into the detection model to be trained, so that the model can perform prediction processing on the road sample image to obtain the corresponding image prediction feature set. Finally, based on the image feature label set and the image prediction feature set, the parameters of the detection model to be trained are adjusted until the adjusted model meets the training cutoff condition. The adjustment is then complete, and the feature detection model is obtained, with one image feature label and one image prediction feature serving as a pair of training data. It should be noted that the training process of the detection model to be trained in this application can refer to the application process of the feature detection model, and will not be elaborated further here. Thus, in this application, a supervised model training method can achieve high-precision prediction of images, thereby improving the detection accuracy of subsequent model applications.

[0090] S204: Perform matching processing on the historical image feature set and the road image feature set to obtain the target feature.

[0091] In this step, the target elements include image elements that have disappeared from the road image element set but existed in the historical image element set. The navigation information for these target elements in the initial map needs to be deleted to improve the accuracy of road navigation. For example... Figure 11 As shown, Figure 11 This is a schematic diagram of a feature set in a map updating method provided in an embodiment of this application. Figure 11The rectangle (a) represents the road image feature set, which includes multiple road feature features and road feature images corresponding to the multiple road feature features; the rectangle (b) represents the historical image feature set, which includes multiple historical feature features and historical feature images corresponding to the multiple historical feature features.

[0092] Specifically, in this application, multiple historical feature elements and multiple road feature elements are first processed together to obtain a feature similarity matrix. Each point in the feature similarity matrix represents the feature similarity between a historical feature element and a road feature element. It should be noted that the multiple historical feature elements include historical image features and historical location features, and the multiple road feature elements include road image features and road location features. Understandably, each feature element has a feature dimension of 1×34, with the first 32 dimensions representing image features and the last 2 dimensions representing location features (i.e., location encoding in the horizontal and vertical axes).

[0093] Furthermore, in this application, historical image features and historical location features can be used to create a feature set to obtain a historical feature matrix, which has a dimension of m×34. Similarly, road image features and road location features can be used to create a feature set to obtain a road feature matrix, which has a dimension of n×34. Finally, the historical feature matrix and the road feature matrix can be multiplied to obtain a feature similarity matrix. Thus, the feature similarity matrix in this application also includes location encoding for more accurate similarity calculation.

[0094] In one feasible implementation, the feature similarity matrix can be obtained by formulas (2)-(4), which are specifically embodied as follows:

[0095]

[0096] Among them, F A The features represent multiple road elements in the road image to be excavated, where x represents one of the road element features in the road image to be excavated, and n represents the upper limit of the values ​​of the multiple road element features. Let z represent multiple historical feature features of the y-th historical image corresponding to the road image to be excavated, z represent one road feature feature in the y-th historical image, m represent the upper limit of the values ​​of multiple historical feature features; T represents matrix transpose; R k This represents the feature similarity matrix.

[0097] Next, in this application, the feature similarity matrix can be filtered based on the feature similarity threshold to obtain the target feature similarity in the feature similarity matrix that is less than or equal to the feature similarity threshold. The feature similarity threshold can be 0.6, that is, if the feature similarity in the feature similarity matrix is ​​greater than 0.6, the element corresponding to the feature similarity can be considered to exist, otherwise it will disappear. This feature similarity threshold is only an example in this application, and the setting of the feature similarity threshold can also be determined in practical applications.

[0098] Furthermore, in this application, target feature features corresponding to the target feature similarity among multiple historical feature features can be determined based on target feature similarity. Then, based on the target feature features, target feature images corresponding to the target feature features in the historical feature images corresponding to the multiple historical feature features are determined. The target feature features and their corresponding target feature images are then used as target features. Thus, this application can accurately determine the features to be deleted by focusing on the current image spatial domain and the historical image spatial domain, thereby improving the accuracy of map updates.

[0099] like Figure 12 As shown, Figure 12 This is a schematic diagram illustrating the acquisition of target features in a map updating method provided in an embodiment of this application. Figure 12 The upper half of the image and the bounding box represent the road image to be excavated and the road image feature set corresponding to the road image to be excavated. The lower half of the image and the bounding box represent the historical image corresponding to the road image to be excavated and the historical image feature set corresponding to the historical image. At this time, the target feature can be obtained by matching the road image feature set and the historical image feature set.

[0100] S205: Update the initial map based on the target features to obtain the target map.

[0101] In this step, based on the target element, the navigation information represented by the target element in the initial map can be determined, and this navigation information can be deleted to obtain the target map (if there is a historical image corresponding to the road image to be excavated), thus improving the accuracy of map updates. It should also be noted that if multiple historical images corresponding to the road image to be excavated exist, a voting method can be used to determine whether the target element needs to be deleted. For example, if the disappearance rate of the target element exceeds 40% of the total, it can be considered that the target point has disappeared; otherwise, it is considered to have disappeared. Thus, by focusing on spatial domain analytical matching, this application can effectively improve the recall rate of disappeared elements, thereby improving the accuracy of map updates.

[0102] like Figure 13 As shown, Figure 13This is a flowchart illustrating the entire process of updating a map in a map updating method provided in an embodiment of this application. Figure 13 The process first acquires the road image to be mined, a historical image dataset, and an initial map. Then, it uses an element detection model to mine the road image to obtain a set of road image elements corresponding to it. It also filters the historical image dataset to obtain a set of historical image elements corresponding to the road image to be mined. Next, it matches the road image element set and the historical image element set to obtain the target element. Finally, it updates the initial map based on the target element to obtain the target map. This method effectively achieves reverse mining of road elements, thereby improving the accuracy of map updates.

[0103] In summary, this application embodiment first identifies historical images corresponding to the road image to be excavated. Based on the entire historical image, it identifies the set of historical image elements within that image. Furthermore, it uses an element detection model to mine and obtain the set of road image elements corresponding to the road image to be excavated in real time. Subsequently, it matches the historical image element set and the road image element set to obtain the target element, thereby updating the initial map based on this target element. Thus, this application not only identifies historical images identical to the road image to be excavated but also accurately matches the set of historical image elements in the historical image with the set of road element images in the road image to be excavated. This allows for better reverse mining of road elements, avoiding the low accuracy issues of map updates caused by reverse mining method one or reverse mining method two in related technologies, thereby improving the accuracy of map updates. In addition, this application's technical solution can be used for roads with complex scenes to accurately find missing elements in the road image to be excavated, thereby updating the map and restoring the real-time navigation status of the current road.

[0104] Based on the map updating method provided in the preceding embodiments, this application also provides a map updating device. The map updating device provided in this application will be described in detail below.

[0105] See Figure 14 This figure is a schematic diagram of the structure of a map updating device provided in an embodiment of this application. Figure 14 As shown, the map updating device specifically includes:

[0106] Image data acquisition unit 1401 is used to acquire images of the road to be excavated, historical image datasets, and an initial map;

[0107] The historical element set acquisition unit 1402 is used to obtain a historical image element set of the historical image corresponding to the road image to be excavated in the historical image dataset based on the road image to be excavated and the historical image dataset, wherein the historical image element set represents multiple navigation information in the initial map;

[0108] The road element set acquisition unit 1403 is used to perform mining processing on the road image to be mined based on the element detection model to obtain the road image element set corresponding to the road image to be mined, wherein the element detection model is used to obtain image elements in the image, and the road image element set represents multiple navigation information in the initial map.

[0109] The target element acquisition unit 1404 is used to perform matching processing on the historical image element set and the road image element set to obtain target elements, wherein the target elements include image elements that have disappeared from the road image element set;

[0110] The target map acquisition unit 1405 is used to update the initial map based on the target features to obtain the target map.

[0111] In one feasible implementation, the target element acquisition unit 1404 includes:

[0112] The feature set unit is used to perform set processing on the multiple historical feature features and the multiple road feature features to obtain a feature similarity matrix, wherein each matrix point in the feature similarity matrix represents the feature similarity between a historical feature feature and a road feature feature.

[0113] A similarity matrix filtering unit is used to filter the feature similarity matrix based on a feature similarity threshold to obtain the target feature similarity in the feature similarity matrix that is less than or equal to the feature similarity threshold.

[0114] The feature acquisition unit is used to obtain the target feature that is similar to the target feature from the plurality of historical feature features based on the target feature similarity.

[0115] The target element determination unit is used to take the target element features and the target element images corresponding to the target element features as target elements.

[0116] In one feasible implementation, the feature set unit is specifically used for:

[0117] The historical image features and the historical location features are used to form a feature set to obtain a historical feature matrix;

[0118] The road image features and the road location features are used to create a feature set to obtain a road feature matrix;

[0119] The historical feature matrix and the road feature matrix are multiplied together to obtain the feature similarity matrix.

[0120] In one feasible implementation, the historical element set acquisition unit 1402 includes:

[0121] The geographic information acquisition unit is used to parse and process the road image to be excavated and the historical image set to obtain the road geographic information corresponding to the road image to be excavated and the historical geographic information set corresponding to the historical image set, wherein one historical image corresponds to one historical geographic information.

[0122] A geographic information filtering unit is used to filter the historical geographic information set based on the road geographic information to obtain multiple historical geographic information sets that are the same as the road geographic information.

[0123] The historical element set determination unit is used to obtain the historical image element set of the historical image corresponding to the road image to be excavated in the historical image dataset based on the multiple historical geographic information.

[0124] In one feasible implementation, the historical element set determination unit includes:

[0125] A historical image determination unit is used to determine, based on the plurality of historical geographic information, the historical images in the historical image set that correspond to the plurality of historical geographic information respectively;

[0126] The trajectory point sequence acquisition unit is used to extract trajectory points from the image to be excavated and the historical images corresponding to the multiple historical geographic information, and to obtain the road trajectory point sequence corresponding to the image to be excavated and the historical trajectory point sequence corresponding to the multiple historical images.

[0127] The trajectory point sequence processing unit is used to obtain a set of historical image elements in the historical image dataset that corresponds to the road image to be excavated, based on the road trajectory point sequence and multiple historical trajectory point sequences.

[0128] In one feasible implementation, the trajectory point sequence processing unit is specifically used for:

[0129] Alignment processing is performed on the road trajectory point sequence and multiple historical trajectory point sequences to obtain multiple trajectory point similarity sets, one of which is obtained based on the road trajectory point sequence and a historical trajectory point sequence;

[0130] The multiple trajectory point similarity sets are processed to construct a trajectory point similarity matrix;

[0131] Based on the optimal solution of the trajectory point similarity matrix, obtain the historical image corresponding to the road image to be excavated from the plurality of historical images;

[0132] Based on the historical images, obtain the historical image element set of the historical images.

[0133] In one feasible implementation, the road element set acquisition unit 1403 includes:

[0134] The road feature map acquisition unit is used to extract features from the road image to be excavated based on the feature extraction layer in the feature detection model, and obtain the road feature map corresponding to the road image to be excavated.

[0135] The candidate box set acquisition unit is used to select regions from the road feature map based on the region generation layer in the feature detection model, and obtain the region candidate box sets corresponding to multiple road feature features in the road feature map.

[0136] The candidate box set mining unit is used to mine the candidate box sets of the regions corresponding to the road feature map and the multiple road feature features based on the feature mining layer in the feature detection model, so as to obtain the road image feature set corresponding to the road image to be mined.

[0137] In one feasible implementation, the candidate box set mining unit is specifically used for:

[0138] The pooling layer in the feature mining layer of the feature detection model maps the candidate bounding boxes of the regions corresponding to the multiple road feature features according to the road feature map, and obtains the feature region bounding boxes corresponding to the multiple road feature features respectively.

[0139] The classification layer in the feature mining layer of the feature detection model determines the road feature images corresponding to the multiple road feature features based on the feature region boxes corresponding to the multiple road feature features respectively.

[0140] The classification layer in the feature mining layer of the feature detection model uses the multiple road feature features and the road feature images corresponding to the multiple road feature features as the road image feature set corresponding to the road image to be mined.

[0141] In one possible implementation, the apparatus further includes a detection model training unit, which is specifically used for:

[0142] Obtain road sample images and the corresponding image element label sets for the road sample images;

[0143] The road sample image is input into the detection model to be trained, and the detection model to be trained performs prediction processing on the road sample image to obtain the image prediction feature set corresponding to the road sample image;

[0144] Based on the image feature label set and the image prediction feature set, the parameters of the detection model to be trained are adjusted until the adjusted model meets the training cutoff condition. The adjustment ends and the feature detection model is obtained, with one image feature label and one image prediction feature serving as a pair of training data.

[0145] The map updating apparatus provided in this application embodiment has the same beneficial effects as the map updating method provided in the above embodiments, and therefore will not be described again.

[0146] This application provides a map update device, which can be a server. Figure 15 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 900 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 922 (e.g., one or more processors) and memory 932, and one or more storage media 930 (e.g., one or more mass storage devices) for storing application programs 942 or data 944. The memory 932 and storage media 930 can be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the server. Furthermore, the CPU 922 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the server 900.

[0147] Server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input / output interfaces 958, and / or one or more operating systems 941, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0148] CPU 922 is used to perform the following steps:

[0149] Obtain images of the road to be excavated, a dataset of historical images, and an initial map;

[0150] Based on the road image to be excavated and the historical image dataset, a historical image feature set corresponding to the road image to be excavated in the historical image dataset is obtained, wherein the historical image feature set represents multiple navigation information in the initial map;

[0151] Based on the feature detection model, the road image to be mined is processed to obtain the road image feature set corresponding to the road image to be mined. The feature detection model is used to obtain image features in the image, and the road image feature set represents multiple navigation information in the initial map.

[0152] The historical image feature set and the road image feature set are matched to obtain target features, wherein the target features include image features that have disappeared from the road image feature set.

[0153] The initial map is updated based on the target features to obtain the target map.

[0154] This application also provides another computer device, which can be a terminal device. For example... Figure 16 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. Taking a mobile phone as an example:

[0155] Figure 16 The diagram shown is a block diagram of a portion of the structure of a mobile phone provided in an embodiment of this application. (Reference) Figure 16 The mobile phone includes: a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090, etc. Those skilled in the art will understand that... Figure 16 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0156] The following is combined Figure 16 A detailed introduction to each component of a mobile phone:

[0157] The RF circuit 1010 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 1080; additionally, it transmits uplink data to the base station. Typically, the RF circuit 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF circuit 1010 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0158] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 1020. The memory 1020 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 1020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0159] The input unit 1030 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1031), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1080, and can also receive and execute commands sent by the processor 1080. In addition, the touch panel 1031 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1031, the input unit 1030 may also include other input devices 1032. Specifically, other input devices 1032 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0160] The display unit 1040 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1040 may include a display panel 1041, which may optionally be configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel 1041. Further, a touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it transmits the information to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides corresponding visual output on the display panel 1041 according to the type of touch event. Although in Figure 16 In this embodiment, the touch panel 1031 and the display panel 1041 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0161] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1041 according to the ambient light level, and the proximity sensor can turn off the display panel 1041 and / or the backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0162] The audio circuit 1060, speaker 1061, and microphone 1062 provide an audio interface between the user and the mobile phone. The audio circuit 1060 converts the received audio data into electrical signals and transmits them to the speaker 1061, where the speaker 1061 converts them into sound signals for output. On the other hand, the microphone 1062 converts the collected sound signals into electrical signals, which are then received by the audio circuit 1060, converted into audio data, and then processed by the processor 1080 before being transmitted via the RF circuit 1010 to, for example, another mobile phone, or the audio data can be output to the memory 1020 for further processing.

[0163] WiFi is a short-range wireless transmission technology. Through the WiFi module 1070, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 16 The WiFi module 1070 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0164] The processor 1080 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 1020 and calls data stored in the memory 1020 to perform various functions and process data, thereby collecting overall data and information from the phone. Optionally, the processor 1080 may include one or more processing units; preferably, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1080.

[0165] The mobile phone also includes a power supply 1090 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 1080 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0166] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0167] In this embodiment of the application, the processor 1080 included in the mobile phone also has the following functions:

[0168] Obtain images of the road to be excavated, a dataset of historical images, and an initial map;

[0169] Based on the road image to be excavated and the historical image dataset, a historical image feature set corresponding to the road image to be excavated in the historical image dataset is obtained, wherein the historical image feature set represents multiple navigation information in the initial map;

[0170] Based on the feature detection model, the road image to be mined is processed to obtain the road image feature set corresponding to the road image to be mined. The feature detection model is used to obtain image features in the image, and the road image feature set represents multiple navigation information in the initial map.

[0171] The historical image feature set and the road image feature set are matched to obtain target features, wherein the target features include image features that have disappeared from the road image feature set.

[0172] The initial map is updated based on the target features to obtain the target map.

[0173] This application also provides a computer-readable storage medium for storing a computer program that, when run on a computer device, causes the computer device to perform any one of the map update methods described in the foregoing embodiments.

[0174] This application also provides a computer program product including a computer program, which, when run on a computer device, causes the computer device to execute any one of the implementation methods of a map update method described in the foregoing embodiments.

[0175] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and equipment described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0176] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of the system is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple systems may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0177] The system described as separate components may or may not be physically separate. Components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0179] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing computer programs.

[0180] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0181] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A map update method characterized by comprising: include: Obtain images of the road to be excavated, a dataset of historical images, and an initial map; Based on the road image to be excavated and the historical image dataset, a historical image feature set corresponding to the road image to be excavated in the historical image dataset is obtained, wherein the historical image feature set represents multiple navigation information in the initial map; Based on the feature detection model, the road image to be mined is processed to obtain the road image feature set corresponding to the road image to be mined. The feature detection model is used to obtain image features in the image, and the road image feature set represents multiple navigation information in the initial map. The historical image feature set and the road image feature set are matched to obtain target features, wherein the target features include image features that have disappeared from the road image feature set. The initial map is updated based on the target features to obtain the target map.

2. The method of claim 1, wherein, The historical image feature set includes multiple historical feature features and historical feature images corresponding to each of the multiple historical feature features. The road image feature set includes multiple road feature features and road feature images corresponding to each of the multiple road feature features. The matching process between the historical image feature set and the road image feature set to obtain the target feature includes: The multiple historical feature features and the multiple road feature features are processed together to obtain a feature similarity matrix, wherein each matrix point in the feature similarity matrix represents the feature similarity between a historical feature feature and a road feature feature; The feature similarity matrix is ​​filtered based on a feature similarity threshold to obtain the target feature similarity in the feature similarity matrix that is less than or equal to the feature similarity threshold. Based on the target feature similarity, obtain the target element feature that is similar to the target feature among the multiple historical element features; The target feature and the target feature image corresponding to the target feature are used as the target feature.

3. The method of claim 2, wherein, The multiple historical feature elements include historical image features and historical location features, and the multiple road feature elements include road image features and road location features. The process of aggregating the multiple historical feature elements and the multiple road feature elements to obtain a feature similarity matrix includes: The historical image features and the historical location features are used to form a feature set to obtain a historical feature matrix; The road image features and the road location features are used to create a feature set to obtain a road feature matrix; The historical feature matrix and the road feature matrix are multiplied together to obtain the feature similarity matrix.

4. The method of claim 1, wherein, The historical image dataset includes a historical image set and multiple historical image feature sets corresponding to the historical image set, where one historical image corresponds to one historical image feature set. The step of obtaining the historical image feature set of the historical image corresponding to the road image to be excavated from the historical image dataset, based on the road image to be excavated and the historical image dataset, includes: The road image to be excavated and the historical image set are parsed to obtain the road geographic information corresponding to the road image to be excavated and the historical geographic information set corresponding to the historical image set, wherein one historical image corresponds to one historical geographic information. Based on the road geographic information, the historical geographic information set is filtered to obtain multiple historical geographic information sets that are identical to the road geographic information; Based on the aforementioned historical geographic information, a set of historical image elements corresponding to the road image to be excavated is obtained from the historical image dataset.

5. The method of claim 4, wherein, The step of obtaining the historical image feature set of the historical image corresponding to the road image to be excavated in the historical image dataset based on the multiple historical geographic information includes: Based on the aforementioned historical geographic information, determine the historical images in the historical image set that correspond to the aforementioned historical geographic information, respectively; Trajectory points are extracted from the image to be excavated and the historical images corresponding to the multiple historical geographic information to obtain the road trajectory point sequence corresponding to the image to be excavated and the historical trajectory point sequence corresponding to the multiple historical images. Based on the road trajectory point sequence and multiple historical trajectory point sequences, a set of historical image elements corresponding to the road image to be excavated in the historical image dataset is obtained.

6. The method of claim 5, wherein, The step of obtaining a set of historical image elements corresponding to the road image to be mined from the historical image dataset based on the road trajectory point sequence and multiple historical trajectory point sequences includes: Alignment processing is performed on the road trajectory point sequence and multiple historical trajectory point sequences to obtain multiple trajectory point similarity sets, one of which is obtained based on the road trajectory point sequence and a historical trajectory point sequence; The multiple trajectory point similarity sets are processed to construct a trajectory point similarity matrix; Based on the optimal solution of the trajectory point similarity matrix, obtain the historical image corresponding to the road image to be excavated from the plurality of historical images; Based on the historical images, obtain the historical image element set of the historical images.

7. The method of claim 1, wherein, The feature detection model includes a feature extraction layer, a region generation layer, and a feature mining layer. The feature detection model is used to mine the road image to be mined, obtaining a road image feature set corresponding to the road image to be mined, including: Based on the feature extraction layer in the feature detection model, feature extraction is performed on the road image to be excavated to obtain the road feature map corresponding to the road image to be excavated. Based on the region generation layer in the feature detection model, region selection is performed on the road feature map to obtain a set of region candidate boxes corresponding to multiple road feature features in the road feature map. Based on the feature mining layer in the feature detection model, the road feature map and the region candidate box sets corresponding to the multiple road feature features are mined to obtain the road image feature set corresponding to the road image to be mined.

8. The method of claim 7, wherein, The feature mining layer includes a pooling layer and a classification layer. The feature mining layer in the feature detection model performs mining processing on the road feature map and the candidate bounding boxes corresponding to the multiple road feature features to obtain the road image feature set corresponding to the road image to be mined, including: The pooling layer in the feature mining layer of the feature detection model maps the candidate bounding boxes of the regions corresponding to the multiple road feature features according to the road feature map, and obtains the feature region bounding boxes corresponding to the multiple road feature features respectively. The classification layer in the feature mining layer of the feature detection model determines the road feature images corresponding to the multiple road feature features based on the feature region boxes corresponding to the multiple road feature features respectively. The classification layer in the feature mining layer of the feature detection model uses the multiple road feature features and the road feature images corresponding to the multiple road feature features as the road image feature set corresponding to the road image to be mined.

9. The method of claim 1, wherein, The feature detection model was trained in the following way: Obtain road sample images and the corresponding image element label sets for the road sample images; The road sample image is input into the detection model to be trained, and the detection model to be trained performs prediction processing on the road sample image to obtain the image prediction feature set corresponding to the road sample image; Based on the image feature label set and the image prediction feature set, the parameters of the detection model to be trained are adjusted until the adjusted model meets the training cutoff condition. The adjustment ends and the feature detection model is obtained, with one image feature label and one image prediction feature serving as a pair of training data.

10. A map update device characterized by comprising: include: The image data acquisition unit is used to acquire images of the road to be excavated, historical image datasets, and an initial map; The historical element set acquisition unit is used to obtain a historical image element set of the historical image corresponding to the road image to be excavated in the historical image dataset based on the road image to be excavated and the historical image dataset, wherein the historical image element set represents multiple navigation information in the initial map; The road element set acquisition unit is used to perform mining processing on the road image to be mined based on the element detection model to obtain the road image element set corresponding to the road image to be mined, wherein the element detection model is used to obtain image elements in the image, and the road image element set represents multiple navigation information in the initial map. The target element acquisition unit is used to perform matching processing on the historical image element set and the road image element set to obtain target elements, wherein the target elements include image elements that have disappeared from the road image element set; The target map acquisition unit is used to update the initial map based on the target features to obtain the target map.

11. A computer device, comprising: The device includes a processor and a memory: The memory is used to store computer programs and to transfer the computer programs to the processor; The processor is configured to execute the steps of the map update method according to any one of claims 1 to 9, based on instructions in the computer program.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when executed by a computer device, implements the steps of the map update method according to any one of claims 1 to 9.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a computer device, implements the steps of the map updating method according to any one of claims 1 to 9.

Images