Image matching method, map information updating method, and related apparatus

Abstract

The present application discloses an image matching method, a map information updating method, and a related apparatus. The image matching method of the present application comprises: performing feature extraction processing on a first image to be matched, so as to obtain K first feature maps; performing feature extraction processing on a second image to be matched, so as to obtain K second feature maps; on the basis of the K first feature maps, obtaining a first feature vector of each first feature point among M first feature points of said first image; on the basis of the K second feature maps, obtaining a second feature vector of each second feature point among N second feature points of said second image; on the basis of the first feature vectors of the first feature points and the second feature vectors of the second feature points, determining the number of feature point pairs; and on the basis of the number of feature point pairs, determining an image matching result. According to the present application, by extracting semantic features and physical description features of images, image information can be learned more comprehensively, thereby helping to improve the accuracy of image matching.

Description

Image matching method, map information updating method and related devices

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on July 7, 2023, application number 202310831318.3, and application name “A method for image matching, a method for updating map information, and related devices”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to image matching technology and map information updating technology. Background Art

[0003] During the map road data collection process, in order to update the map information, it is usually necessary to compare the newly collected data with the results in the historical data. For example, the similarity of the newly collected image and the image in the historical data can be compared to find the elements that have changed in the map, and then the map can be updated.

[0004] Currently, convolutional neural networks can be trained using large amounts of annotated data to extract and classify high-level semantic features from images, ultimately yielding image recognition results. Related technologies utilize object detection networks to identify features in new and historical images. By comparing the features in the two images to determine whether they are consistent, it can be used to determine whether a map update is necessary.

[0005] However, current solutions have at least the following issues: Because images contain numerous elements, the number of elements extracted by the object detection network is limited. Consequently, element recognition may be inaccurate, leading to a high error rate in image matching. No effective solution has yet been proposed to address these issues.

[0006] Summary of the Invention

[0007] The present invention provides an image matching method, a map information updating method, and related devices. By extracting semantic features and physical descriptive features of an image, the method enables more comprehensive learning of image information. Consequently, matching feature points using feature vectors enhances overall image understanding and improves image matching accuracy.

[0008] In view of this, the present application provides, on one hand, a method for image matching, which is performed by a computer device and includes:

[0009] Performing feature extraction processing on the first image to be matched to obtain K first feature maps, wherein the first image to be matched has M first feature points, each first feature map includes the M first feature points, K is an integer greater than or equal to 1, and M is an integer greater than 1;

[0010] Performing feature extraction processing on the second image to be matched to obtain K second feature maps, wherein the second image to be matched has N second feature points, each second feature map includes the N second feature points, and N is an integer greater than 1;

[0011] Obtaining, based on the K first feature maps, a first feature vector for each of the M first feature points, wherein the first feature vector includes K first elements, each first element being derived from a different first feature map, and the M first feature vectors corresponding to the first image to be matched are used to describe a first semantic feature and a first physical description feature of the first image to be matched;

[0012] Obtaining, according to the K second feature maps, a second feature vector for each of the N second feature points, wherein the second feature vector includes K second elements, each second element is derived from a different second feature map, and the N second feature vectors corresponding to the second image to be matched are used to describe the second semantic feature and the second physical description feature of the second image to be matched;

[0013] Determining the number of feature point pairs based on the first feature vector of each first feature point and the second feature vector of each second feature point, wherein the number of feature point pairs represents the number of successful matches between the first feature point and the second feature point;

[0014] An image matching result between the first image to be matched and the second image to be matched is determined according to the number of feature point pairs.

[0015] Another aspect of the present application provides a method for updating map information, the method being executed by a computer device, comprising:

[0016] Performing feature extraction processing on the historical road image to obtain K first feature maps, wherein the historical road image has M first feature points, each first feature map includes the M first feature points, K is an integer greater than or equal to 1, and M is an integer greater than 1;

[0017] Performing feature extraction processing on the road image to be processed to obtain K second feature maps, wherein the road image to be processed is acquired later than the historical road image, the road image to be processed has N second feature points, and each second feature map includes the N second feature points, where N is an integer greater than 1;

[0018] Obtaining, based on the K first feature maps, a first feature vector for each of the M first feature points, wherein the first feature vector includes K first elements, each first element being derived from a different first feature map, and the M first feature vectors corresponding to the historical road image are used to describe a first semantic feature and a first physical description feature of the historical road image;

[0019] Obtaining, based on the K second feature maps, a second feature vector for each of the N second feature points, wherein the second feature vector includes K second elements, each second element being derived from a different second feature map, and the N second feature vectors corresponding to the road image to be processed are used to describe a second semantic feature and a second physical description feature of the road image to be processed;

[0020] Determining the number of feature point pairs based on the first feature vector of each first feature point and the second feature vector of each second feature point, wherein the number of feature point pairs represents the number of successful matches between the first feature point and the second feature point;

[0021] If it is determined based on the number of feature point pairs that the historical road image and the road image to be processed fail to match, generating an image element set based on the element recognition results of the historical road image and the element recognition results of the road image to be processed, wherein the image element set is derived from at least one of the historical road image and the road image to be processed;

[0022] Update the map information based on the image feature set.

[0023] Another aspect of the present application provides an image matching device, which is deployed on a computer device and includes:

[0024] a processing module, configured to perform feature extraction processing on the first image to be matched to obtain K first feature maps, wherein the first image to be matched has M first feature points, each first feature map includes the M first feature points, K is an integer greater than or equal to 1, and M is an integer greater than 1;

[0025] The processing module is further configured to perform feature extraction processing on the second image to be matched to obtain K second feature maps, wherein the second image to be matched has N second feature points, each second feature map includes the N second feature points, and N is an integer greater than 1;

[0026] an acquisition module, configured to acquire, based on the K first feature maps, a first feature vector for each of the M first feature points, wherein the first feature vector includes K first elements, each first element being derived from a different first feature map, and the M first feature vectors corresponding to the first image to be matched are used to describe a first semantic feature and a first physical description feature of the first image to be matched;

[0027] The acquisition module is further used to obtain a second feature vector for each second feature point in the N second feature points based on the K second feature maps, wherein the second feature vector includes K second elements, each second element is derived from a different second feature map, and the N second feature vectors corresponding to the second image to be matched are used to describe the second semantic feature and the second physical description feature of the second image to be matched;

[0028] a determination module, configured to determine a number of feature point pairs based on the first feature vector of each first feature point and the second feature vector of each second feature point, wherein the number of feature point pairs represents a number of successful matches between the first feature point and the second feature point;

[0029] The determination module is further configured to determine an image matching result between the first image to be matched and the second image to be matched according to the number of feature point pairs.

[0030] Another aspect of the present application provides a map information updating device, which is deployed on a computer device and includes:

[0031] a processing module, configured to perform feature extraction processing on the historical road image to obtain K first feature maps, wherein the historical road image has M first feature points, each first feature map includes the M first feature points, K is an integer greater than or equal to 1, and M is an integer greater than 1;

[0032] The processing module is further configured to perform feature extraction processing on the road image to be processed to obtain K second feature maps, wherein the road image to be processed is acquired later than the historical road image, the road image to be processed has N second feature points, and each second feature map includes the N second feature points, where N is an integer greater than 1;

[0033] an acquisition module, configured to acquire, based on the K first feature maps, a first feature vector for each of the M first feature points, wherein the first feature vector includes K first elements, each first element being derived from a different first feature map, and the M first feature vectors corresponding to the historical road image are used to describe a first semantic feature and a first physical description feature of the historical road image;

[0034] The acquisition module is further configured to acquire, based on the K second feature maps, a second feature vector for each of the N second feature points, wherein the second feature vector includes K second elements, each second element being derived from a different second feature map, and the N second feature vectors corresponding to the road image to be processed are used to describe the second semantic feature and the second physical description feature of the road image to be processed;

[0035] a determination module, configured to determine a number of feature point pairs based on the first feature vector of each first feature point and the second feature vector of each second feature point, wherein the number of feature point pairs represents a number of successful matches between the first feature point and the second feature point;

[0036] The determination module is further configured to, when it is determined based on the number of feature point pairs that the historical road image and the road image to be processed fail to match, generate an image element set based on the element recognition results of the historical road image and the element recognition results of the road image to be processed, wherein the image element set is derived from at least one of the historical road image and the road image to be processed;

[0037] The update module is used to update the map information according to the image feature set.

[0038] On the other hand, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned methods when executing the computer program.

[0039] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned methods when the computer program is executed by a processor.

[0040] Another aspect of the present application provides a computer program product, including a computer program, which implements the above-mentioned methods when executed by a processor.

[0041] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0042] In an embodiment of the present application, a method for image matching is provided. First, feature extraction processing is performed on a first image to be matched to obtain K first feature maps, and feature extraction processing is performed on a second image to be matched to obtain K second feature maps. The first image to be matched has M first feature points, each of which includes the M first feature points, and the second image to be matched has N second feature points, each of which includes the N second feature points. A first feature vector is then obtained for each first feature point based on the K first feature maps, and a second feature vector is obtained for each second feature point based on the K second feature maps. The M first feature vectors corresponding to the first image to be matched are used to describe the first semantic features and first physical descriptive features of the first image to be matched, and can reflect the global features of the first image to be matched. The N second feature vectors corresponding to the second image to be matched are used to describe the second semantic features and second physical descriptive features of the second image to be matched, and can reflect the global features of the second image to be matched. Then, the number of feature point pairs is determined based on each first feature vector and each second feature vector. Finally, the image matching result is determined based on the number of feature point pairs. By extracting deep features from both images, we obtain feature vectors for each feature point in each image. These feature vectors represent the semantic and physical features of the image (i.e., global features), enabling a more comprehensive understanding of the image information. Using feature vectors to match feature points improves overall understanding of the image, thereby increasing the accuracy of image matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] FIG1 is a schematic diagram of an implementation environment of the image matching method according to an embodiment of the present application;

[0044] FIG2 is a schematic diagram of an implementation framework of the image matching method in an embodiment of the present application;

[0045] FIG3 is a flow chart of an image matching method according to an embodiment of the present application;

[0046] FIG4 is a schematic diagram of adjusting the size of an image to be matched in an embodiment of the present application;

[0047] FIG5 is another schematic diagram of adjusting the size of the image to be matched according to an embodiment of the present application;

[0048] FIG6 is a schematic diagram of generating a feature vector based on an image to be matched in an embodiment of the present application;

[0049] FIG7 is a schematic diagram of constructing a feature vector based on a feature graph in an embodiment of the present application;

[0050] FIG8 is a schematic diagram of feature point matching between images in an embodiment of the present application;

[0051] FIG9 is another schematic diagram of feature point matching between images according to an embodiment of the present application;

[0052] FIG10 is a schematic diagram of feature point matching based on K nearest neighbors in an embodiment of the present application;

[0053] FIG11 is a flow chart of a method for updating map information according to an embodiment of the present application;

[0054] FIG12 is a schematic diagram of global scene understanding according to an embodiment of the present application;

[0055] FIG13 is a schematic diagram showing a set of image elements in an embodiment of the present application;

[0056] FIG14 is a schematic diagram of an image matching device according to an embodiment of the present application;

[0057] FIG15 is a schematic diagram of a map information updating device according to an embodiment of the present application;

[0058] FIG16 is a schematic structural diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The embodiments of the present application provide an image matching method, a map information updating method, and related devices, which use elements used to describe physical description features to construct feature vectors, and match the feature points of the image based on these feature vectors, which can improve the ability to understand the entire image, and thus help improve the accuracy of image matching.

[0060] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the numbers used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "corresponding to" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0061] Image similarity algorithms are methods used to assess the similarity between two different images. With the rapid development of computer vision (CV) technology in recent years, image similarity algorithms have received widespread attention and have broad application prospects. They can be used to identify complex images, analyze and extract image content, make more accurate decisions and judgments, and provide reliable data for artificial intelligence (AI) technology. Currently, deep learning-based classification networks can be used to recognize images and determine whether different images are similar based on the recognition results. Alternatively, shallow image features (such as texture, edges, and corners) can be extracted and used to determine whether different images are similar based on these shallow features. Regardless of the method used, the accuracy of image matching still needs to be improved.

[0062] Based on this, the present application provides a method for image matching, which extracts global features from different images respectively, and uses the global features to construct a feature vector for each feature point. Image similarity comparison is performed based on the feature vectors to determine the image difference result. Among them, the global features include the semantic features and physical description features of the image. Utilizing the global features of feature points can improve the ability to understand the image as a whole, thereby improving the accuracy of image matching. For the image matching method of the present application, at least one of the following scenarios is included in the application.

[0063] 1. Map information update scenario;

[0064] During the map road data collection process, newly collected road images need to be compared with historical road images in order to update the map information. For example, a large number of historical road images are stored in the backend database. These road images can be uploaded by users or captured by collection vehicles. Each historical road image can also record its corresponding collection location (e.g., longitude and latitude information) and collection time.

[0065] Based on this, when a new road image is acquired, one or more historical road images closest to the acquisition location can be searched from the backend database based on the acquisition location. Furthermore, based on the acquisition time of the historical road image, the most recently acquired historical road image can be obtained. This historical road image is then compared with the newly acquired road image for similarity, identifying any changed features on the map and subsequently updating the map.

[0066] 2. Security monitoring scenarios;

[0067] Monitoring systems are deployed in public areas such as streets, buildings, and schools, regularly capturing images of these areas. First, personnel select a single image from the captured data as a reference image. Subsequently, each subsequent image is compared against the reference image for similarity. If the similarity between the images is low, personnel will inspect the scene to determine if there are any safety hazards, such as a skewed shop sign or leaning tree. This allows for timely identification and resolution of these public safety hazards.

[0068] 3. Image screening scenario;

[0069] In the field of machine learning, a large number of images are often collected for training. However, these images may contain a large number of duplicates or similar situations, so they need to be screened and eliminated. In order to improve the efficiency of screening and reduce the labor cost and time cost required for data screening, the image matching method provided by this application can be used to perform similarity comparison between two images. If the similarity between the images is high, the two images are considered to be duplicates. Therefore, one of the images can be automatically eliminated, thereby achieving the purpose of automatic image screening.

[0070] It should be noted that the above application scenarios are only examples, and the image matching method provided in this embodiment can also be applied to other scenarios, which are not limited here.

[0071] It is understandable that the present application relates to the field of automatic image recognition, and specifically to CV technology. CV is a science that studies how to make machines "see". To put it more concretely, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing so that the computer processing becomes an image that is more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, CV studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. CV technology generally includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, three-dimensional (3D) technology, virtual reality, augmented reality, simultaneous positioning and map construction, autonomous driving, smart transportation and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0072] The method provided herein can be applied to the implementation environment shown in FIG1 , which includes a terminal 110 and a server 120. Furthermore, the terminal 110 and the server 120 can communicate with each other via a communication network 130. The communication network 130 utilizes standard communication technologies and / or protocols, typically the Internet, but can also be any network, including but not limited to Bluetooth, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile network, a dedicated network, or any combination of virtual private networks. In some embodiments, customized or dedicated data communication technologies can be used in place of or in addition to the aforementioned data communication technologies.

[0073] The terminals 110 involved in this application include, but are not limited to, mobile phones, driving recorders, vehicle-mounted camera devices, tablet computers, laptop computers, desktop computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc. The client is deployed on the terminal 110 and can be run on the terminal 110 in the form of a browser or an independent application (APP).

[0074] The server 120 involved in this application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and AI platforms.

[0075] In combination with the above implementation environment, in step S1, the terminal 110 captures the first image to be matched. In step S2, the terminal 110 sends the first image to be matched to the server 120 via the communication network 130. In step S3, the server 120 obtains the second image to be matched from the database. Based on this, in step S4, the server 120 calls the feature extraction network to extract global features of the first image to be matched and the second image to be matched, respectively. In step S5, the server 120 constructs a first feature vector for each feature point in the first image to be matched based on the global features of the first image to be matched, and constructs a second feature vector for each feature point in the second image to be matched based on the global features of the second image to be matched. In step S6, based on each first feature vector and each second feature point vector, the feature points in the two images to be matched are matched to generate an image matching result.

[0076] It should be noted that this application uses the configuration of the feature extraction network deployed on the server 120 as an example for description. In some embodiments, the configuration of the feature extraction network can also be deployed on the terminal 110. In some embodiments, part of the configuration of the feature extraction network is deployed on the terminal 110, and part of the configuration is deployed on the server 120.

[0077] Based on the implementation environment shown in Figure 1, an overall process of the image matching method will be introduced below in conjunction with Figure 2. Please refer to Figure 2, which is a schematic diagram of an implementation framework of the image matching method in an embodiment of the present application. As shown in the figure, in step A1, the image to be matched is collected by the terminal. In step A2, the global scene is understood. Among them, step A2 specifically includes step A21, step A22 and step A23. In step A3, based on the understanding of the global scene, the image is output, that is, the image after the global scene understanding can be stored in the database for subsequent similarity comparison.

[0078] In step A21, a deep learning and feature extraction network is used to extract the entire image features of the captured image to obtain the global features required for the image, that is, including the semantic features and physical description features of the image. In step A22, after obtaining the global features, a feature vector is constructed for each feature point in the image. Thus, the feature points in the two images are matched to obtain the number of feature point pairs in the two images. In step A23, an image matching result is generated based on the number of feature point pairs. If the image matching result indicates that the two images are successfully matched, image-to-image differencing can be performed. Otherwise, image-to-image differencing cannot be performed.

[0079] Since this application involves some terms related to professional fields, they will be explained below for ease of understanding.

[0080] (1) Image elements: refers to useful physical point information in map data images, such as traffic restriction signs, speed limit signs, and electronic cameras.

[0081] (2) Convolutional neural networks (CNN): It is a type of feedforward neural networks (FNN) that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning.

[0082] (3) Classification network: It is used to identify the categories of image elements using neural networks. The input of the classification network is image data, and the output of the classification network is the category of elements contained in the image.

[0083] (4) Feature similarity: A measure used to evaluate the similarity between two spatial features. For example, distance or angle is used to measure the similarity.

[0084] (5) Image-to-image difference: For two images, if a difference is found, it is considered that the scene has changed. If two images are similar, it is considered that the content of the two images is consistent and can be differentiated.

[0085] In combination with the above introduction, the image matching method in this application will be introduced below. Please refer to Figure 3. The image matching method in the embodiment of this application can be completed independently by the server, independently by the terminal, or by the terminal and the server in cooperation. The method of this application includes:

[0086] 210. Perform feature extraction on the first image to be matched to obtain K first feature maps, wherein the first image to be matched has M first feature points, each first feature map includes the M first feature points, K is an integer greater than or equal to 1, and M is an integer greater than 1;

[0087] In one or more embodiments, a first image to be matched is obtained. It is understandable that the first image to be matched can be an image uploaded by a user, or an image stored in a backend database, or an image crawled from a web page, etc., which is not limited here.

[0088] Feature points may be points in an image that are distinctive or unique, such as corner points and edge points. These feature points may be detected using an algorithm, such as a Scale-invariant Feature Transform (SIFT) algorithm or a Speeded-Up Robust Features (SURF) algorithm. In the embodiment of the present application, the first image to be matched is detected to obtain M first feature points of the first image to be matched.

[0089] Feature extraction processing can refer to the process of extracting effective information from a set of data or raw data (in the embodiment of the present application, the first image to be matched and the second image to be matched). This information is called features, and feature extraction processing can also bring better interpretability. In the embodiment of the present application, the features obtained by feature extraction processing are embodied in the form of feature maps. In a possible implementation, a feature extraction network can be used to perform feature extraction processing on the first image to be matched, thereby obtaining K first feature maps. Among them, the feature extraction network can specifically adopt CNN, or residual network (ResNet), or visual geometry group network (VGG network), etc. The feature extraction network uses K convolution kernels (kernels) for feature extraction, and each kernel is used to extract the features of a channel, thereby obtaining the first feature maps of K channels. Among them, each first feature map has the same size, and each first feature map includes the M first feature points detected above. The M first feature points of the first image to be matched are respectively reflected in the K first feature maps, but the same first feature point may have different manifestations in different first feature maps. For example, if the size of the first feature map is 100×100, then M is 10000.

[0090] 220. Perform feature extraction on the second image to be matched to obtain K second feature maps, wherein the second image to be matched has N second feature points, and each second feature map includes the N second feature points, where N is an integer greater than 1;

[0091] In one or more embodiments, a second image to be matched is obtained. It is understood that the second image to be matched can be an image uploaded by a user, or an image stored in a backend database, or an image crawled from a web page, etc., which is not limited here. The first image to be matched and the second image to be matched are both black and white images, or both are color (red, green, blue, RGB) images. For black and white images, a two-dimensional kernel is used, for example, the size of the two-dimensional kernel is 5×5. For RGB images, a three-dimensional kernel is used, for example, the size of the three-dimensional kernel is 5×5×3.

[0092] In an embodiment of the present application, the second image to be matched is detected to obtain N second feature points of the second image to be matched. In one possible implementation method, a feature extraction network can be used to perform feature extraction processing on the second image to be matched, thereby obtaining K second feature maps. Each second feature map has the same size, and each second feature map includes the N second feature points detected above. The N second feature points of the second image to be matched are respectively reflected in the K second feature maps, but the same second feature point may have different expressions in different second feature maps. For example, the size of the second feature map is 100×100, then N is 10000. N and M can be the same value or different values, which is not limited here.

[0093] 230. Obtain a first feature vector for each of the M first feature points based on the K first feature maps, wherein the first feature vector includes K first elements, each first element is derived from a different first feature map, and the M first feature vectors corresponding to the first image to be matched are used to describe a first semantic feature and a first physical description feature of the first image to be matched;

[0094] In one or more embodiments, each first feature map includes M first elements, that is, each first feature point in the first feature map corresponds to a first element. Since the M first feature points of the first image to be matched have different representations in the K first feature maps, in order to more richly and comprehensively reflect the characteristics of each first feature point, for each first feature point, that is, the first feature point belonging to the same position in the K first feature maps, the first element corresponding to the first feature point can be obtained from each first feature map to form a first feature vector. For example, after obtaining the K first feature maps, the first elements corresponding to the K first feature points belonging to the same position are spliced ​​to obtain the first feature vector of the first feature point, thereby obtaining the first feature vectors corresponding to the M first feature points of the first image to be matched. Since each first element in the first feature vector comes from a different first feature map, M first feature vectors can be generated based on the K first feature maps, and each first feature vector includes K first elements.

[0095] A first feature point belongs to the same position of the first image to be matched, so the first feature vector can reflect the global features of the corresponding position. By combining the first elements of the first feature points at the same position of different first feature maps, a richer and more comprehensive feature representation of the first feature points can be obtained, thereby improving the matching accuracy.

[0096] In one possible implementation, some of the K kernels are used to extract semantic features of the image, while others are used to extract physical descriptive features of the image. Semantic features can effectively summarize semantic information, such as features like "traffic restriction sign" and "electronic eye." Physical descriptive features can describe the physical properties of semantic features, including but not limited to spatial features, rotational properties, and color properties. Based on this, M first feature vectors can be used to describe the first semantic features and first physical descriptive features of the first image to be matched.

[0097] 240. Obtain a second feature vector for each of the N second feature points based on the K second feature maps, wherein the second feature vector includes K second elements, each second element is derived from a different second feature map, and the N second feature vectors corresponding to the second image to be matched are used to describe the second semantic feature and the second physical description feature of the second image to be matched;

[0098] In one or more embodiments, each second feature map includes N second elements, that is, each second feature point in the second feature map corresponds to a second element. Since the N second feature points of the second image to be matched have different representations in the K second feature maps, in order to more richly and comprehensively reflect the characteristics of each second feature point, for each second feature point, that is, the second feature points belonging to the same position in the K second feature maps, the second element corresponding to the second feature point can be obtained from each second feature map to form a second feature vector. For example, after obtaining the K second feature maps, the second elements corresponding to the K second feature points belonging to the same position are spliced ​​to obtain the second feature vector of the second feature point, thereby obtaining the second feature vectors corresponding to the N second feature points of the second image to be matched. Since each second element in the second feature vector comes from a different second feature map, N second feature vectors can be generated based on the K second feature maps, and each second feature vector includes K second elements. Similarly, the N second feature vectors can be used to describe the second semantic features and second physical description features of the second image to be matched.

[0099] A second feature point belongs to the same position of the second image to be matched, so the second feature vector can reflect the global features of the corresponding position. By combining the first elements of the second feature points at the same position in different second feature maps, a richer and more comprehensive feature representation of the second feature points can be obtained, thereby improving the matching accuracy.

[0100] 250. Determine the number of feature point pairs based on the first feature vector of each first feature point and the second feature vector of each second feature point, wherein the number of feature point pairs represents the number of successful matches between the first feature point and the second feature point;

[0101] In one or more embodiments, first feature points of a first image to be matched are matched with second feature points of a second image to be matched, and the number of successfully matched feature point pairs is calculated. A successfully matched feature point pair includes a first feature point and a second feature point. Assuming the number of feature point pairs is 5, this means that five first feature points are successfully matched with five second feature points, one by one.

[0102] It should be noted that when determining the number of feature point pairs, for the first and second feature points to be matched, the first feature vector of the first feature point can be compared with the second feature vector of the second feature point. The comparison can be performed by calculating the similarity between the first and second feature vectors, and then determining whether a match is successful based on the similarity. In some cases, the higher the similarity, the more likely a match is successful.

[0103] The embodiments of the present application do not limit the method for calculating the similarity. For example, the similarity between the first eigenvector and the second eigenvector can be reflected by the distance between the first eigenvector and the second eigenvector. Generally, the larger the distance, the smaller the similarity. For example, the similarity between the first eigenvector and the second eigenvector can also be directly calculated, such as cosine similarity.

[0104] 260. Determine an image matching result between the first image to be matched and the second image to be matched according to the number of feature point pairs.

[0105] It is understood that the number of feature point pairs indicates the number of successful matches between the first feature point and the second feature point. A greater number of feature point pairs indicates a greater number of successful matches between the first feature point and the second feature point, i.e., the greater the number of similar feature points between the first image to be matched and the second image to be matched, the more similar the first image to be matched and the second image to be matched are. The image matching result may include a successful match or a failed match. The greater the similarity between the first image to be matched and the second image to be matched, the more likely the first image to be matched and the second image to be matched are to be successfully matched. Otherwise, the more likely the match is to fail. Therefore, in the embodiment of the present application, the image matching result between the first image to be matched and the second image to be matched can be determined based on the number of feature point pairs.

[0106] In one or more embodiments, the image matching result between the first image to be matched and the second image to be matched can be determined based on the ratio of the number of feature point pairs to the total number of feature points involved in the matching. If the ratio is sufficiently large, it indicates that the number of successfully matched feature points meets the requirement, and therefore the image matching result is a successful match between the two images. Otherwise, it indicates that the two images failed to match.

[0107] In an embodiment of the present application, a method for image matching is provided. By performing deep feature extraction on two images, feature vectors are obtained for each feature point in each image. These feature vectors can represent the semantic and physical features of the image, thereby enabling a more comprehensive understanding of the image information. Based on this, matching feature points using feature vectors can enhance the ability to understand the overall image, thereby improving the accuracy of image matching.

[0108] Based on one or more embodiments corresponding to FIG. 3 above, another optional embodiment provided by the embodiment of the present application may further include:

[0109] Acquire a first initial image to be matched and a second initial image to be matched;

[0110] When the size of the first initial image to be matched is larger than the preset size, the first initial image to be matched is reduced in size to obtain the first image to be matched;

[0111] When the size of the first initial image to be matched is smaller than the preset size, the first initial image to be matched is enlarged to obtain the first image to be matched, or the first initial image to be matched is filled to obtain the first image to be matched;

[0112] When the size of the second initial image to be matched is larger than the preset size, the second initial image to be matched is reduced in size to obtain a second image to be matched;

[0113] When the size of the second initial image to be matched is smaller than the preset size, the second initial image to be matched is enlarged to obtain the second image to be matched, or the second initial image to be matched is filled to obtain the second image to be matched.

[0114] In one or more embodiments, a method for resizing an initial image to be matched is described. As can be seen from the aforementioned embodiments, the initial images to be matched (a first initial image to be matched and a second initial image to be matched) are resized so that the resulting first image to be matched and the second image to be matched have the same size. Based on this, the number of first feature points extracted from the first image to be matched is consistent with the number of second feature points extracted from the second image to be matched, i.e., M = N.

[0115] 1. Reduce the size of the image;

[0116] For ease of understanding, please refer to FIG4 , which is a schematic diagram of adjusting the size of the initial image to be matched in an embodiment of the present application. As shown in FIG4 (A), assuming that the image is the first initial image to be matched, and assuming that the size of the first initial image to be matched is larger than a preset size. Based on this, the first initial image to be matched can be proportionally reduced in size to obtain the first image to be matched, so that the width of the obtained first image to be matched meets the preset width, or the height meets the preset height.

[0117] As shown in FIG4 (B), after the first to-be-matched initial image is scaled down, its width can meet the preset width, but its height is less than the preset height. Based on this, the redundant part can also be filled, for example, with black pixels.

[0118] It should be noted that the second initial image to be matched can also be reduced in size in a similar manner, which will not be described in detail here.

[0119] 2. Enlarge the image size;

[0120] For ease of understanding, please refer to FIG5 , which is another schematic diagram of adjusting the size of the initial image to be matched in an embodiment of the present application. As shown in FIG5 (A), it is assumed that the image is the first initial image to be matched, and it is assumed that the size of the first initial image to be matched is smaller than the preset size. Based on this, the first initial image to be matched can be scaled up to obtain the first image to be matched, or the first initial image to be matched can be padded to obtain the first image to be matched, so that the width of the first image to be matched can meet the preset width, or the height can meet the preset height.

[0121] As shown in FIG5 (B), after the first image to be matched is enlarged in proportion, its width can meet the preset width, but its height is less than the preset height. Based on this, the redundant part can also be filled, for example, with black pixels.

[0122] It should be noted that the second image to be matched can also be enlarged in a similar manner, which will not be described in detail here.

[0123] Secondly, embodiments of the present application provide a method for resizing the initial image to be matched. This method allows the images involved in matching to be scaled to a uniform size. This allows the same image preprocessing to be maintained during both the training and inference phases of the feature extraction network, thereby fully leveraging the model's inference performance.

[0124] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the present application, feature extraction processing is performed on the first image to be matched to obtain K first feature maps, specifically including:

[0125] Based on the first image to be matched, obtaining K first convolution feature maps through the convolution layer included in the feature extraction network;

[0126] The K first convolution feature maps are respectively normalized by a normalization layer included in the feature extraction network to obtain K first normalized feature maps;

[0127] Performing nonlinear mapping on the K first normalized feature maps through an activation layer included in a feature extraction network to obtain K first feature maps;

[0128] Perform feature extraction on the second image to be matched to obtain K second feature maps, specifically including:

[0129] Based on the second image to be matched, obtaining K second convolution feature maps through the convolution layer included in the feature extraction network;

[0130] The K second convolution feature maps are respectively normalized by a normalization layer included in the feature extraction network to obtain K second normalized feature maps;

[0131] The K second normalized feature maps are respectively nonlinearly mapped through the activation layer included in the feature extraction network to obtain K second feature maps.

[0132] In one or more embodiments, a method for extracting feature maps using a feature extraction network is described. As can be seen from the aforementioned embodiments, the feature extraction network can be used to extract feature maps for a first image to be matched and a second image to be matched. The feature extraction network includes K kernels, each of which is used to extract a feature map.

[0133] For ease of understanding, please refer to Figure 6, which is a schematic diagram of generating feature vectors based on the image to be matched in an embodiment of the present application. As shown in the figure, taking the first image to be matched as an example, it is assumed that the first image to be matched is an 8×8 RGB image, that is, expressed as 8×8×3. Assume that the feature extraction network uses 5 kernels, and the size of each kernel is 3×3×3. Based on this, each kernel is used to perform feature extraction on the first image to be matched respectively. Based on this, 5 kernels can extract 5 first feature maps, and it is assumed that the size of each first feature map is 6×6. Therefore, the first elements corresponding to the first feature points belonging to the same position in the 5 first feature maps are spliced ​​to obtain 36 first feature vectors, and the dimension of each first feature vector is 5.

[0134] In practical applications, feature extraction networks include not only convolutional layers but also batch normalization (BN) and activation layers. The activation layer can use a rectified linear unit (ReLU).

[0135] Taking the first image to be matched as an example, first, the convolutional layer included in the feature extraction network is used to extract basic features such as image edge texture, thereby obtaining K first convolutional feature maps. Then, the K first convolutional feature maps extracted by the convolutional layer are normalized according to the normal distribution using the batch normalization layer included in the feature extraction network to filter out noise features in the features, thereby obtaining K first normalized feature maps. Finally, the K first normalized feature maps are nonlinearly mapped using the activation layer included in the feature extraction network to obtain K first feature maps.

[0136] It should be noted that the second image to be matched can also be processed in a similar manner to obtain K second feature maps, which will not be described in detail here.

[0137] Secondly, the embodiments of the present application provide a method for extracting feature maps using a feature extraction network. Through the above method, the convolutional layers included in the feature extraction network can extract the basic features of the image. The normalization layer can filter out noise in the features, making the model converge more quickly. The activation layer can enhance the generalization ability of the model.

[0138] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, obtaining a first feature vector of each first feature point in M ​​first feature points according to K first feature maps specifically includes:

[0139] According to the K first feature maps, a first feature sub-son and a first descriptor of the first image to be matched are generated, wherein the first feature sub-son is used to describe the first semantic feature of the first image to be matched, and the first descriptor is used to describe the first physical description feature of the first feature sub-son. The size of the first feature sub-son is (w×h×d), and the size of the first descriptor is (w×h×t), where w represents the width of the first feature map, h represents the height of the first feature map, d represents depth information, and t represents the number of types of the first physical description feature. w, h, d, and t are all integers greater than 1, and the sum of d and t is equal to K;

[0140] A first feature vector of each of the M first feature points is generated according to the first feature element and the first descriptor, where M is equal to the product of w and h.

[0141] In one or more embodiments, a method for constructing a first feature vector is described. As can be seen from the aforementioned embodiments, a feature extraction network is used to extract features from the entire first image to be matched, thereby obtaining K first feature maps. Of the K first feature maps, d first feature maps constitute the first feature sub-image of the first image to be matched. After removing d first feature maps from the K first feature maps, the remaining t first feature maps constitute the first descriptor of the first image to be matched.

[0142] It can be understood that the first feature sub-item is used to describe the first semantic feature of the first image to be matched, and the first descriptor is used to describe the first physical description feature (for example, spatial feature, rotation attribute, color attribute, etc.) of the first feature sub-item.

[0143] For ease of understanding, please refer to Figure 7, which is a schematic diagram of constructing a feature vector based on a feature map in an embodiment of the present application. As shown in the figure, it is assumed that nine first feature maps are generated based on the first image to be matched, where (A) to (F) in Figure 7 are first feature sub-maps. Figures (G) to (I) in Figure 7 are first descriptors.

[0144] The size of the first feature sub-element is (w×h×d), that is, the first feature sub-element can be expressed as Wherein, w represents the width of the first feature map, h represents the height of the first feature map, and d represents the depth information. Taking FIG7 as an example, the size of the first feature sub-image is (5×5×6).

[0145] The size of the first descriptor is (w×h×t), that is, the first descriptor can be expressed as Wherein, w represents the width of the first feature map, h represents the height of the first feature map, and t represents the number of types of the first physical description feature (i.e., the descriptive information of the first feature sub-image). Taking Figure 7 as an example, the size of the first feature sub-image is (5×5×3). For example, the first feature map shown in (G) of Figure 7 is used to describe the spatial features of the first feature sub-image, the first feature map shown in (H) of Figure 7 is used to describe the rotation properties of the first feature sub-image, and the first feature map shown in (I) of Figure 7 is used to describe the color properties of the first feature sub-image. After obtaining the first feature sub-image and the first descriptor, the elements at the same position can be fused for subsequent feature matching.

[0146] For example, in the first method, the first feature sub-element and the first descriptor may be directly spliced ​​in the depth direction, that is:

[0147] in, represents the M first eigenvectors. Represents the first feature. Represents the first descriptor. Indicates that the two feature maps are spliced ​​in the depth direction. That is, The dimensions are then w×h×(d+t).

[0148] Taking Figure 7 as an example, the first eigenvector corresponding to the first feature point at the top left corner is (0.8, 0.1, 0.9, 0.4, 0.2, 0.7, 0.3, 0.4, 0.6). Similarly, the first eigenvectors corresponding to the 25 first feature points can be obtained.

[0149] For example, in the second method, the first feature sub-element and the first descriptor may be directly concatenated in the depth direction and a convolution operation may be performed, that is:

[0150] in, represents the M first eigenvectors. Represents the first feature. Represents the first descriptor. Indicates that the two feature maps are spliced ​​in the depth direction. Indicates the convolution operation on the concatenated feature vector.

[0151] Secondly, in an embodiment of the present application, a method for constructing a first feature vector is provided. Through the above method, the features and descriptors of the first image to be matched are integrated when constructing the first feature vector. Therefore, the first feature vector contains both the semantic information of the image and the key point features of the image and the relative positional relationship information between the key points. This can improve the ability to understand the entire image and help improve the accuracy of image matching.

[0152] On the basis of one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, obtaining a second feature vector for each of the N second feature points according to the K second feature maps specifically includes:

[0153] Generate a second feature sub-son and a second descriptor of the second image to be matched according to the K second feature maps, wherein the second feature sub-son is used to describe the second semantic feature of the second image to be matched, and the second descriptor is used to describe the second physical description feature of the second feature sub-son. The size of the second feature sub-son is (W×H×d), and the size of the second descriptor is (W×H×t). W represents the width of the second feature map, H represents the height of the second feature map, d represents depth information, and t represents the number of types of the second physical description feature. W, H, d, and t are all integers greater than 1, and the sum of d and t is equal to K.

[0154] A second feature vector of each of the N second feature points is generated according to the second feature sub-item and the second descriptor, where N is equal to the product of W and H.

[0155] In one or more embodiments, a method for constructing a second feature vector is described. As can be seen from the aforementioned embodiments, a feature extraction network is used to extract features from the entire second image to be matched, thereby obtaining K second feature maps. Of the K second feature maps, d second feature maps constitute the second feature sub-image of the second image to be matched. After removing d second feature maps from the K second feature maps, the remaining t second feature maps constitute the second descriptor of the second image to be matched.

[0156] It can be understood that the second feature sub-item is used to describe the second semantic feature of the second image to be matched, and the second descriptor is used to describe the second physical description feature (for example, spatial feature, rotation attribute, color attribute, etc.) of the second feature sub-item.

[0157] For easier understanding, please refer to Figure 7 again. As shown in the figure, assume that nine second feature maps are generated based on the second image to be matched, where (A) to (F) in Figure 7 are second feature sub-maps. (G) to (I) in Figure 7 are second descriptors.

[0158] The size of the second feature sub-element is (W×H×d), that is, the second feature sub-element can be expressed as Where W represents the width of the second feature map, H represents the height of the second feature map, and d represents the depth information. Taking Figure 7 as an example, the size of the second feature sub-image is (5×5×6).

[0159] The size of the second descriptor is (W×H×t), that is, the second descriptor can be expressed as 4=F W×H×t . Wherein, w represents the width of the second feature map, h represents the height of the second feature map, and t represents the number of types of the second physical description feature (i.e., the descriptive information of the second feature sub-image). Taking Figure 7 as an example, the size of the second feature sub-image is (5×5×3). For example, the second feature map shown in (G) of Figure 7 is used to describe the spatial features of the second feature sub-image, the second feature map shown in (H) of Figure 7 is used to describe the rotation properties of the second feature sub-image, and the second feature map shown in (I) of Figure 7 is used to describe the color properties of the second feature sub-image. After obtaining the second feature sub-image and the second descriptor, the elements at the same position can be fused for subsequent feature matching.

[0160] For example, in the first approach, the second feature element and the second descriptor may be directly concatenated in the depth direction, that is:

[0161] in, represents the N second eigenvectors. Represents the second feature. Represents the second descriptor. Indicates that the two feature maps are spliced ​​in the depth direction. That is, The dimensions are then W×H×(d+t).

[0162] For example, in the second method, the second feature sub-element and the second descriptor can be directly spliced ​​in the depth direction and a convolution operation can be performed, that is:

[0163] in, represents the N second eigenvectors. Represents the second feature. Represents the second descriptor. Indicates that the two feature maps are spliced ​​in the depth direction. Indicates the convolution operation on the concatenated feature vector.

[0164] Secondly, in an embodiment of the present application, a method for constructing a second feature vector is provided. Through the above method, the feature sub-vectors and descriptors of the second image to be matched are integrated when constructing the second feature vector. Therefore, the second feature vector contains both the semantic information of the image and the key point features of the image and the relative positional relationship information between the key points. This can improve the ability to understand the entire image and help improve the accuracy of image matching.

[0165] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, determining the number of feature point pairs based on the first feature vector of each first feature point and the second feature vector of each second feature point specifically includes:

[0166] Matching the first feature vector of each of the M first feature points with the second feature vector of each of the N second feature points to obtain a successfully matched feature point pair, wherein a feature point pair includes a first feature point and a second feature point;

[0167] The number of feature point pairs is determined based on the successfully matched feature point pairs.

[0168] In one or more embodiments, a method for determining the number of feature point pairs based on the full set of feature points is described. As can be seen from the aforementioned embodiments, feature point extraction is performed on the first image to be matched, obtaining M first feature points. Feature point extraction is performed on the second image to be matched, obtaining N second feature points. Thus, the M first feature points can be directly matched with the N second feature points.

[0169] For ease of understanding, please refer to Figure 8, which is a schematic diagram of feature point matching between images in an embodiment of the present application. Assume that the image shown in Figure 8 (A) is the first image to be matched, wherein each small square represents a first feature point. That is, it includes 96 feature points. In this case, M=96. Assume that the image shown in Figure 8 (B) is the second image to be matched, wherein each small square represents a second feature point. That is, it includes 96 feature points. In this case, N=96. Match the first feature vector of each first feature point in the M first feature points with the second feature vector of each second feature point in the N second feature points to obtain 9216 feature point pairs. Thus, find the successfully matched feature point pairs from the 9216 feature point pairs. Assuming that 2000 feature point pairs are successfully matched, the number of feature point pairings is 2000.

[0170] It should be noted that, in order to improve matching efficiency, the matching range can also be narrowed. For example, the first feature point in the upper left corner is matched with the second feature point in the upper left corner.

[0171] Secondly, in an embodiment of the present application, a method for determining the number of feature point pairs based on the total number of feature points is provided. By using this method, each feature point involved in the two images to be matched can be matched pairwise, thereby exhaustively enumerating all possible feature point pairs that have a matching relationship, thereby improving the accuracy of feature point matching.

[0172] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, determining the number of feature point pairs based on the first feature vector of each first feature point and the second feature vector of each second feature point specifically includes:

[0173] According to the first feature vector of each first feature point, A first feature points to be matched are obtained from the M first feature points, where A is an integer greater than or equal to 1 and less than or equal to M;

[0174] According to the second feature vector of each second feature point, B second feature points to be matched are obtained from the N second feature points, where B is an integer greater than or equal to 1 and less than or equal to N;

[0175] Matching the first feature vector of each first feature point in the A first feature points with the second feature vector of each second feature point in the B second feature points to obtain a successfully matched feature point pair, wherein a feature point pair includes a first feature point and a second feature point;

[0176] The number of feature point pairs is determined based on the successfully matched feature point pairs.

[0177] In one or more embodiments, a method for determining the number of feature point pairs based on partial feature points is described. As can be seen from the aforementioned embodiments, feature point extraction is performed on the first image to be matched to obtain M first feature points. Based on the first feature vector of each first feature point, A first feature points for matching are selected from the M first feature points. Similarly, feature point extraction is performed on the second image to be matched to obtain N second feature points. Based on the second feature vector of each second feature point, B second feature points for matching are selected from the N second feature points.

[0178] For ease of understanding, please refer to Figure 9, which is another schematic diagram of feature point matching between images in an embodiment of the present application. Assume that the image shown in Figure (A) in Figure 9 is the first image to be matched, wherein the black dots are A first feature points selected from M first feature points. In this case, A=22. Assume that the image shown in Figure (B) in Figure 9 is the second image to be matched, wherein the black dots are B second feature points selected from N second feature points. In this case, B=18. Match the first feature vector of each first feature point in the A first feature points with the second feature vector of each second feature point in the B second feature points to obtain 396 feature point pairs. Thus, find the feature point pairs that are successfully matched from the 396 feature point pairs. Assuming that 18 feature point pairs are successfully matched, the number of feature point pairings is 18.

[0179] It should be noted that, in order to improve matching efficiency, the matching range can also be narrowed. For example, the first feature point in the upper left corner is matched with the second feature point in the upper left corner.

[0180] Secondly, an embodiment of the present application provides a method for determining the number of feature point pairs based on partial feature points. By using this method, partial feature points are selected from the two images to be matched, thereby reducing the number of feature point matches, thereby reducing data processing complexity, saving matching resources, and improving matching efficiency.

[0181] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, obtaining A first feature points to be matched from M first feature points according to the first feature vector of each first feature point specifically includes:

[0182] For each first feature point among the M first feature points, if each first element in the first feature vector of the first feature point is greater than or equal to a first threshold, the first feature point is determined as a first feature point to be matched;

[0183] Obtaining B second feature points to be matched from the N second feature points according to the second feature vector of each second feature point specifically includes:

[0184] For each second feature point among the N second feature points, if each second element in the second feature vector of the second feature point is greater than or equal to the first threshold, the second feature point is determined as the second feature point to be matched.

[0185] In one or more embodiments, a method for selecting feature points is introduced. As can be seen from the above embodiments, since each feature point has a corresponding feature vector, the feature vector can be used to determine and select the corresponding feature points.

[0186] Taking the first eigenvector corresponding to a certain first feature point as an example, assuming that the first eigenvector is represented as (0.8, 0.1, 0.9, 0.4, 0.2, 0.7, 0.3, 0.4, 0.6). Based on this, each first element in the first eigenvector is judged to be greater than or equal to the first threshold. Assuming that the first threshold is 0.5, it can be seen that the five first elements "0.1", "0.4", "0.2", "0.3" and "0.4" included in the first eigenvector do not meet the requirements. Therefore, the first feature point needs to be eliminated. Assuming that the first eigenvector of a certain first feature point is represented as (0.8, 0.9, 0.9, 0.6, 0.6, 0.8, 0.5, 0.9, 1.0). It can be seen that each first element included in the first eigenvector meets the requirements. Therefore, the first feature point is used as the first feature point for subsequent matching.

[0187] It should be noted that similar processing is performed on the first eigenvectors corresponding to other first feature points and the second eigenvectors corresponding to each second feature point, which will not be described in detail here.

[0188] Again, in an embodiment of the present application, a method for filtering feature points is provided. By using the above method, feature points with weaker semantic expression are filtered out based on the elements of the feature vector. This reduces the amount of data required for feature point matching, thereby improving matching efficiency and saving matching resources.

[0189] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, obtaining A first feature points to be matched from M first feature points according to the first feature vector of each first feature point specifically includes:

[0190] For each of the M first feature points, calculate an element average of the first feature point according to the first eigenvector of the first feature point;

[0191] For each first feature point among the M first feature points, if the element average value of the first feature point is greater than or equal to the second threshold, the first feature point is determined as the first feature point to be matched;

[0192] Obtaining B second feature points to be matched from the N second feature points according to the second feature vector of each second feature point specifically includes:

[0193] For each second feature point of the N second feature points, calculate an element average value of the second feature point according to the second eigenvector of the second feature point;

[0194] For each second feature point among the N second feature points, if the element average value of the second feature point is greater than or equal to the second threshold, the second feature point is determined as the second feature point to be matched.

[0195] In one or more embodiments, another method for selecting feature points is introduced. As can be seen from the above embodiments, since each feature point has a corresponding feature vector, the feature vector can be used to determine and select the corresponding feature points.

[0196] Taking the first eigenvector corresponding to a first feature point as an example, assume that the first eigenvector is represented as (0.8, 0.1, 0.9, 0.4, 0.2, 0.7, 0.3, 0.4, 0.6). Based on this, the element average of the first eigenvector is calculated, and the element average of the first feature point is 0.49. Assuming the second threshold is 0.4, it can be seen that the element average of the first feature point is greater than the second threshold. Therefore, the first feature point can be used as the first feature point for subsequent matching. Conversely, if the element average of the first feature point is less than the second threshold, the first feature point needs to be eliminated.

[0197] It should be noted that similar processing is performed on the first eigenvectors of other first feature points and the second eigenvectors of each second feature point, which will not be described in detail here.

[0198] Again, in the embodiments of the present application, another method for filtering feature points is provided. Through the above method, some feature points with weaker semantic expression effects are filtered out based on the average value of the elements of the feature vector, thereby reducing the amount of data for feature point matching, thereby improving matching efficiency and saving matching resources.

[0199] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, obtaining A first feature points to be matched from M first feature points according to the first feature vector of each first feature point specifically includes:

[0200] For each first feature point of the M first feature points, calculate the number of elements of the first feature point according to the first feature vector of the first feature point, where the number of elements of the first feature point is the number of first elements in the first feature vector that are greater than or equal to the element threshold;

[0201] For each first feature point among the M first feature points, if the number of elements of the first feature point is greater than or equal to a third threshold, determine the first feature point as a first feature point to be matched;

[0202] Obtaining B second feature points to be matched from the N second feature points according to the second feature vector of each second feature point specifically includes:

[0203] For each second feature point of the N second feature points, calculate the number of elements of the second feature point according to the second feature vector of the second feature point, where the number of elements of the second feature point is the number of second elements in the second feature vector that are greater than or equal to the element threshold;

[0204] For each second feature point among the N second feature points, if the number of elements of the second feature point is greater than or equal to a third threshold, the second feature point is determined as a second feature point to be matched.

[0205] In one or more embodiments, another method for selecting feature points is introduced. As can be seen from the above embodiments, since each feature point has a corresponding feature vector, the feature vector can be used to determine and select the corresponding feature points.

[0206] Taking the first eigenvector corresponding to a first feature point as an example, assume that the first eigenvector is expressed as (0.8, 0.1, 0.9, 0.4, 0.2, 0.7, 0.3, 0.4, 0.6). Based on this, count the number of first elements in the first eigenvector that are greater than or equal to the element threshold. Assuming the element threshold is 0.5, it can be seen that 4 first elements in the first eigenvector are greater than the element threshold, that is, the number of elements of the first feature point is 4. Assuming the third threshold is 6, then the number of elements of the first feature point is less than the third threshold, so the first feature point needs to be eliminated. Conversely, if the number of elements of the first feature point is greater than or equal to the third threshold, the first feature point is used as the first feature point for subsequent matching.

[0207] It should be noted that similar processing is performed on the first eigenvectors of other first feature points and the second eigenvectors of each second feature point, which will not be described in detail here.

[0208] Again, in the embodiments of the present application, another method for filtering feature points is provided. By using the above method, feature points with weaker semantic expression are filtered out based on the element statistics of the feature vector. This reduces the amount of data required for feature point matching, thereby improving matching efficiency and saving matching resources.

[0209] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, a first feature vector of each first feature point in A first feature points is matched with a second feature vector of each second feature point in B second feature points to obtain a successfully matched feature point pair, specifically including:

[0210] For each first feature point in the A first feature points, calculate the distance between the first feature point and each second feature point in the B second feature points based on the first eigenvector of the first feature point and the second eigenvector of each second feature point in the B second feature points;

[0211] For each of the A first feature points, obtain a second feature point corresponding to the nearest neighbor distance and a second feature point corresponding to the next nearest neighbor distance;

[0212] For each of the A first feature points, the ratio of the nearest neighbor distance to the next nearest neighbor distance is used as the nearest neighbor distance ratio;

[0213] For each first feature point among the A first feature points, if the nearest neighbor distance ratio is less than or equal to the distance ratio threshold, the second feature point and the first feature point corresponding to the nearest neighbor distance are determined as a set of successfully matched feature point pairs.

[0214] In one or more embodiments, a method for performing feature point matching is described. As can be seen from the aforementioned embodiments, a K-nearest neighbor (KNN) algorithm can be used for feature point matching. By finding the closest feature points in the feature space as a matching relationship, a matching result for the feature points corresponding to the two images is obtained. The following diagrams illustrate the feature point matching process.

[0215] For example, for ease of understanding, please refer to Figure 10, which is a schematic diagram of feature point matching based on K nearest neighbors in an embodiment of the present application. As shown in Figure 10 (A), taking the first feature point a1 as an example, first, the distance between the first feature point a1 and B second feature points is calculated respectively. Generally, the smaller the distance between two feature vectors, the closer the two feature points corresponding to the two feature vectors are. Then, based on the distance between the first feature point a1 and each of the other second feature points, the second feature point corresponding to the nearest neighbor distance (i.e., the second feature point b1) and the second feature point corresponding to the next nearest neighbor distance (i.e., the second feature point c1) are found.

[0216] Based on this, the nearest neighbor distance ratio is calculated as follows:

[0217] LR=D1 / D2;Formula (5)

[0218] Where LR represents the nearest neighbor distance ratio. D1 represents the nearest neighbor distance, i.e., the distance between the first feature point a1 and the second feature point b1. D2 represents the next nearest neighbor distance, i.e., the distance between the first feature point a1 and the second feature point c1.

[0219] If the nearest neighbor distance ratio is less than or equal to the distance ratio threshold, the first feature point a1 and the second feature point b1 are successfully matched. That is, the first feature point a1 and the second feature point b1 form a successfully matched feature point pair. The distance ratio threshold can be set to 0.5 or other parameters and is not limited here.

[0220] As shown in Figure 10 (B), taking the first feature point a2 as an example, first, the distances between the first feature point a2 and B second feature points are calculated. Then, based on the distances between the first feature point a2 and each of the other second feature points, the second feature point corresponding to the nearest neighbor distance (i.e., the second feature point b2) and the second feature point corresponding to the next nearest neighbor distance (i.e., the second feature point c2) are found. Based on formula (5), it can be seen that at this time, D1 represents the distance between the first feature point a2 and the second feature point b2, and D2 represents the distance between the first feature point a2 and the second feature point c2. If the nearest neighbor distance ratio is greater than the distance ratio threshold, it means that the first feature point a2 has not been matched to the second feature point.

[0221] It should be noted that the present application may also use other methods to match feature points in two images, for example, using an oriented FAST and rotated BRIEF (ORB) algorithm, or a fast library for approximate nearest neighbors (FLANN) algorithm.

[0222] Again, in an embodiment of the present application, a method for performing feature point matching is provided. Using the KNN algorithm for feature point matching in this manner is simple and effective. Furthermore, it is suitable for automatic matching with large sample sizes and offers high matching accuracy.

[0223] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, a first feature vector of each first feature point in A first feature points is matched with a second feature vector of each second feature point in B second feature points to obtain a successfully matched feature point pair, specifically including:

[0224] For each first feature point in the A first feature points, calculate the distance between the first feature point and each second feature point in the B second feature points based on the first eigenvector of the first feature point and the second eigenvector of each second feature point in the B second feature points;

[0225] For each of the A first feature points, if there is at least one distance less than or equal to the distance threshold, the second feature point corresponding to the minimum distance among the at least one distance and the first feature point are determined as a set of successfully matched feature point pairs.

[0226] In one or more embodiments, another method for feature point matching is described. As can be seen from the aforementioned embodiments, the distance between the first feature point and the second feature point can be calculated based on the first feature vector of the first feature point and the second feature vector of the second feature point. A smaller distance indicates a closer match between the feature points, i.e., a higher degree of match.

[0227] Taking any first feature point and any second feature point as an example, the Euclidean distance between the first feature point and the second feature point can be calculated in the following manner:

[0228] Where d represents the Euclidean distance between the first feature point and the second feature point. K represents the dimension of the feature vector. i Represents the first element of the first eigenvector. i represents the second element of the i-th second eigenvector.

[0229] Based on this, formula (6) can be used to calculate the distance between a first feature point and each second feature point. If these distances are all greater than the distance threshold, it means that the first feature point has no second feature point matching it. If there is only one second feature point with a distance less than or equal to the distance threshold from the first feature point, then the first feature point and the second feature point are directly regarded as a set of feature point pairs that have been successfully matched. If there are at least two second feature points with a distance less than or equal to the distance threshold from the first feature point, it is necessary to first determine the second feature point corresponding to the minimum distance, and then the first feature point and the second feature point are directly regarded as a set of feature point pairs that have been successfully matched.

[0230] It is understood that the above embodiment is introduced by calculating the Euclidean distance as an example. In actual applications, other types of distances between other feature points can also be used, such as Manhattan distance, Chebyshev distance, cosine distance, etc., which are not exhaustive here.

[0231] Again, in the embodiment of the present application, another method for matching feature points is provided. In this way, the similarity distance between feature vectors is used as the basis for determining whether two feature points match, thereby increasing the feasibility and operability of the solution.

[0232] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the present application, determining an image matching result between the first image to be matched and the second image to be matched based on the number of feature point pairs specifically includes:

[0233] Obtaining a maximum number of feature points participating in feature point matching based on the M first feature points and the N second feature points, wherein the maximum number of feature points is the maximum value of the number of first feature points participating in matching and the number of second feature points participating in matching;

[0234] Get the ratio of the number of feature point pairs to the maximum number of feature points;

[0235] If the quantity ratio is greater than the ratio threshold, determining that the image matching result between the first image to be matched and the second image to be matched is a successful image matching;

[0236] If the quantity ratio is less than or equal to the ratio threshold, it is determined that the image matching result between the first image to be matched and the second image to be matched is an image matching failure.

[0237] In one or more embodiments, a method for determining image matching results is introduced. As can be seen from the above embodiments, after obtaining the number of feature point pairs, it is possible to further determine whether the two images are successfully matched based on the number of first feature points and the number of second feature points.

[0238] 1. Based on full feature point matching;

[0239] M first feature points are extracted from the first image to be matched, and N second feature points are extracted from the second image to be matched. Therefore, based on the M first feature points and the N second feature points, the maximum number of feature points that can be used for feature point matching is determined. That is, if M is greater than N, the maximum number of feature points is M. If N is greater than M, the maximum number of feature points is N.

[0240] Based on this, the quantity ratio can be calculated as follows:

[0241] Where C represents the number of feature point pairs. max(M,N) represents the maximum number of feature points. M represents the number of first feature points. N represents the number of second feature points. Threshold represents a ratio threshold value, which can be set according to actual needs. For example, the ratio threshold value can be set to 0.8, which is not limited in the present embodiment.

[0242] 2. Matching based on the filtered feature points;

[0243] Based on the first image to be matched, M first feature points are extracted, and A first feature points to be matched are obtained from the M first feature points. Based on the second image to be matched, N second feature points are extracted, and B second feature points to be matched are obtained from the N second feature points. Thus, based on the A first feature points and the B second feature points, the maximum number of feature points that can be matched is determined. That is, if A is greater than B, the maximum number of feature points is A. If B is greater than A, the maximum number of feature points is B.

[0244] Based on this, the quantity ratio can be calculated as follows:

[0245] Where C represents the number of feature point pairs. max(A,B) represents the maximum number of feature points. A represents the number of first feature points. B represents the number of second feature points. Indicates a numerical ratio. Threshold indicates the ratio threshold. For example, the ratio threshold can be set to 0.8.

[0246] If the number ratio is greater than the ratio threshold, it means that the first image to be matched and the second image to be matched are successfully matched, and image-to-image difference can be performed. Conversely, if the number ratio is less than or equal to the ratio threshold, it means that the first image to be matched and the second image to be matched have failed to match, and image-to-image difference cannot be performed.

[0247] Secondly, embodiments of the present application provide a method for determining image matching results. Using this method, a sufficient number of feature point matches is determined based on the ratio between the number of feature point pairs and the maximum number of feature points. This allows for generating an image matching result, thereby improving the reliability of image matching.

[0248] The following describes the method for updating map information in the present application. Referring to FIG. 11 , the method for updating map information in the embodiment of the present application can be completed independently by the server, independently by the terminal, or in cooperation with the terminal and the server. The method of the present application includes:

[0249] 310. Perform feature extraction on the historical road image to obtain K first feature maps, where the historical road image has M first feature points, each first feature map includes the M first feature points, K is an integer greater than or equal to 1, and M is an integer greater than 1;

[0250] In one or more embodiments, a historical road image is obtained. It is understood that the historical road image is an image obtained by taking a photo of the road ahead by an onboard camera, or a road image uploaded by a user through a terminal.

[0251] In the embodiment of the present application, the historical road image is detected to obtain M first feature points of the historical road image.

[0252] In one possible implementation, a feature extraction network can be used to extract features from historical road images, thereby obtaining K first feature maps. The feature extraction network uses K kernels for feature extraction, with each kernel extracting features for a channel. Thus, K first feature maps for each channel are obtained, each of which has the same size.

[0253] Step 310 in this embodiment is similar to step 210 in the embodiment shown in FIG. 3 , and will not be described in detail here.

[0254] 320. Perform feature extraction on the road image to be processed to obtain K second feature maps, wherein the road image to be processed is collected later than the historical road images, the road image to be processed has N second feature points, and each second feature map includes the N second feature points, where N is an integer greater than 1.

[0255] In one or more embodiments, the road image to be processed is obtained by photographing the road ahead using an onboard camera, or by a user uploading a road image via a terminal. The road image to be processed is acquired later than the historical road image, and typically, the road image to be processed and the historical road image are acquired from the same or similar locations (e.g., the same street or parking lot). Both the road image to be processed and the historical road image are black and white images, or both are RGB images.

[0256] In one possible implementation, a feature extraction network may be used to perform feature extraction on the road image to be processed, thereby obtaining K second feature maps, each of which has the same size and includes the N second feature points detected above.

[0257] Step 320 in this embodiment is similar to step 220 in the embodiment shown in FIG. 3 , and will not be described in detail here.

[0258] 330. Obtain a first feature vector for each of the M first feature points based on the K first feature maps, wherein the first feature vector includes K first elements, each first element being derived from a different first feature map. The M first feature vectors corresponding to the historical road image are used to describe a first semantic feature and a first physical description feature of the historical road image.

[0259] In one or more embodiments, step 330 is similar to step 230 in the embodiment shown in FIG. 3 , wherein the M first feature vectors may be used to describe the first semantic features and the first physical description features of the historical road image, which will not be described in detail here.

[0260] 340. Obtain a second feature vector for each of the N second feature points based on the K second feature maps, wherein the second feature vector includes K second elements, each second element is derived from a different second feature map, and the N second feature vectors corresponding to the road image to be processed are used to describe the second semantic feature and the second physical description feature of the road image to be processed;

[0261] In one or more embodiments, step 340 is similar to step 240 in the embodiment shown in FIG. 3 , wherein the N second feature vectors may be used to describe the second semantic features and the second physical description features of the road image to be processed, which will not be described in detail here.

[0262] 350. Determine the number of feature point pairs based on the first feature vector of each first feature point and the second feature vector of each second feature point, wherein the number of feature point pairs represents the number of successful matches between the first feature point and the second feature point;

[0263] In one or more embodiments, step 350 is similar to step 250 in the embodiment shown in FIG. 3 , and the details are not repeated here.

[0264] 360. When it is determined based on the number of feature point pairs that the historical road image and the road image to be processed fail to match, generating an image element set based on the element recognition results of the historical road image and the element recognition results of the road image to be processed, wherein the image element set is derived from at least one of the historical road image and the road image to be processed;

[0265] In one or more embodiments, a ratio between the number of feature point pairs and the maximum number of feature points participating in the feature point matching is used to determine whether the ratio is greater than a ratio threshold. If so, the image matching result between the first image to be matched and the second image to be matched is determined to be a successful image match. Otherwise, the match fails.

[0266] 370. Update the map information based on the image feature set.

[0267] In one or more embodiments, after obtaining an image element set, a determination is made as to whether the map information needs to be updated based on the category information corresponding to the elements included in the image element set. If the category information corresponding to the elements in the image element set is updateable, the map information can be updated. Updateable category information includes, but is not limited to, road signs, indicator lights, and electronic cameras.

[0268] For ease of understanding, please refer to Figure 12, which is a schematic diagram of global scene understanding in an embodiment of the present application. As shown in the figure, global feature extraction is performed on historical road images and road images to be processed. Taking the feature extraction of historical road images as an example, first, the historical road image is input into the feature extraction network, and the feature extraction network outputs K first feature maps, which are represented as F w×h×K . Where w represents the width of the first feature map, h represents the height of the first feature map, and K represents the number of first feature maps. Further, D K Represents a single first feature map. d ij Represents the first physical description feature corresponding to the feature point in the i-th row and j-th column.

[0269] Based on this, each first feature point in the historical road image and each second feature point in the road image to be processed can be obtained. Then, the global features corresponding to each first feature point (i.e., first feature vector) are matched with the global features corresponding to each second feature point (i.e., second feature vector). The global feature matching method can be a KNN algorithm, an ORB algorithm, or a FLANN algorithm, etc., which is not limited here.

[0270] After generating feature point matching results between the historical road image and the road image to be processed, the need for map updating based on distinguishing features can be determined based on the category and location information of each feature obtained by the soft detection module.

[0271] In the soft detection module, taking K first feature maps as an example, the feature point in the i-th row and j-th column of the R-th first feature map is represented as Based on the ratio-to-max method, the first feature point with the highest confidence in each channel is found from the K first feature maps. Soft non-maximum suppression (soft-NMS) is then used to find the first feature point with the highest confidence among each first feature point. This generates a confidence score for each first feature point, thereby obtaining the category and location information of each feature in the historical road image.

[0272] It should be noted that in practical applications, in addition to using Soft NMS for target detection, non-maximum suppression (NMS), distance intersection over union NMS (DIOU NMS), or weighted non-maximum suppression (weighted NMS) can also be used for target detection, which is not limited here.

[0273] In an embodiment of the present application, a method for updating map information is provided. Through the above-mentioned method, the deep features of the two images are extracted respectively to obtain the feature vectors of each feature point in each image. These feature vectors can represent the semantic features and physical description features of the image, and therefore, the image information can be learned more comprehensively. Based on this, the use of feature vectors to achieve matching of feature points can improve the ability to understand the entire image, which is conducive to improving the accuracy of image matching. Then, the change points can be discovered and updated based on image matching, thereby improving the ability to update map information and solving the problem of map update errors caused by errors in matching new and old data during map information updates.

[0274] Based on one or more embodiments corresponding to FIG. 11 above, another optional embodiment provided by the embodiment of the present application may further include:

[0275] Performing target recognition on the historical road image to obtain an element recognition result of the historical road image, wherein the element recognition result of the historical road image includes category information and location information corresponding to at least one element;

[0276] Performing target recognition on the road image to be processed to obtain an element recognition result of the road image to be processed, wherein the element recognition result of the road image to be processed includes category information and position information corresponding to at least one element;

[0277] Based on the feature recognition results of the historical road image and the feature recognition results of the road image to be processed, an image feature set is generated, specifically including:

[0278] Determining a second feature point set for which matching fails from the road image to be processed, wherein the second feature point set includes at least one second feature point;

[0279] Determining a candidate element set based on the second feature point set and an element recognition result of the road image to be processed;

[0280] The candidate feature set is compared with the feature recognition results of the historical road image to determine the image feature set.

[0281] In one or more embodiments, a method for automatically identifying a set of image elements is introduced. As can be seen from the aforementioned embodiments, a feature extraction network is used to extract features of historical road images and road images to be processed. The feature extraction network is part of a target detection model. The target detection model used in this application can be a regional convolutional neural network (RCNN) or a fast region convolutional neural network (faster RCNN). Based on this, the target detection model can detect the feature recognition results of the historical road images and the feature recognition results of the road images to be processed.

[0282] It should be noted that the feature recognition result includes feature category information (eg, license plate, electronic eye, traffic sign, etc.) and location information, wherein the location information can be represented as a bounding box (BBOX).

[0283] For ease of understanding, please refer to Figure 13, which is a schematic diagram showing a set of image elements in an embodiment of the present application. Figure 13 (A) shows a historical road image, B1 is used to indicate the location information of element A, and the category information of element A is "tree". B2 is used to indicate the location information of element B, and the category information of element B is "car". B3 is used to indicate the location information of element C, and the category information of element C is "tree". Figure 13 (B) shows a road image to be processed, C1 is used to indicate the location information of element X, and the category information of element X is "tree". C2 is used to indicate the location information of element Y, and the category information of element Y is "tree".

[0284] Figure 13 (C) shows the first feature points involved in the matching of the historical road image, and Figure 13 (D) shows the second feature points involved in the matching of the road image to be processed. The matching results show that some second feature points in the road image to be processed failed to match, resulting in a set of second feature points that failed to match.

[0285] Based on the positions corresponding to the respective second feature points in the second feature point set and the feature recognition results of the road image to be processed, unmatched features can be determined. Taking Figure 13 as an example, the unmatched features include the feature indicated by B2. Therefore, it is determined that the image feature set includes the feature indicated by B2.

[0286] Secondly, embodiments of this application provide a method for automatically identifying sets of image elements. This method, utilizing feature point matching and object detection algorithms, can automatically identify mismatched image elements in two images, thereby enabling map updates based on these image elements. This reduces map update costs and achieves the goal of automated detection.

[0287] The image matching device in the present application is described in detail below. Please refer to FIG14 , which is a schematic diagram of an embodiment of the image matching device in the present application. The image matching device 40 includes:

[0288] A processing module 410 is configured to perform feature extraction processing on the first image to be matched to obtain K first feature maps, wherein the first image to be matched has M first feature points, each first feature map includes the M first feature points, K is an integer greater than or equal to 1, and M is an integer greater than 1;

[0289] The processing module 410 is further configured to perform feature extraction processing on the second image to be matched to obtain K second feature maps, wherein the second image to be matched has N second feature points, and each second feature map includes the N second feature points, where N is an integer greater than 1;

[0290] An acquisition module 420 is configured to acquire a first feature vector for each of the M first feature points based on the K first feature maps, wherein the first feature vector includes K first elements, each first element being derived from a different first feature map, and the M first feature vectors corresponding to the first image to be matched are used to describe a first semantic feature and a first physical description feature of the first image to be matched;

[0291] The acquisition module 420 is further used to obtain a second feature vector of each second feature point in the N second feature points based on the K second feature maps, wherein the second feature vector includes K second elements, each second element is derived from a different second feature map, and the N second feature vectors corresponding to the second image to be matched are used to describe the second semantic feature and the second physical description feature of the second image to be matched

[0292] a determination module 430 for determining a number of feature point pairs based on the first feature vector of each first feature point and the second feature vector of each second feature point, wherein the number of feature point pairs represents the number of successful matches between the first feature point and the second feature point;

[0293] The determination module 430 is further configured to determine an image matching result between the first image to be matched and the second image to be matched according to the number of feature point pairs.

[0294] Optionally, based on the embodiment corresponding to FIG. 14 , in another embodiment of the image matching device 40 provided in the embodiment of the present application,

[0295] The acquisition module 420 is further configured to acquire a first initial image to be matched and a second initial image to be matched;

[0296] The processing module 410 is further configured to, when the size of the first initial image to be matched is larger than a preset size, reduce the size of the first initial image to be matched to obtain a first image to be matched;

[0297] The processing module 410 is further configured to, if the size of the first initial image to be matched is smaller than a preset size, perform a size enlargement process on the first initial image to be matched to obtain the first image to be matched, or perform an image filling process on the first initial image to be matched to obtain the first image to be matched;

[0298] The processing module 410 is further configured to, when the size of the second initial image to be matched is larger than a preset size, reduce the size of the second initial image to be matched to obtain a second image to be matched;

[0299] The processing module 410 is further configured to enlarge the size of the second initial image to be matched to obtain a second image to be matched, or to fill the second initial image to be matched to obtain a second image to be matched, when the size of the second initial image to be matched is smaller than a preset size.

[0300] Optionally, based on the embodiment corresponding to FIG. 14 , in another embodiment of the image matching device 40 provided in the embodiment of the present application,

[0301] The processing module 410 is specifically configured to obtain K first convolution feature maps based on the first image to be matched through the convolution layer included in the feature extraction network;

[0302] The K first convolution feature maps are respectively normalized by a normalization layer included in the feature extraction network to obtain K first normalized feature maps;

[0303] Performing nonlinear mapping on the K first normalized feature maps through an activation layer included in a feature extraction network to obtain K first feature maps;

[0304] The processing module 410 is specifically configured to obtain K second convolution feature maps based on the second image to be matched through the convolution layer included in the feature extraction network;

[0305] The K second convolution feature maps are respectively normalized by a normalization layer included in the feature extraction network to obtain K second normalized feature maps;

[0306] The K second normalized feature maps are respectively nonlinearly mapped through the activation layer included in the feature extraction network to obtain K second feature maps.

[0307] Optionally, based on the embodiment corresponding to FIG. 14 , in another embodiment of the image matching device 40 provided in the embodiment of the present application,

[0308] The acquisition module 420 is specifically configured to generate a first feature sub-item and a first descriptor of the first image to be matched based on the K first feature maps, wherein the first feature sub-item is used to describe the first semantic feature of the first image to be matched, and the first descriptor is used to describe the first physical description feature of the first feature sub-item. The size of the first feature sub-item is (w×h×d), and the size of the first descriptor is (w×h×t), where w represents the width of the first feature map, h represents the height of the first feature map, d represents depth information, and t represents the number of types of the first physical description feature. w, h, d, and t are all integers greater than 1, and the sum of d and t is equal to K.

[0309] A first feature vector of each of the M first feature points is generated according to the first feature element and the first descriptor, where M is equal to the product of w and h.

[0310] Optionally, based on the embodiment corresponding to FIG. 14 , in another embodiment of the image matching device 40 provided in the embodiment of the present application,

[0311] An acquisition module 420 is specifically configured to generate a second feature sub-item and a second descriptor of a second image to be matched based on the K second feature maps, wherein the second feature sub-item is used to describe a second semantic feature of the second image to be matched, and the second descriptor is used to describe a second physical description feature of the second feature sub-item. The size of the second feature sub-item is (W×H×d), and the size of the second descriptor is (W×H×t), where W represents the width of the second feature map, H represents the height of the second feature map, d represents depth information, and t represents the number of types of the second physical description feature. W, H, d, and t are all integers greater than 1, and the sum of d and t is equal to K.

[0312] A second feature vector of each of the N second feature points is generated according to the second feature sub-item and the second descriptor, where N is equal to the product of W and H.

[0313] Optionally, based on the embodiment corresponding to FIG. 14 , in another embodiment of the image matching device 40 provided in the embodiment of the present application,

[0314] The determining module 430 is specifically configured to match a first feature vector of each of the M first feature points with a second feature vector of each of the N second feature points to obtain a successfully matched feature point pair, wherein a feature point pair includes a first feature point and a second feature point;

[0315] The number of feature point pairs is determined based on the successfully matched feature point pairs.

[0316] Optionally, based on the embodiment corresponding to FIG. 14 , in another embodiment of the image matching device 40 provided in the embodiment of the present application,

[0317] The determination module 430 is specifically configured to obtain A first feature points to be matched from the M first feature points according to the first feature vector of each first feature point, where A is an integer greater than or equal to 1 and less than or equal to M;

[0318] According to the second feature vector of each second feature point, B second feature points to be matched are obtained from the N second feature points, where B is an integer greater than or equal to 1 and less than or equal to N;

[0319] Matching the first feature vector of each first feature point in the A first feature points with the second feature vector of each second feature point in the B second feature points to obtain a successfully matched feature point pair, wherein a feature point pair includes a first feature point and a second feature point;

[0320] The number of feature point pairs is determined based on the successfully matched feature point pairs.

[0321] Optionally, based on the embodiment corresponding to FIG. 14 , in another embodiment of the image matching device 40 provided in the embodiment of the present application,

[0322] The determination module 430 is specifically configured to, for each first feature point among the M first feature points, determine the first feature point as a first feature point to be matched if each first element in the first feature vector of the first feature point is greater than or equal to a first threshold;

[0323] The determination module 430 is specifically configured to determine, for each second feature point among the N second feature points, as a second feature point to be matched if each second element in the second feature vector of the second feature point is greater than or equal to a first threshold.

[0324] Optionally, based on the embodiment corresponding to FIG. 14 , in another embodiment of the image matching device 40 provided in the embodiment of the present application,

[0325] The determining module 430 is specifically configured to calculate, for each of the M first feature points, an element average value of the first feature point according to the first eigenvector of the first feature point;

[0326] For each first feature point among the M first feature points, if the element average value of the first feature point is greater than or equal to the second threshold, the first feature point is determined as the first feature point to be matched;

[0327] The determining module 430 is specifically configured to calculate, for each second feature point among the N second feature points, an element average value of the second feature point according to the second feature vector of the second feature point;

[0328] For each second feature point among the N second feature points, if the element average value of the second feature point is greater than or equal to the second threshold, the second feature point is determined as the second feature point to be matched.

[0329] Optionally, based on the embodiment corresponding to FIG. 14 , in another embodiment of the image matching device 40 provided in the embodiment of the present application,

[0330] The determining module 430 is specifically configured to calculate, for each first feature point among the M first feature points, the number of elements of the first feature point according to the first feature vector of the first feature point, wherein the number of elements of the first feature point is the number of first elements in the first feature vector that are greater than or equal to the element threshold;

[0331] For each first feature point among the M first feature points, if the number of elements of the first feature point is greater than or equal to a third threshold, determine the first feature point as a first feature point to be matched;

[0332] The determining module 430 is specifically configured to calculate, for each second feature point of the N second feature points, the number of elements of the second feature point according to the second feature vector of the second feature point, wherein the number of elements of the second feature point is the number of second elements in the second feature vector that are greater than or equal to the element threshold;

[0333] For each second feature point among the N second feature points, if the number of elements of the second feature point is greater than or equal to a third threshold, the second feature point is determined as a second feature point to be matched.

[0334] Optionally, based on the embodiment corresponding to FIG. 14 , in another embodiment of the image matching device 40 provided in the embodiment of the present application,

[0335] a determining module 430 specifically configured to calculate, for each first feature point among the A first feature points, a distance between the first feature point and each second feature point among the B second feature points based on a first feature vector of the first feature point and a second feature vector of each second feature point among the B second feature points;

[0336] For each of the A first feature points, obtain a second feature point corresponding to the nearest neighbor distance and a second feature point corresponding to the next nearest neighbor distance;

[0337] For each of the A first feature points, the ratio of the nearest neighbor distance to the next nearest neighbor distance is used as the nearest neighbor distance ratio;

[0338] For each first feature point among the A first feature points, if the nearest neighbor distance ratio is less than or equal to the distance ratio threshold, the second feature point and the first feature point corresponding to the nearest neighbor distance are determined as a set of successfully matched feature point pairs.

[0339] Optionally, based on the embodiment corresponding to FIG. 14 , in another embodiment of the image matching device 40 provided in the embodiment of the present application,

[0340] a determining module 430 specifically configured to calculate, for each first feature point among the A first feature points, a distance between the first feature point and each second feature point among the B second feature points based on a first feature vector of the first feature point and a second feature vector of each second feature point among the B second feature points;

[0341] For each first feature point among the A first feature points, if there is at least one distance less than or equal to the distance threshold, the second feature point corresponding to the minimum distance among the at least one distance and the first feature point are determined as a set of successfully matched feature point pairs.

[0342] Optionally, based on the embodiment corresponding to FIG. 14 , in another embodiment of the image matching device 40 provided in the embodiment of the present application,

[0343] The determination module 430 is specifically configured to obtain a maximum number of feature points participating in feature point matching based on the M first feature points and the N second feature points, wherein the maximum number of feature points is the maximum value of the number of first feature points participating in matching and the number of second feature points participating in matching;

[0344] Get the ratio of the number of feature point pairs to the maximum number of feature points;

[0345] If the quantity ratio is greater than the ratio threshold, determining that the image matching result between the first image to be matched and the second image to be matched is a successful image matching;

[0346] If the quantity ratio is less than or equal to the ratio threshold, it is determined that the image matching result between the first image to be matched and the second image to be matched is an image matching failure.

[0347] The map information updating device in the present application is described in detail below. Please refer to FIG15 , which is a schematic diagram of an embodiment of the map information updating device in the present application. The map information updating device 50 includes:

[0348] A processing module 510 is configured to perform feature extraction processing on the historical road image to obtain K first feature maps, wherein the historical road image has M first feature points, each first feature map includes the M first feature points, K is an integer greater than or equal to 1, and M is an integer greater than 1;

[0349] The processing module 510 is further configured to perform feature extraction processing on the road image to be processed to obtain K second feature maps, wherein the road image to be processed is collected later than the historical road images, the road image to be processed has N second feature points, and each second feature map includes the N second feature points, where N is an integer greater than 1;

[0350] An acquisition module 520 is configured to acquire a first feature vector for each of the M first feature points based on the K first feature maps, wherein the first feature vector includes K first elements, each first element being derived from a different first feature map, and the M first feature vectors corresponding to the historical road image are used to describe a first semantic feature and a first physical description feature of the historical road image;

[0351] The acquisition module 520 is further configured to acquire a second feature vector for each of the N second feature points based on the K second feature maps, wherein the second feature vector includes K second elements, each second element is derived from a different second feature map, and the N second feature vectors corresponding to the road image to be processed are used to describe the second semantic feature and the second physical description feature of the road image to be processed;

[0352] a determination module 530 for determining a number of feature point pairs based on the first feature vector of each first feature point and the second feature vector of each second feature point, wherein the number of feature point pairs represents the number of successful matches between the first feature point and the second feature point;

[0353] The generating module 540 is further configured to generate an image element set based on the element recognition results of the historical road image and the element recognition results of the road image to be processed, if it is determined based on the number of feature point pairs that the historical road image and the road image to be processed fail to match, wherein the image element set is derived from at least one of the historical road image and the road image to be processed;

[0354] The updating module 550 is used to update the map information according to the image element set.

[0355] Optionally, based on the embodiment corresponding to FIG. 15 , in another embodiment of the map information updating device 50 provided in the embodiment of the present application, the map information updating device 50 further includes an identification module 560 ;

[0356] The recognition module 560 is configured to perform target recognition on the historical road image to obtain an element recognition result of the historical road image, wherein the element recognition result of the historical road image includes category information and location information corresponding to at least one element;

[0357] The recognition module 560 is further configured to perform target recognition on the road image to be processed to obtain an element recognition result of the road image to be processed, wherein the element recognition result of the road image to be processed includes category information and position information corresponding to at least one element;

[0358] A generating module 540 is specifically configured to determine a second feature point set for which matching fails from the road image to be processed, wherein the second feature point set includes at least one second feature point;

[0359] Determining a candidate element set based on the second feature point set and an element recognition result of the road image to be processed;

[0360] The candidate feature set is compared with the feature recognition results of the historical road image to determine the image feature set.

[0361] Figure 16 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The computer device 600 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 622 (for example, one or more processors) and a memory 632, and one or more storage media 630 (for example, one or more mass storage devices) for storing application programs 642 or data 644. Among them, the memory 632 and the storage medium 630 can be temporary storage or permanent storage. The program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the computer device. Furthermore, the central processing unit 622 can be configured to communicate with the storage medium 630 to execute a series of instruction operations in the storage medium 630 on the computer device 600.

[0362] The computer device 600 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input and output interfaces 658, and / or one or more operating systems 641, such as Windows Server 2008. TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM etc.

[0363] The steps executed by the computer device in the above embodiment may be based on the computer device structure shown in FIG16 .

[0364] A computer-readable storage medium is also provided in an embodiment of the present application, on which a computer program is stored. When the computer program is executed by a processor, the steps of the methods described in the above embodiments are implemented.

[0365] A computer program product is also provided in an embodiment of the present application, including a computer program. When the computer program is executed by a processor, the steps of the methods described in the above embodiments are implemented.

[0366] It is understandable that in the specific implementation of this application, user information, road images and other related data are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0367] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0368] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0369] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0370] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0371] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a server or terminal device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store computer programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0372] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for image matching, the method being performed by a computer device, comprising: Performing feature extraction processing on the first image to be matched to obtain K first feature maps, wherein the first image to be matched has M first feature points, each of the first feature maps includes the M first feature points, K is an integer greater than or equal to 1, and M is an integer greater than 1; Performing feature extraction processing on the second image to be matched to obtain K second feature maps, wherein the second image to be matched has N second feature points, each of the second feature maps includes the N second feature points, and N is an integer greater than 1; According to the K first feature maps, obtaining a first feature vector of each first feature point in the M first feature points, wherein the first feature vector includes K first elements, each first element is respectively derived from a different first feature map, and the M first feature vectors corresponding to the first image to be matched are used to describe a first semantic feature and a first physical description feature of the first image to be matched; According to the K second feature maps, obtaining a second feature vector of each second feature point in the N second feature points, wherein the second feature vector includes K second elements, each second element is respectively derived from a different second feature map, and the N second feature vectors corresponding to the second image to be matched are used to describe the second semantic feature and the second physical description feature of the second image to be matched; Determine the number of feature point pairs according to the first feature vector of each first feature point and the second feature vector of each second feature point, wherein the number of feature point pairs represents the number of successful matches between the first feature point and the second feature point; An image matching result between the first image to be matched and the second image to be matched is determined according to the number of feature point pairs.

2. The method according to claim 1, further comprising: Acquire a first initial image to be matched and a second initial image to be matched; When the size of the first to-be-matched initial image is larger than a preset size, reducing the size of the first to-be-matched initial image to obtain the first to-be-matched image; In the case that the size of the first to-be-matched initial image is smaller than the preset size, performing a size enlargement process on the first to-be-matched initial image to obtain the first to-be-matched image, or performing an image filling process on the first to-be-matched initial image to obtain the first to-be-matched image; When the size of the second initial image to be matched is larger than the preset size, reducing the size of the second initial image to be matched to obtain the second image to be matched; When the size of the second initial image to be matched is smaller than the preset size, the second initial image to be matched is enlarged to obtain the second image to be matched, or the second initial image to be matched is filled to obtain the second image to be matched.

3. According to the method of claim 1 or 2, the step of performing feature extraction processing on the first image to be matched to obtain K first feature maps comprises: Based on the first image to be matched, obtaining K first convolution feature maps through a convolution layer included in a feature extraction network; The K first convolutional feature maps are respectively normalized by a normalization layer included in the feature extraction network to obtain K first normalized feature maps; By using the activation layer included in the feature extraction network, nonlinear mapping is performed on the K first normalized feature maps to obtain the K first feature maps; The step of performing feature extraction processing on the second image to be matched to obtain K second feature maps includes: Based on the second image to be matched, obtaining K second convolution feature maps through the convolution layer included in the feature extraction network; The K second convolutional feature maps are respectively normalized by a normalization layer included in the feature extraction network to obtain K second normalized feature maps; The K second normalized feature maps are respectively nonlinearly mapped through the activation layer included in the feature extraction network to obtain the K second feature maps.

4. The method according to any one of claims 1 to 3, wherein obtaining, based on the K first feature maps, a first feature vector of each of the M first feature points comprises: Generate a first feature sub-son and a first descriptor of the first image to be matched according to the K first feature maps, wherein the first feature sub-son is used to describe a first semantic feature of the first image to be matched, and the first descriptor is used to describe a first physical description feature of the first feature sub-son, and the size of the first feature sub-son is (w×h×d), and the size of the first descriptor is (w×h×t), wherein w represents a width of the first feature map, h represents a height of the first feature map, d represents depth information, and t represents the number of types of the first physical description feature, and w, h, d, and t are all integers greater than 1, and the sum of d and t is equal to K; A first feature vector of each of the M first feature points is generated according to the first feature element and the first descriptor, wherein M is equal to the product of w and h.

5. According to the method according to any one of claims 1 to 4, obtaining the second feature vector of each second feature point in the N second feature points according to the K second feature maps comprises: Generate a second feature sub-son and a second descriptor of the second image to be matched according to the K second feature maps, wherein the second feature sub-son is used to describe the second semantic feature of the second image to be matched, and the second descriptor is used to describe the second physical description feature of the second feature sub-son, the size of the second feature sub-son is (W×H×d), the size of the second descriptor is (W×H×t), the W represents the width of the second feature map, the H represents the height of the second feature map, the d represents the depth information, the t represents the number of types of the second physical description feature, the W, the H, the d and the t are all integers greater than 1, and the sum of the d and the t is equal to the K; A second feature vector of each of the N second feature points is generated according to the second feature sub-signal and the second descriptor, wherein N is equal to the product of W and H.

6. The method according to any one of claims 1 to 5, wherein determining the number of feature point pairs according to the first feature vector of each first feature point and the second feature vector of each second feature point comprises: Matching a first feature vector of each of the M first feature points with a second feature vector of each of the N second feature points to obtain a successfully matched feature point pair, wherein one feature point pair includes one first feature point and one second feature point; The number of feature point pairings is determined according to the successfully matched feature point pairs.

7. The method according to any one of claims 1 to 5, wherein determining the number of feature point pairs according to the first feature vector of each first feature point and the second feature vector of each second feature point comprises: According to the first feature vector of each first feature point, A first feature points to be matched are obtained from the M first feature points, wherein A is an integer greater than or equal to 1 and less than or equal to M; According to the second feature vector of each second feature point, B second feature points to be matched are obtained from the N second feature points, wherein B is an integer greater than or equal to 1 and less than or equal to N; Matching the first feature vector of each of the A first feature points with the second feature vector of each of the B second feature points to obtain a successfully matched feature point pair, wherein one feature point pair includes one first feature point and one second feature point; The number of feature point pairings is determined according to the successfully matched feature point pairs.

8. The method according to claim 7, wherein obtaining A first feature points to be matched from the M first feature points according to the first feature vector of each first feature point comprises: For each first feature point among the M first feature points, if each first element in the first feature vector of the first feature point is greater than or equal to a first threshold, the first feature point is determined as a first feature point to be matched; The acquiring B second feature points to be matched from the N second feature points according to the second feature vector of each second feature point comprises: For each second feature point among the N second feature points, if each second element in the second feature vector of the second feature point is greater than or equal to the first threshold, the second feature point is determined as a second feature point to be matched.

9. The method according to claim 7, wherein obtaining A first feature points to be matched from the M first feature points according to the first feature vector of each first feature point comprises: For each first feature point of the M first feature points, calculating an element average value of the first feature point according to a first feature vector of the first feature point; For each first feature point among the M first feature points, if an element average value of the first feature point is greater than or equal to a second threshold, determining the first feature point as a first feature point to be matched; The acquiring B second feature points to be matched from the N second feature points according to the second feature vector of each second feature point comprises: For each second feature point of the N second feature points, calculating an element average value of the second feature point according to a second feature vector of the second feature point; For each second feature point among the N second feature points, if the element average value of the second feature point is greater than or equal to the second threshold, the second feature point is determined as the second feature point to be matched.

10. The method according to claim 7, wherein obtaining A first feature points to be matched from the M first feature points according to the first feature vector of each first feature point comprises: For each first feature point of the M first feature points, the number of elements of the first feature point is calculated according to the first feature vector of the first feature point, wherein the number of elements of the first feature point is the number of first elements in the first feature vector that are greater than or equal to the element threshold; For each first feature point among the M first feature points, if the number of elements of the first feature point is greater than or equal to a third threshold, determining the first feature point as a first feature point to be matched; The acquiring B second feature points to be matched from the N second feature points according to the second feature vector of each second feature point comprises: For each second feature point of the N second feature points, according to the second feature vector of the second feature point, the number of elements of the second feature point is calculated, wherein the number of elements of the second feature point is the number of second elements in the second feature vector that are greater than or equal to the element threshold; For each second feature point among the N second feature points, if the number of elements of the second feature point is greater than or equal to the third threshold, the second feature point is determined as the second feature point to be matched.

11. The method according to any one of claims 7 to 10, wherein the first feature vector of each of the A first feature points is matched with the second feature vector of each of the B second feature points to obtain a successfully matched feature point pair, comprising: For each first feature point among the A first feature points, calculate a distance between the first feature point and each second feature point among the B second feature points according to a first feature vector of the first feature point and a second feature vector of each second feature point among the B second feature points; For each first feature point among the A first feature points, obtaining a second feature point corresponding to a nearest neighbor distance and a second feature point corresponding to a second nearest neighbor distance; For each first feature point among the A first feature points, taking the ratio between the nearest neighbor distance and the second nearest neighbor distance as the nearest neighbor distance ratio; For each first feature point among the A first feature points, if the nearest neighbor distance ratio is less than or equal to the distance ratio threshold, the second feature point corresponding to the nearest neighbor distance and the first feature point are determined as a set of successfully matched feature point pairs.

12. The method according to any one of claims 7 to 10, wherein the first feature vector of each of the A first feature points is matched with the second feature vector of each of the B second feature points to obtain a successfully matched feature point pair, comprising: For each first feature point among the A first feature points, calculate a distance between the first feature point and each second feature point among the B second feature points according to a first feature vector of the first feature point and a second feature vector of each second feature point among the B second feature points; For each first feature point among the A first feature points, if there is at least one distance less than or equal to the distance threshold, the second feature point corresponding to the minimum distance among the at least one distance and the first feature point are determined as a set of feature point pairs that are successfully matched.

13. The method according to any one of claims 1 to 12, wherein determining the image matching result between the first image to be matched and the second image to be matched according to the number of feature point pairs comprises: According to the M first feature points and the N second feature points, a maximum number of feature points participating in feature point matching is obtained, wherein the maximum number of feature points is a maximum value of the number of first feature points participating in matching and the number of second feature points participating in matching; Obtaining a ratio of the number of feature point pairs to the maximum number of feature points; If the quantity ratio is greater than the ratio threshold, determining that the image matching result between the first image to be matched and the second image to be matched is a successful image matching; If the quantity ratio is less than or equal to the ratio threshold, it is determined that the image matching result between the first image to be matched and the second image to be matched is an image matching failure.

14. A method for updating map information, the method being executed by a computer device, comprising: Performing feature extraction processing on the historical road image to obtain K first feature maps, wherein the historical road image has M first feature points, each of the first feature maps includes the M first feature points, K is an integer greater than or equal to 1, and M is an integer greater than 1; Performing feature extraction processing on the road image to be processed to obtain K second feature maps, wherein the acquisition time of the road image to be processed is later than the acquisition time of the historical road image, the road image to be processed has N second feature points, and each of the second feature maps includes the N second feature points, where N is an integer greater than 1; According to the K first feature maps, obtaining a first feature vector of each first feature point in the M first feature points, wherein the first feature vector includes K first elements, each first element is respectively derived from a different first feature map, and the M first feature vectors corresponding to the historical road image are used to describe a first semantic feature and a first physical description feature of the historical road image; According to the K second feature maps, obtaining a second feature vector of each second feature point in the N second feature points, wherein the second feature vector includes K second elements, each second element is respectively derived from a different second feature map, and the N second feature vectors corresponding to the road image to be processed are used to describe the second semantic feature and the second physical description feature of the road image to be processed; Determine the number of feature point pairs according to the first feature vector of each first feature point and the second feature vector of each second feature point, wherein the number of feature point pairs represents the number of successful matches between the first feature point and the second feature point; When it is determined that the historical road image and the road image to be processed fail to match according to the number of feature point pairs, generating an image element set according to the element recognition result of the historical road image and the element recognition result of the road image to be processed, wherein the image element set is derived from at least one of the historical road image and the road image to be processed; The map information is updated according to the set of image elements.

15. The updating method according to claim 14, further comprising: Performing target recognition on the historical road image to obtain an element recognition result of the historical road image, wherein the element recognition result of the historical road image includes category information and location information corresponding to at least one element; Performing target recognition on the road image to be processed to obtain an element recognition result of the road image to be processed, wherein the element recognition result of the road image to be processed includes category information and position information corresponding to at least one element; The step of generating an image element set according to the element recognition result of the historical road image and the element recognition result of the road image to be processed includes: Determining a second feature point set for which matching fails from the road image to be processed, wherein the second feature point set includes at least one second feature point; Determining a candidate element set according to the second feature point set and the element recognition result of the road image to be processed; The candidate element set is compared with the element recognition result of the historical road image to determine the image element set.

16. An image matching device, the device being deployed on a computer device, comprising: a processing module, configured to perform feature extraction processing on the first image to be matched to obtain K first feature maps, wherein the first image to be matched has M first feature points, each of the first feature maps includes the M first feature points, K is an integer greater than or equal to 1, and M is an integer greater than 1; The processing module is further used to perform feature extraction processing on the second image to be matched to obtain K second feature maps, wherein the second image to be matched has N second feature points, each of the second feature maps includes the N second feature points, and N is an integer greater than 1; An acquisition module, configured to acquire a first feature vector of each of the M first feature points according to the K first feature maps, wherein the first feature vector includes K first elements, each first element is derived from a different first feature map, and the M first feature vectors corresponding to the first image to be matched are used to describe a first semantic feature and a first physical description feature of the first image to be matched; The acquisition module is further used to acquire a second feature vector of each second feature point in the N second feature points according to the K second feature maps, wherein the second feature vector includes K second elements, each second element is respectively derived from a different second feature map, and the N second feature vectors corresponding to the second image to be matched are used to describe the second semantic feature and the second physical description feature of the second image to be matched; A determination module, configured to determine the number of feature point pairs according to the first feature vector of each first feature point and the second feature vector of each second feature point, wherein the number of feature point pairs represents the number of successful matches between the first feature point and the second feature point; The determination module is further configured to determine an image matching result between the first image to be matched and the second image to be matched according to the number of feature point pairs.

17. A map information updating device, the device being deployed on a computer device, comprising: A processing module, configured to perform feature extraction processing on the historical road image to obtain K first feature maps, wherein the historical road image has M first feature points, each of the first feature maps includes the M first feature points, K is an integer greater than or equal to 1, and M is an integer greater than 1; The processing module is further used to perform feature extraction processing on the road image to be processed to obtain K second feature maps, wherein the acquisition time of the road image to be processed is later than the acquisition time of the historical road image, the road image to be processed has N second feature points, and each of the second feature maps includes the N second feature points, where N is an integer greater than 1; an acquisition module, configured to acquire, according to the K first feature maps, a first feature vector of each of the M first feature points, wherein the first feature vector includes K first elements, each first element is derived from a different first feature map, and the M first feature vectors corresponding to the historical road image are used to describe a first semantic feature and a first physical description feature of the historical road image; The acquisition module is further used to acquire a second feature vector of each second feature point in the N second feature points according to the K second feature maps, wherein the second feature vector includes K second elements, each second element is respectively derived from a different second feature map, and the N second feature vectors corresponding to the road image to be processed are used to describe the second semantic feature and the second physical description feature of the road image to be processed; A determination module, configured to determine the number of feature point pairs according to the first feature vector of each first feature point and the second feature vector of each second feature point, wherein the number of feature point pairs represents the number of successful matches between the first feature point and the second feature point; The determination module is further configured to generate an image element set based on an element recognition result of the historical road image and an element recognition result of the road image to be processed, when it is determined according to the number of feature point pairs that the historical road image and the road image to be processed fail to match, wherein the image element set is derived from at least one of the historical road image and the road image to be processed; An updating module is used to update the map information according to the image element set.

18. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the processor implements the steps of the method described in any one of claims 1 to 13, or implements the steps of the updating method described in any one of claims 14 to 15.

19. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 13, or implements the steps of the updating method according to any one of claims 14 to 15.

20. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 13, or implements the steps of the updating method according to any one of claims 14 to 15.

Images