Image matching method and apparatus

Abstract

The application provides an image matching method and device, which comprises: obtaining a first image and a second image; performing image segmentation on the first image and the second image respectively, and dividing each image into at least a first image region and a second image region; wherein the first image region and the second image region represent image regions of different spatial objects in each image; based on a matching method corresponding to the first image region, establishing a matching relationship of at least one group of feature points between the first image region in the first image and the first image region in the second image; and based on a matching method corresponding to the second image region, establishing a matching relationship of at least one group of feature points between the second image region in the first image and the second image region in the second image. The scheme of the application can accurately locate the feature points matched between two images.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image matching method and apparatus. Background Technology

[0002] Feature point matching is involved in many scenarios such as image stitching, object tracking and detection. Feature point matching refers to finding the correctly matched feature points in two images that need to be registered.

[0003] Accuracy of feature point matching is crucial for image stitching and target tracking based on feature points. However, in real-world scenarios, factors such as the imaging angle and pose of the camera corresponding to different images can easily lead to inaccurate feature point matching between images. Summary of the Invention

[0004] This application provides an image feature point matching method and apparatus.

[0005] One image matching method includes:

[0006] Obtain the first image and the second image;

[0007] The first image and the second image are segmented respectively, and each image is divided into at least a first image region and a second image region; wherein the first image region and the second image region respectively represent the image regions of different spatial objects in each image;

[0008] Based on the matching method corresponding to the first image region, at least one set of feature point matching relationships is established between the first image region in the first image and the first image region in the second image;

[0009] Based on the matching method corresponding to the second image region, at least one set of feature point matching relationships is established between the second image region in the first image and the second image region in the second image.

[0010] In another possible implementation, the method further includes: obtaining camera information, the camera information including: the first camera intrinsic parameters of the first camera corresponding to the first image, the second camera intrinsic parameters of the second camera corresponding to the second image, and the relative camera extrinsic parameters between the first camera and the second camera;

[0011] The first image region is a non-ground region;

[0012] The method based on the matching region corresponding to the first image region establishes a matching relationship of at least one set of feature points between the first image region in the first image and the first image region in the second image, including:

[0013] Based on the first camera intrinsic parameters, the second camera intrinsic parameters, and the relative camera extrinsic parameters, a matching relationship of at least one set of feature points is established between the non-ground regions in the first image and the non-ground regions in the second image.

[0014] Another possible implementation includes:

[0015] Determine the pixel coordinates of at least one first candidate feature point within a non-ground region of the first image;

[0016] Determine the pixel coordinates of at least one second candidate feature point in a non-ground region of the second image;

[0017] The step of establishing a matching relationship of at least one set of feature points between non-ground regions in the first image and non-ground regions in the second image based on the first camera intrinsic parameters, the second camera intrinsic parameters, and relative camera extrinsic parameters includes:

[0018] Based on the first camera intrinsic parameters, the second camera intrinsic parameters, and the relative camera extrinsic parameters, at least one pair of first feature points that match the non-ground region of the first image and the non-ground region of the second image are determined. The first pair of feature points includes: a first candidate feature point and a second candidate feature point whose pixel coordinates match each other.

[0019] In another possible implementation, the method further includes: obtaining camera information, the camera information including: a first camera intrinsic parameter of a first camera corresponding to the first image, a second camera intrinsic parameter of a second camera corresponding to the second image, a first relative extrinsic parameter between the first camera and the ground, and a second relative extrinsic parameter between the second camera and the ground;

[0020] The second image region is the ground region;

[0021] The matching method based on the second image region establishes at least one set of feature point matching relationships between the second image region in the first image and the second image region in the second image, including:

[0022] Based on the first camera intrinsic parameters, the second camera intrinsic parameters, the first relative extrinsic parameters, and the second relative extrinsic parameters, at least one set of feature point matching relationships is established between the ground region in the first image and the ground region in the second image.

[0023] In another possible implementation, the camera information further includes: relative camera extrinsic parameters between the first camera and the second camera;

[0024] The step of establishing a matching relationship of at least one set of feature points between the ground region in the first image and the ground region in the second image based on the first camera intrinsic parameters, the second camera intrinsic parameters, the first relative extrinsic parameters, and the second relative extrinsic parameters includes:

[0025] By combining the first camera intrinsic parameters and the first relative extrinsic parameters, an inverse perspective transformation is performed on the ground region in the first image to obtain the first spatial coordinates of each first pixel in the ground region of the first image in the first coordinate system.

[0026] By combining the intrinsic parameters of the second camera and the extrinsic parameters, an inverse perspective transformation is performed on the ground region in the second image to obtain the second spatial coordinates of each second pixel in the ground region of the second image in the second coordinate system.

[0027] By combining the first relative extrinsic parameter, the second relative extrinsic parameter, and the relative camera extrinsic parameter, at least one pair of second feature points whose coordinates match each other between the ground region of the first image and the ground region of the second image is determined. The second feature point pair includes: a first pixel point and a second pixel point whose first spatial coordinates match each other with the second spatial coordinates.

[0028] In another possible implementation, the first coordinate system is a coordinate system constructed with the projection point of the center point of the first camera on the ground as the origin, and the second coordinate system is a coordinate system constructed with the projection point of the center point of the second camera on the ground as the origin.

[0029] In another possible implementation, determining at least one set of first feature point pairs that match the non-ground regions of the first image and the non-ground regions of the second image based on the first camera intrinsic parameters, the second camera intrinsic parameters, and the relative camera extrinsic parameters includes:

[0030] Based on the first camera intrinsic parameters and the pixel coordinates of the first candidate feature point, determine the camera space coordinates of the first candidate feature point in the camera coordinate system of the first camera.

[0031] Based on the second camera intrinsic parameters and the pixel coordinates of the second candidate feature points, the camera space coordinates of the second candidate feature points in the camera coordinate system of the second camera are determined.

[0032] By combining the relative camera extrinsic parameters, the camera spatial coordinates of the first candidate feature point and the camera spatial coordinates of the second candidate feature point, at least one pair of first feature points that match each other between the non-ground region of the first image and the non-ground region of the second image are determined. The first pair of feature points includes: a first candidate feature point and a second candidate feature point whose camera spatial coordinates match each other.

[0033] In another possible implementation, dividing each image into at least a first image region and a second image region further includes:

[0034] Segment the object image region of each object contained in the non-ground region of each image;

[0035] Determining the pixel coordinates of at least one first candidate feature point within a non-ground region of the first image includes:

[0036] Feature point detection is performed on each object image area in the non-ground region of the first image to obtain the pixel coordinates of at least one first candidate feature point.

[0037] Determining the pixel coordinates of at least one second candidate feature point in a non-ground region of the second image includes:

[0038] Feature point detection is performed on each object image area in the non-ground region of the second image to obtain the pixel coordinates of at least one second candidate feature point.

[0039] In another possible implementation, after determining the matching relationship of at least one set of feature points, the following is also included:

[0040] Obtain the first feature information of the first feature point in the first image and the second feature information of the second feature point in the second image in each group of feature points. The first feature point and the second feature point are two feature points in each group of feature points, and the first feature point comes from the first image and the second feature point comes from the second image.

[0041] For any given set of feature points, if the first feature information of the first feature point and the second feature information of the second feature point in the set do not meet the conditions, then the set of feature points is deleted.

[0042] One type of image matching device includes:

[0043] An image acquisition unit is used to acquire a first image and a second image;

[0044] The image segmentation unit is used to segment the first image and the second image respectively, dividing each image into at least a first image region and a second image region; wherein the first image region and the second image region respectively represent the image regions of different spatial objects in each image;

[0045] The first matching unit is used to establish a matching relationship of at least one set of feature points between the first image region in the first image and the first image region in the second image based on a matching method corresponding to the first image region.

[0046] The second matching unit is used to establish a matching relationship of at least one set of feature points between the second image region in the first image and the second image region in the second image based on the matching method corresponding to the second image region.

[0047] As can be seen from the above, in this application, for the first and second images that require feature point matching, image regions representing different spatial objects in each image are segmented to divide each image into at least a first image region and a second image region. Based on this, this application reduces the influence of factors such as imaging angle and pose on the matched image feature points by performing feature point matching between image regions representing the same spatial object in different images. Furthermore, for any image region in the first and second image regions representing different spatial objects, this application employs a matching method corresponding to that type of image region to establish the feature point matching relationship between that type of image region in the first and second images. This allows for the reasonable selection of matching methods based on the characteristics of image regions representing different spatial objects, thereby enabling more accurate localization of the matching feature points between the two images. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0049] Figure 1 This paper illustrates a flowchart of an image feature point matching method provided in an embodiment of this application.

[0050] Figure 2 This illustration shows another flowchart of the image feature point matching method provided in the embodiments of this application;

[0051] Figure 3 This illustration shows another flowchart of the image feature point matching method provided in the embodiments of this application;

[0052] Figure 4 An example diagram shows the camera coordinate system of this application and the coordinate system constructed from the projection point of the camera's projection on the ground;

[0053] Figure 5 This illustration shows a schematic diagram of the composition structure of an image feature point matching device provided in an embodiment of this application;

[0054] Figure 6 A schematic diagram of the composition structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0055] The solution in this application embodiment is applicable to determining the matching feature points between different images, so as to improve the accuracy of matching feature points between two images.

[0056] For example, before image stitching, matching feature points in the two images to be stitched can be determined based on the scheme of this application. Alternatively, in a target tracking scenario, matching feature points in two images acquired at two different times can be determined.

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0058] like Figure 1 The diagram illustrates a flowchart of an image matching method provided in an embodiment of this application. The method of this embodiment can be applied to an electronic device with image processing capabilities, such as a personal computer or a server, without limitation.

[0059] The method in this embodiment may include:

[0060] S101, obtain the first image and the second image.

[0061] The first image and the second image are the images for which feature point matching needs to be performed.

[0062] For example, in an image stitching scenario, the first image and the second image are two images that need to be stitched together. Before stitching these two images, it is necessary to first use the solution of this embodiment to determine the feature points that can be matched between the two images.

[0063] S102, perform image segmentation on the first image and the second image respectively, dividing each image into at least a first image region and a second image region.

[0064] The first image region and the second image region represent image regions of different spatial objects in each image. Different spatial objects in an image refer to objects located in different spatial regions within the scene presented in the image.

[0065] It is understandable that different spatial objects have different imaging attributes, such as the texture variation of spatial objects and the degree to which spatial objects are affected by factors such as imaging angle and lighting. As a result, the feature point matching methods suitable for the image regions corresponding to different types of spatial objects will also be different. Based on this, this application can be applied to spatial objects with imaging attributes.

[0066] S103, based on the matching method corresponding to the first image region, establish at least one set of feature point matching relationships between the first image region in the first image and the first image region in the second image.

[0067] S104, Based on the matching method corresponding to the second image region, establish at least one set of feature point matching relationships between the second image region in the first image and the second image region in the second image.

[0068] It is understandable that the different matching methods corresponding to the first image region and the second image region can be determined by combining the imaging attribute features of the spatial objects corresponding to the first image region and the second image region respectively, without any restrictions.

[0069] Each set of feature points may include a first feature point and a second feature point that have a matching relationship, wherein the first feature point comes from the first image and the second feature point comes from the second image. Moreover, two feature points within the same set of feature points come from the image regions corresponding to the same spatial object in the first and second images.

[0070] For example, the matching relationship between each set of feature points established in step S104 may include: the matching relationship between a first feature point in the second image region of the first image and a second feature point in the second image region of the second image.

[0071] In this embodiment, for the first and second images requiring feature point matching, image regions representing different spatial objects are segmented in each image, thus dividing each image into at least a first image region and a second image region. Based on this, this application reduces the influence of factors such as imaging angle and pose on the matched image feature points by performing feature point matching between image regions representing the same spatial object in different images. Furthermore, for any image region representing different spatial objects in the first and second image regions, this application employs a matching method corresponding to that type of image region to establish a matching relationship between feature points of that type of image region in the first and second images. This allows for the reasonable selection of matching methods based on the characteristics of image regions representing different spatial objects, thereby enabling more accurate localization of matching feature points between different images.

[0072] In this application, there are several possible methods for image segmentation of the first image and the second image. The following describes several possible scenarios.

[0073] In one possible implementation, considering that the texture and color features of the ground do not vary much, and that areas with repetitive textures or colors are easily found, making it difficult to locate key points with distinctive or distinguishable features from the ground, this application can combine the ground as a spatial object with spatial objects outside the ground to segment the image into ground regions and non-ground regions. For example, the first image region can be a non-ground region, while the second image region is a ground region.

[0074] In another possible implementation, considering that the sky has imaging properties similar to those of the ground, if the image includes both the sky and objects outside the sky, this application can also divide the first image and the second image into sky regions and non-sky regions respectively.

[0075] Of course, the first and second images can also be segmented into sky regions, ground regions, and non-ground sky regions. In practical applications, there are other possibilities for the segmentation of the first and second images, and there are no restrictions on this.

[0076] For ease of understanding, considering that in daily life and applications, there are many scenarios where images are captured and processed using cameras set up in the middle and on both sides of roads, in parks, and on various ground buildings or facilities, this application uses the first image area as a non-ground area and the second image area as a ground area as an example for explanation.

[0077] like Figure 2 As shown, it illustrates another flowchart of the image matching method provided in this application embodiment. The method of this embodiment may include:

[0078] S201, acquire the first image, the second image, and camera information.

[0079] The camera information includes: the first camera intrinsic parameters of the first camera corresponding to the first image, the second camera intrinsic parameters of the second camera corresponding to the second image, the relative camera extrinsic parameters between the first camera and the second camera, the first relative extrinsic parameters between the first camera and the ground, and the second relative extrinsic parameters between the second camera and the ground.

[0080] Here, the camera corresponding to the image refers to the camera that generated the image. Camera intrinsic parameters characterize parameters related to the camera's own features, such as focal length and pixel size. Camera intrinsic parameters are the basis for transforming from the camera's coordinate system to the image's pixel coordinate system.

[0081] For example, camera intrinsic parameters can be transformation coefficients from the camera coordinate system to the image's pixel coordinate system, or transformation coefficients from the image's pixel coordinate system to the camera coordinate system. These transformation coefficients can be transformation matrices or relevant parameters required for coordinate system transformation.

[0082] In this application, for ease of distinction, the camera corresponding to the first image is referred to as the first camera, and the camera intrinsics of the first camera are referred to as the first camera intrinsics. Correspondingly, the camera corresponding to the second image is referred to as the second camera, and the camera intrinsics of the second camera are referred to as the second camera intrinsics.

[0083] The relative camera extrinsic parameters between the first and second cameras refer to the camera parameters relative to each other, such as relative camera position and relative rotation direction. These parameters serve as the basis for conversion between the camera coordinate system of the first camera and the camera coordinate system of the second camera.

[0084] For example, the relative camera extrinsic parameters may include: a transformation matrix from the camera coordinate system of the first camera to the camera coordinate system of the second camera, or a transformation matrix from the camera coordinate system of the second camera to the camera coordinate system of the first camera. The transformation matrix may include a rotation matrix and a translation vector.

[0085] The relative extrinsic parameters between the camera and the ground can characterize information such as the camera's position, orientation, and angle relative to the ground. These parameters serve as the basis for transformation from the camera's coordinate system to the ground's coordinate system. For example, the relative extrinsic parameters can include the aforementioned information such as the camera's position, orientation, and angle relative to the ground; or they can include a transformation matrix from the camera's coordinate system to a coordinate system established based on the camera's projection point on the ground.

[0086] In this application, taking the example that the relative positional relationship between the first camera corresponding to the first image and the second camera corresponding to the second image is determined and known, and the relative relationship between the first camera and the ground and the second camera and the ground is also determined and known, the camera information mentioned above can be obtained.

[0087] S202, perform image segmentation on the first image and the second image respectively, and divide each image into at least a ground region and a non-ground region.

[0088] In this context, the ground region in an image refers to the area of ​​the image containing the ground. The non-ground region refers to the area of ​​the image outside the area containing the ground.

[0089] For example, if an image contains objects such as the ground, pedestrians, and vehicles, then the image can be divided into a ground area and a non-ground area containing pedestrians and vehicles.

[0090] In this application, there are various possible methods for segmenting the first image and the second image, such as using a set image extraction algorithm or image recognition algorithm to identify and extract the ground area and non-ground area, etc., without any limitation.

[0091] S203, based on the first camera intrinsic parameters, the second camera intrinsic parameters, and the relative camera extrinsic parameters, establish at least one set of feature point matching relationships between the non-ground regions in the first image and the non-ground regions in the second image.

[0092] It is understandable that for two images, the feature points that match each other must be points within the same spatial object's corresponding image region. Therefore, this application performs feature point matching between non-ground regions and between ground regions of the two images, which can reduce feature mismatches to a certain extent.

[0093] As can be seen from the previous introduction of the first camera intrinsic parameters, the second camera intrinsic parameters, and the relative camera extrinsic parameters, combining the first camera intrinsic parameters, the second camera intrinsic parameters, and the relative camera extrinsic parameters can reflect the coordinate position relationship of the feature points in the non-ground region of the first image and the feature points in the non-ground region of the second image in the same geometric space. Naturally, at least one set of feature points that match each other between the non-ground region of the first image and the non-ground region of the second image can be determined, thereby constructing the matching relationship between feature points.

[0094] For example, by combining the intrinsic parameters of the first camera, the intrinsic parameters of the second camera, and the relative extrinsic parameters of the camera, the coordinates of pixels in the non-ground region of the first image and pixels in the non-ground region of the second image within the same camera coordinate system (such as the camera coordinate system of the first camera or the camera coordinate system of the second camera) can be determined. Based on this, a group of pixels in the non-ground region of the first image and the non-ground region of the second image with the same coordinates in the same camera coordinate system can be identified as a group of feature points.

[0095] For example, if pixel 1 in the non-ground region of the first image and pixel 2 in the non-ground region of the second image have the same coordinates in the camera coordinate system of the first camera (or the camera coordinate system of the second camera), then a matching relationship can be established between pixel 1 and pixel 2, and pixel 1 and pixel 2 can be determined as a set of matching feature points.

[0096] As can be seen from the above, in this embodiment, the feature points that match the spatial positions in the non-ground regions of the first and second images are determined geometrically by combining the camera intrinsic parameters corresponding to the first and second images respectively, as well as the relative camera extrinsic parameters between the cameras corresponding to the two images. Since the spatial position of pixels in geometric space is not affected or is minimally affected by factors such as texture, camera pose, and angle, the matching feature points in the two images can be determined relatively accurately.

[0097] In one possible implementation, in order to further accurately match feature points in non-ground regions of the first image and the second image, this application may first determine the pixel coordinates of at least one first candidate feature point in the non-ground region of the first image, and the pixel coordinates of at least one second candidate feature point in the non-ground region of the second image.

[0098] For either the first or second image, candidate feature points in the non-ground regions of the image can be representative key points detected by performing landmark detection on the non-ground regions. These key points can be points that can significantly represent the object features of objects in the non-ground regions.

[0099] Based on this, this application can determine at least one pair of first feature points that match between non-ground regions in the first image and non-ground regions in the second image, based on the intrinsic parameters of the first camera, the intrinsic parameters of the second camera, and the relative extrinsic parameters of the camera. The pair of first feature points includes: first candidate feature points and second candidate feature points whose pixel coordinates match. For example, matching pixel coordinates can mean that the pixel coordinates are the same in the same spatial coordinate system (such as the camera coordinate system of the first camera or the second camera) or that the distance is less than a set threshold.

[0100] S204, based on the first camera intrinsic parameters, the second camera intrinsic parameters, the first relative extrinsic parameters, and the second relative extrinsic parameters, establish at least one set of feature point matching relationships between the ground region in the first image and the ground region in the second image.

[0101] As introduced above, the transformation relationship from the image coordinate system to the camera coordinate system can be determined based on the camera intrinsic parameters of the image. Therefore, by combining the camera intrinsic parameters of the image, the coordinates of pixels in the ground region of the image in the corresponding camera coordinate system can be determined. Based on this, the coordinate transformation relationship from the camera coordinate system to the ground can be determined by combining the relative extrinsic parameters of each camera and the ground. Based on this, using the first camera intrinsic parameters, the second camera intrinsic parameters, the first relative extrinsic parameters, and the second relative extrinsic parameters, the feature points that can be matched between the ground regions in the first image and the ground regions in the second image within the same coordinate space can be determined.

[0102] The inventors of this application have discovered through research that, currently, in the process of image feature point matching, the pixel features of each pixel in two images are generally determined, and then pixels with the same or similar pixel features in the two images are identified as mutually matching feature points. The pixel features of a pixel in an image are determined based on the feature values ​​of that pixel and its neighboring pixels.

[0103] However, due to factors such as lighting, the images captured by the camera from different perspectives will vary greatly. Moreover, different perspectives naturally have differences in pose, and some objects naturally have weak texture features. As a result, the pixel features of feature points that should match each other in different images are quite different, leading to inaccurate feature point matching.

[0104] Unlike existing feature point matching methods that rely on pixel features from different images, this application combines features from both ground and non-ground regions of the images. It also incorporates at least some information from the camera intrinsic parameters, relative camera extrinsic parameters, and relative extrinsic parameters of each camera relative to the ground in both images. This allows for the construction of matching relationships between feature points in the ground regions and feature points in the non-ground regions of the two images. This effectively utilizes the invariance of the coordinate position of the same point within the same object region in geometric space for feature point matching, reducing the influence of factors such as lighting and texture on feature point matching and thus improving the accuracy of feature point matching.

[0105] In particular, for ground regions in an image, where there are areas with weak or repetitive textures, feature point matching based solely on pixel features is more prone to errors such as mismatch. For example, the ground region in an image may include a road surface, and the texture and other features of different road sections are very similar. It is difficult to distinguish pixels on different road sections by simply relying on pixel features in the image. However, the solution proposed in this application does not need to consider the pixel features of the ground region, thus effectively avoiding mismatches of feature points due to the similarity of texture and other features among different pixels in the ground region.

[0106] Understandably, given the various camera information sources mentioned earlier, there are multiple possibilities for determining matching feature points in the ground and non-ground regions of two images. Specifically, transformations between camera coordinate systems, spatial coordinate systems, and ground coordinate systems can be performed on the points in the images as needed, without any restrictions.

[0107] In one possible implementation, considering that during the process of the camera shooting the ground, due to the existence of perspective effect, objects that are originally parallel may appear to intersect in the image, based on this, in order to eliminate this perspective effect, this application can also combine inverse perspective transformation to transform the pixels in the ground area of ​​the image.

[0108] The image matching method of this application will be introduced below with reference to one possible implementation.

[0109] like Figure 3 As shown, it illustrates another flowchart of the image matching method provided in this application embodiment. The method of this embodiment may include:

[0110] S301, acquires the first image, the second image, and camera information.

[0111] The camera information includes: the first camera intrinsic parameters of the first camera corresponding to the first image, the second camera intrinsic parameters of the second camera corresponding to the second image, the relative camera extrinsic parameters between the first camera and the second camera, the first relative extrinsic parameters between the first camera and the ground, and the second relative extrinsic parameters between the second camera and the ground.

[0112] S302, perform image segmentation on the first image and the second image respectively, and divide each image into at least a ground region and a non-ground region.

[0113] The above two steps can be referred to in the relevant descriptions of the previous embodiments, and will not be repeated here.

[0114] S303, determine the pixel coordinates of at least one first candidate feature point in a non-ground region of the first image.

[0115] S304, determine the pixel coordinates of at least one second candidate feature point in the non-ground region of the second image.

[0116] Candidate feature points are key points detected in the non-ground regions of an image. Key points are points with significant features in the non-ground regions, making them easier to distinguish from other points in the non-ground regions of the image. Therefore, they are more conducive to accurate feature point matching between different images.

[0117] The pixel coordinates of a candidate feature point refer to the coordinates of the candidate feature point in the image coordinate system.

[0118] For example, a feature point detection algorithm can be used to detect at least one key point and its coordinates from the non-ground region of an image, and the detected key point can be identified as a candidate feature point.

[0119] For ease of distinction, this application refers to candidate feature points in non-ground regions of the first image as first candidate feature points, and candidate feature points in non-ground regions of the second image as second candidate feature points.

[0120] In an alternative approach, in order to extract candidate feature points with salient features more accurately from the non-ground regions of the image, this application may further segment the object image region of each object contained in the non-ground region of each image after segmenting the image into ground regions and non-ground regions.

[0121] Based on this, for the first image, feature point detection can be performed on each object image area in the non-ground region of the first image to obtain the pixel coordinates of at least one first candidate feature point.

[0122] Similarly, feature point detection can be performed on each object image area in the non-ground region of the second image to obtain the pixel coordinates of at least one second candidate feature point.

[0123] Among them, the object image area of ​​each object in the non-ground area of ​​the image can be the image area of ​​people, animals, objects or buildings in the non-ground area of ​​the image.

[0124] Correspondingly, the candidate feature points extracted from the object image region can be the key points that can characterize the object's features. For example, taking a person as an example, key points of the eyes, nose, eyebrows, ears, etc. in the face can be extracted from the person's image region; key points of the human torso can also be extracted, such as finger joints, elbow positions, and torso joints.

[0125] By extracting the object image region from the non-ground region of the image and then performing feature point detection on the object image region, feature point detection can be achieved more specifically. This is beneficial for more accurate detection of key points that can characterize the object features and can also reduce the detection of some insignificant invalid key points from some background regions in the non-ground region.

[0126] S305, for each first candidate feature point, based on the first camera intrinsic parameters and the pixel coordinates of the first candidate feature point, determine the camera space coordinates of the first candidate feature point in the camera coordinate system of the first camera.

[0127] Since the camera intrinsics corresponding to an image are the basis for transforming the image coordinate system to the camera coordinate system, for an image, if the camera intrinsics corresponding to the image are determined, the pixel coordinates of any point in the image after transformation to the camera coordinate system can be determined, that is, the camera spatial coordinates.

[0128] For example, assuming the intrinsic parameter of the first camera is a transformation matrix that transforms from the image coordinate system to the camera coordinate system of the first camera, then the pixel coordinates of the first candidate feature point can be multiplied by the transformation matrix to obtain the camera space coordinates of the first candidate feature point in the camera coordinate system of the first camera.

[0129] S306, for each second candidate feature point, based on the second camera intrinsic parameters and the pixel coordinates of the second candidate feature point, determine the camera space coordinates of the second candidate feature point in the camera coordinate system of the second camera.

[0130] The specific implementation of determining the camera space coordinates of the second candidate feature point in the camera coordinate system of the second camera, as well as the related introduction of the previous step S305, will not be repeated here.

[0131] S307, combining relative camera extrinsic parameters, camera spatial coordinates of the first candidate feature point, and camera spatial coordinates of the second candidate feature point, determine at least one pair of first feature points that match the non-ground region of the first image and the non-ground region of the second image.

[0132] The first feature point pair includes a first candidate feature point and a second candidate feature point whose camera spatial coordinates match each other. That is, in each first feature point pair, the camera spatial coordinates of the first candidate feature point and the second candidate feature point are the same in the same coordinate system. Of course, it is also possible to set the coordinate difference between the two candidate feature points in the same coordinate system to be less than a set threshold.

[0133] For example, by combining relative camera extrinsic parameters, the camera spatial coordinates of the first candidate feature point and the second candidate feature point can be transformed into a specified spatial coordinate system, resulting in the first specified coordinates of each candidate first feature point and the second specified coordinates of each candidate second feature point in the specified spatial coordinate system. The first specified coordinates of the first candidate feature point and the second specified coordinates of the second candidate feature point are then compared. If they are the same, it is confirmed that the first candidate feature point and the second candidate feature point are a match.

[0134] To reduce the complexity of coordinate transformation, the specified spatial coordinate system can be either the first camera coordinate system or the second camera coordinate system. For example, the specified spatial coordinate system can be the first camera coordinate system.

[0135] For example: Suppose there is a candidate feature point 'a' in image A and a candidate feature point 'b' in image B. Assume the camera space coordinates of candidate feature point 'a' in the first camera coordinate system corresponding to the first camera in image A are coordinates a1, while the coordinates of candidate feature point 'b' in the second camera coordinate system corresponding to the first camera in image B are coordinates b1. Then, based on this relative camera extrinsic parameter, the coordinates of candidate feature point 'b1' after transformation to the first camera coordinate system are also a1. Therefore, candidate feature point 'a' in image A and candidate feature point 'b' in image B are a pair of feature points whose camera space coordinates match.

[0136] It should be noted that S305 to S307 are examples of one implementation of determining at least one pair of first feature points in a non-ground region of the first image and the second image. Other methods mentioned in the previous embodiments are also applicable to this embodiment, and will not be described again.

[0137] S308, combining the first camera intrinsic parameters and the first relative extrinsic parameters, performs an inverse perspective transformation on the ground region in the first image to obtain the first spatial coordinates of each first pixel point in the ground region of the first image in the first coordinate system.

[0138] Inverse perspective transformation is used to eliminate the impact of perspective effects on the imaging of ground areas in an image.

[0139] There are many ways to implement Inverse Perspective Mapping (IPM), and this application does not impose any restrictions on this.

[0140] In this application, during the inverse perspective transformation process, while eliminating the perspective effect, it is necessary to map the ground region of the first image after the final inverse perspective transformation onto the first coordinate system, thereby obtaining the spatial coordinates of each first pixel in the ground region of the first image after the inverse perspective transformation in the first coordinate system. For ease of distinction, the coordinates of each pixel in the ground region of the first image in the first coordinate system are referred to as the first spatial coordinates.

[0141] Specifically, based on the intrinsic parameters of the first camera and the first relative extrinsic parameters between the first camera and the ground, points in the image coordinate system of the first image can be mapped to the first coordinate system space. The first coordinate system space can have multiple possibilities.

[0142] For example, in one possible implementation, the first coordinate system is a coordinate system constructed with the projection point of the center point of the first camera on the ground as the origin.

[0143] Understandably, given that the intrinsic parameters of the first camera are determined, the coordinates of each pixel in the ground region of the first image within the first camera coordinate system can be determined. Based on this, since the extrinsic parameters of the first camera relative to the ground are also determined, after constructing a first coordinate system with the projection of the center point of the first camera onto the ground as its origin, the coordinates of each pixel in the ground region of the first image within the first camera coordinate system can be converted to coordinates within the first coordinate system by combining the extrinsic parameters.

[0144] Let's take a common scenario as an example. Figure 4 Let's take a scenario where the ground area includes a road surface as an example for illustration. Figure 4 The diagram shows the camera coordinate system and the coordinates constructed from the camera's projection point on the ground.

[0145] exist Figure 4 In this diagram, a camera coordinate system is established with the camera's location C1 as the origin. At this point, the camera angle is 40°. The three coordinate axes in the camera coordinate system are X1, Y1, and Z1.

[0146] Based on the camera's location projected onto the ground, the camera's projection point C2 on the ground can be determined. This projection point C2 is located on the road surface. A coordinate system constructed with this projection point C2 as the origin has three coordinate axes: X2, Y2, and Z2.

[0147] Combination Figure 4 As can be seen, when the relative external parameters of the camera relative to the ground are known, by combining inverse perspective transformation, the perspective effect can be eliminated, and the coordinates of each point in the camera coordinate system to the coordinate system on the ground can be determined.

[0148] S309, combining the intrinsic parameters of the second camera and the extrinsic parameters of the second relative camera, performs an inverse perspective transformation on the ground region in the second image to obtain the second spatial coordinates of each second pixel point in the ground region of the second image in the second coordinate system.

[0149] Similar to step S308, while performing reverse perspective on the ground region of the second image, this application can determine the second spatial coordinates of each pixel in the ground region of the second image in a second coordinate system by combining the extrinsic parameters of the second camera and the second relative extrinsic parameters between the second camera and the ground, so as to facilitate subsequent comparison of the spatial coordinates of each pixel in the first coordinate system and the second coordinate system.

[0150] Similar to the previous one, the second coordinate system can have several possibilities. In one possible case, the second coordinate system can be constructed with the projection point of the center point of the second camera onto the ground as the origin. For details on constructing this second coordinate system, please refer to the relevant description in step S308 above; it will not be repeated here.

[0151] As can be understood from the preceding description, due to the relatively weak texture of the ground, it is impossible to guarantee the accurate identification of points with distinctiveness or salience as key points, i.e., candidate feature points, from the ground region of the image. Based on this, this application does not perform feature point detection on non-ground regions of the image, but directly determines the spatial coordinates of the ground region in the image in the first or second coordinate system, and then directly performs feature point matching based on the spatial coordinates of the pixels to improve the accuracy of feature point matching.

[0152] S310, combining the first relative extrinsic parameter, the second relative extrinsic parameter, and the relative camera extrinsic parameter, determine at least one pair of second feature points whose coordinates match the ground region of the first image and the ground region of the second image.

[0153] The second feature point pair includes a first pixel point and a second pixel point whose first spatial coordinates and second spatial coordinates match each other.

[0154] As described above, by combining the first relative extrinsic parameter, the second relative extrinsic parameter, and the relative camera extrinsic parameter, the transformation relationship between the camera coordinate system corresponding to the image and the camera coordinate system relative to the ground can be determined. Based on this, this application can determine the pixel pairs whose coordinates match in the ground region of the first image and the ground region of the second image through coordinate transformation, and define the pixel pairs as a set of second feature point pairs.

[0155] Of course, there can be multiple specific coordinate transformations to complete feature point matching, and there are no restrictions on this.

[0156] For example, consider a first coordinate system established with the projection point of the first camera on the ground as its origin, and a second coordinate system also established with the projection point of the first camera on the ground as its origin. Combining the first relative extrinsic parameters, the second relative extrinsic parameters, and the relative camera extrinsic parameters, determine the transformation matrix from the first coordinate system to the second coordinate system (or, similarly, the transformation matrix from the second coordinate system to the first coordinate system). Based on this, multiply the first spatial coordinates of each pixel in the ground region of the first image in the first coordinate system by this transformation matrix to obtain the second spatial coordinates of each pixel in the ground region of the first image in the second coordinate system. Then, pixels in the ground regions of the first and second images with the same second spatial coordinates are a pair of feature points, i.e., a second feature point pair.

[0157] It is understood that the above embodiments are all illustrated by dividing each of the first and second images into ground regions and non-ground regions. In practical applications, if the first and second images also include sky regions, each image can be divided into ground regions, sky regions, and non-ground and sky regions. The processing of non-ground and sky regions can be the same as that for non-ground regions, while the processing of sky regions can be the same as that for ground regions. Of course, the first and second images can also have other segmentation methods, which are not limited.

[0158] In this application, in order to further improve the accuracy of feature point matching, after determining the matching relationship of at least one set of feature points, the first feature information of the first feature point in the first image and the second feature information of the second feature point in the second image can also be obtained in each set of feature points. The first feature point and the second feature point are two feature points in each set of feature points, and the first feature point comes from the first image and the second feature point comes from the second image.

[0159] Correspondingly, for any given set of feature points, if the first feature information of the first feature point and the second feature information of the second feature point in the set do not meet the conditions, the set of feature points is deleted.

[0160] The feature information of a feature point can be the feature of the feature point itself, or it can be the feature of the feature point and its neighborhood.

[0161] For example, if the similarity between the first feature information of the first feature point and the second feature information of the second feature point is less than a set threshold, then the condition can be considered not met.

[0162] It is understandable that the feature information of feature points that can be matched in two images must be the same or similar. Therefore, after determining the feature point information in the previous way, we can further filter each set of feature points by whether the feature information of two feature points in a set are similar, which can further reduce the number of mismatched feature points.

[0163] Corresponding to the image matching method provided in the embodiments of this application, this application also provides an image matching apparatus. For example... Figure 5 The diagram illustrates a structural composition of an image matching apparatus provided in an embodiment of this application. The apparatus includes:

[0164] Image acquisition unit 501 is used to acquire a first image and a second image;

[0165] The image segmentation unit 502 is used to segment the first image and the second image respectively, dividing each image into at least a first image region and a second image region; wherein the first image region and the second image region respectively represent the image regions of different spatial objects in each image;

[0166] The first matching unit 503 is used to establish at least one set of feature point matching relationships between the first image region in the first image and the first image region in the second image based on the matching method corresponding to the first image region;

[0167] The second matching unit 504 is used to establish a matching relationship of at least one set of feature points between the second image region in the first image and the second image region in the second image based on the matching method corresponding to the second image region.

[0168] In one possible implementation, the device further includes:

[0169] An information acquisition unit is used to acquire camera information, the camera information including: the first camera intrinsic parameters of the first camera corresponding to the first image, the second camera intrinsic parameters of the second camera corresponding to the second image, and the relative camera extrinsic parameters between the first camera and the second camera;

[0170] The first image region segmented by the image segmentation unit is a non-ground region;

[0171] The first matching unit includes:

[0172] The first feature point matching unit is used to establish a matching relationship of at least one set of feature points between the non-ground region in the first image and the non-ground region in the second image based on the first camera intrinsic parameters, the second camera intrinsic parameters, and the relative camera extrinsic parameters.

[0173] In yet another possible implementation, the device further includes:

[0174] The first feature point determination unit is used to determine the pixel coordinates of at least one first candidate feature point in a non-ground region of the first image.

[0175] The second feature point determination unit is used to determine the pixel coordinates of at least one second candidate feature point in a non-ground region of the second image.

[0176] The first feature point matching unit includes:

[0177] The first feature point matching subunit is used to determine at least one pair of first feature points that match between the non-ground region of the first image and the non-ground region of the second image based on the first camera intrinsic parameters, the second camera intrinsic parameters and the relative camera extrinsic parameters. The first feature point pair includes: a first candidate feature point and a second candidate feature point whose pixel coordinates match each other.

[0178] Furthermore, the first feature point matching subunit includes:

[0179] The first coordinate transformation subunit is used to determine the camera space coordinates of the first candidate feature point in the camera coordinate system of the first camera based on the first camera intrinsic parameters and the pixel coordinates of the first candidate feature point.

[0180] The second coordinate transformation subunit is used to determine the camera space coordinates of the second candidate feature point in the camera coordinate system of the second camera based on the second camera intrinsic parameters and the pixel coordinates of the second candidate feature point.

[0181] The coordinate matching subunit is used to combine the relative camera extrinsic parameters, the camera spatial coordinates of the first candidate feature point and the camera spatial coordinates of the second candidate feature point to determine at least one pair of first feature points that match between the non-ground region of the first image and the non-ground region of the second image. The first feature point pair includes: a first candidate feature point and a second candidate feature point whose camera spatial coordinates match each other.

[0182] Furthermore, the image segmentation unit of the device also includes:

[0183] The object segmentation unit is used to segment the object image region of each object contained in the non-ground region of each image after dividing each image into at least a first image region and a second image region.

[0184] The first feature point determination unit includes:

[0185] The first feature point determination subunit is used to perform feature point detection on each object image area in the non-ground region of the first image to obtain the pixel coordinates of at least one first candidate feature point.

[0186] The second feature point determination unit includes:

[0187] The second feature point determination subunit is used to perform feature point detection on each object image area in the non-ground region of the second image to obtain the pixel coordinates of at least one second candidate feature point.

[0188] In yet another possible implementation, the device further includes:

[0189] An information acquisition unit is used to acquire camera information, the camera information including: a first camera intrinsic parameter of the first camera corresponding to the first image, a second camera intrinsic parameter of the second camera corresponding to the second image, a first relative extrinsic parameter between the first camera and the ground, and a second relative extrinsic parameter between the second camera and the ground;

[0190] The second image region segmented by the image segmentation unit is the ground region;

[0191] The second matching unit includes:

[0192] The second feature point matching unit is used to establish at least one set of feature point matching relationships between the ground region in the first image and the ground region in the second image based on the first camera intrinsic parameters, the second camera intrinsic parameters, the first relative extrinsic parameters, and the second relative extrinsic parameters.

[0193] In one possible implementation, the camera information obtained by the information obtaining unit further includes: the relative camera extrinsic parameters between the first camera and the second camera;

[0194] The second feature point matching unit includes:

[0195] The first inverse perspective subunit is used to combine the first camera intrinsic parameters and the first relative extrinsic parameters to perform inverse perspective transformation on the ground region in the first image to obtain the first spatial coordinates of each first pixel point in the ground region of the first image in the first coordinate system.

[0196] The second inverse perspective subunit is used to combine the second camera intrinsic parameters and the second relative extrinsic parameters to perform inverse perspective transformation on the ground region in the second image, so as to obtain the second spatial coordinates of each second pixel point in the ground region of the second image in the second coordinate system.

[0197] The second feature point matching subunit is used to combine the first relative extrinsic parameter, the second relative extrinsic parameter and the relative camera extrinsic parameter to determine at least one pair of second feature points whose coordinates match between the ground region of the first image and the ground region of the second image. The second feature point pair includes: a first pixel point and a second pixel point whose first spatial coordinates match the second spatial coordinates.

[0198] In one alternative approach, the first coordinate system is a coordinate system constructed with the projection point of the center point of the first camera on the ground as the origin, and the second coordinate system is a coordinate system constructed with the projection point of the center point of the second camera on the ground as the origin.

[0199] In yet another possible implementation, the device further includes:

[0200] The feature acquisition unit is used to obtain, after the first matching unit and the second matching unit determine the matching relationship of at least one set of feature points, the first feature point in the first image and the second feature point in the second image in each set of feature points, wherein the first feature point and the second feature point are two feature points in each set of feature points, and the first feature point comes from the first image and the second feature point comes from the second image.

[0201] The feature point filtering unit is used to delete any set of feature points determined by the first matching unit and the second matching unit if the first feature information of the first feature point and the second feature information of the second feature point in the set of feature points do not meet the conditions.

[0202] Furthermore, this application also provides an electronic device, such as Figure 6 The diagram shows a structural composition of the electronic device, which can be any type of electronic device and includes at least a processor 601 and a memory 602.

[0203] The processor 601 is used to execute the image matching method as described in any of the above embodiments.

[0204] The memory 602 is used to store programs required by the processor to perform operations.

[0205] It is understood that the electronic device may also include a display unit 603 and an input unit 604.

[0206] Of course, the electronic device can also have more than Figure 6 There are no restrictions on the number of components, whether more or fewer.

[0207] On the other hand, this application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the image matching method as described in any of the above embodiments.

[0208] This application also proposes a computer program comprising computer instructions stored in a computer-readable storage medium. When executed on an electronic device, the computer program performs the image matching method as described in any of the above embodiments.

[0209] It is understood that in this application, the terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar parts and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that illustrated herein.

[0210] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Furthermore, the features described in the various embodiments of this specification can be substituted or combined with each other, enabling those skilled in the art to implement or use this application. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0211] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0212] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0213] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image matching method, comprising: Obtain the first image and the second image; The first image and the second image are segmented respectively, and each image is divided into at least a first image region and a second image region; wherein the first image region and the second image region represent the image regions of different spatial objects in each image, and the different spatial objects represent objects located in different spatial regions within the scene presented by the image; Based on the matching method corresponding to the first image region, at least one set of feature point matching relationships is established between the first image region in the first image and the first image region in the second image; Based on the matching method corresponding to the second image region, at least one set of feature point matching relationships is established between the second image region in the first image and the second image region in the second image.

2. The method of claim 1, further comprising: Obtain camera information, which includes: the first camera intrinsic parameters of the first camera corresponding to the first image, the second camera intrinsic parameters of the second camera corresponding to the second image, and the relative camera extrinsic parameters between the first camera and the second camera; The first image region is a non-ground region; The method based on the matching region corresponding to the first image region establishes a matching relationship of at least one set of feature points between the first image region in the first image and the first image region in the second image, including: Based on the first camera intrinsic parameters, the second camera intrinsic parameters, and the relative camera extrinsic parameters, a matching relationship of at least one set of feature points is established between the non-ground regions in the first image and the non-ground regions in the second image.

3. The method according to claim 2, further comprising: Determine the pixel coordinates of at least one first candidate feature point within a non-ground region of the first image; Determine the pixel coordinates of at least one second candidate feature point within a non-ground region of the second image; The step of establishing a matching relationship of at least one set of feature points between non-ground regions in the first image and non-ground regions in the second image based on the first camera intrinsic parameters, the second camera intrinsic parameters, and relative camera extrinsic parameters includes: Based on the first camera intrinsic parameters, the second camera intrinsic parameters, and the relative camera extrinsic parameters, at least one pair of first feature points that match the non-ground region of the first image and the non-ground region of the second image are determined. The first pair of feature points includes: a first candidate feature point and a second candidate feature point whose pixel coordinates match each other.

4. The method of claim 1, further comprising: Obtain camera information, which includes: the first camera intrinsic parameters of the first camera corresponding to the first image, the second camera intrinsic parameters of the second camera corresponding to the second image, the first relative extrinsic parameters of the first camera and the ground, and the second relative extrinsic parameters of the second camera and the ground; The second image region is the ground region; The matching method based on the second image region establishes at least one set of feature point matching relationships between the second image region in the first image and the second image region in the second image, including: Based on the first camera intrinsic parameters, the second camera intrinsic parameters, the first relative extrinsic parameters, and the second relative extrinsic parameters, at least one set of feature point matching relationships is established between the ground region in the first image and the ground region in the second image.

5. The method of claim 4, the camera information further comprising: The relative camera extrinsic parameters between the first camera and the second camera; The step of establishing a matching relationship of at least one set of feature points between the ground region in the first image and the ground region in the second image based on the first camera intrinsic parameters, the second camera intrinsic parameters, the first relative extrinsic parameters, and the second relative extrinsic parameters includes: By combining the first camera intrinsic parameters and the first relative extrinsic parameters, an inverse perspective transformation is performed on the ground region in the first image to obtain the first spatial coordinates of each first pixel in the ground region of the first image in the first coordinate system. By combining the intrinsic parameters of the second camera and the extrinsic parameters, an inverse perspective transformation is performed on the ground region in the second image to obtain the second spatial coordinates of each second pixel in the ground region of the second image in the second coordinate system. By combining the first relative extrinsic parameter, the second relative extrinsic parameter, and the relative camera extrinsic parameter, at least one pair of second feature points whose coordinates match each other between the ground region of the first image and the ground region of the second image is determined. The second feature point pair includes: a first pixel point and a second pixel point whose first spatial coordinates match each other with the second spatial coordinates.

6. The method according to claim 5, wherein the first coordinate system is a coordinate system constructed with the projection point of the center point of the first camera on the ground as the origin, and the second coordinate system is a coordinate system constructed with the projection point of the center point of the second camera on the ground as the origin.

7. The method according to claim 3, wherein determining at least one pair of first feature points that match the non-ground region of the first image and the non-ground region of the second image based on the first camera intrinsic parameters, the second camera intrinsic parameters, and the relative camera extrinsic parameters comprises: Based on the first camera intrinsic parameters and the pixel coordinates of the first candidate feature point, determine the camera space coordinates of the first candidate feature point in the camera coordinate system of the first camera. Based on the second camera intrinsic parameters and the pixel coordinates of the second candidate feature points, the camera space coordinates of the second candidate feature points in the camera coordinate system of the second camera are determined. By combining the relative camera extrinsic parameters, the camera spatial coordinates of the first candidate feature point and the camera spatial coordinates of the second candidate feature point, at least one pair of first feature points that match each other between the non-ground region of the first image and the non-ground region of the second image are determined. The first pair of feature points includes: a first candidate feature point and a second candidate feature point whose camera spatial coordinates match each other.

8. The method according to claim 3, wherein dividing each image into at least a first image region and a second image region further comprises: Segment the object image region of each object contained in the non-ground region of each image; Determining the pixel coordinates of at least one first candidate feature point within a non-ground region of the first image includes: Feature point detection is performed on each object image area in the non-ground region of the first image to obtain the pixel coordinates of at least one first candidate feature point. Determining the pixel coordinates of at least one second candidate feature point in a non-ground region of the second image includes: Feature point detection is performed on each object image area in the non-ground region of the second image to obtain the pixel coordinates of at least one second candidate feature point.

9. The method according to claim 1, further comprising, after determining the matching relationship of at least one set of feature points: Obtain the first feature information of the first feature point in the first image and the second feature information of the second feature point in the second image in each group of feature points. The first feature point and the second feature point are two feature points in each group of feature points, and the first feature point comes from the first image and the second feature point comes from the second image. For any given set of feature points, if the first feature information of the first feature point and the second feature information of the second feature point in the set do not meet the conditions, then the set of feature points is deleted.

10. An image matching device, comprising: An image acquisition unit is used to acquire a first image and a second image; The image segmentation unit is used to segment the first image and the second image respectively, dividing each image into at least a first image region and a second image region; wherein the first image region and the second image region respectively represent the image regions of different spatial objects in each image, and the different spatial objects represent objects located in different spatial regions within the scene presented by the image; The first matching unit is used to establish at least one set of feature point matching relationships between the first image region in the first image and the first image region in the second image based on the matching method corresponding to the first image region; The second matching unit is used to establish a matching relationship of at least one set of feature points between the second image region in the first image and the second image region in the second image based on the matching method corresponding to the second image region.

Images