A method, apparatus, computer device, and storage medium for extracting homonyms.

Abstract

This invention discloses a method, apparatus, device, and storage medium for extracting corresponding points. The method includes: determining a video image and a 3D model based on image data and model data corresponding to a target region, respectively; aligning the 3D model and the video image; determining an image cache based on the 3D model aligned with the video image; determining a first set of feature points based on the image cache; determining a second set of feature points based on the video image; matching the first set of feature points and the second set of feature points to obtain an initial set of point pairs; removing noise from the initial set of point pairs to obtain a target set of point pairs; determining the 3D coordinates of the first target feature point in the 3D model; determining the 2D coordinates of the second target feature point in the video image; and determining a set of corresponding points based on the 3D coordinates of the first target feature point and the 2D coordinates of the second target feature point. This technical solution achieves automatic acquisition of a more accurate set of corresponding points, improving the efficiency of acquiring such a set.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, apparatus, computer device and storage medium for extracting corresponding points. Background Technology

[0002] Matching corresponding points in images is a core problem in computer graphics and photogrammetry. Accurate matching of corresponding points is a key technology in image processing, including image stitching and fusion, 3D reconstruction, and the identification of objects with the same name within images. The accuracy of matching directly affects the quality of image stitching and fusion, the precision of 3D reconstruction, and the accuracy of object identification. With the development of computer graphics, pattern recognition, and other technologies, corresponding point matching technology has also made significant progress.

[0003] In the existing technology, a first feature point is first determined in a three-dimensional image, and then a second feature point corresponding to the first feature point is manually determined in a two-dimensional image, thus defining the first feature point and the second feature point as a pair of corresponding points.

[0004] This method of extracting corresponding points has low extraction efficiency and a large error rate, making it difficult to efficiently obtain a large number of accurate corresponding points. Summary of the Invention

[0005] This invention provides a method, apparatus, computer device, and storage medium for extracting corresponding points, so as to quickly obtain a large number of accurate corresponding points.

[0006] In a first aspect, embodiments of the present invention provide a method for extracting corresponding points, including:

[0007] Based on the image data and model data corresponding to the target area, determine the video image and the 3D model respectively, and align the 3D model and the video image;

[0008] After determining the image cache based on the 3D model aligned with the video image, a first feature point set is determined according to the image cache, and a second feature point set is determined according to the video image. The first feature point set and the second feature point set are matched to obtain an initial point pair set.

[0009] The initial set of point pairs is subjected to noise removal to obtain a set of target point pairs, wherein the set of target point pairs includes at least one set of target point pairs, and the target point pairs include corresponding first target feature points and second target feature points;

[0010] The three-dimensional coordinates of the first target feature point in the three-dimensional model are determined, and the two-dimensional coordinates of the second target feature point in the video image are determined. Based on the three-dimensional coordinates of the first target feature point and the two-dimensional coordinates of the second target feature point, a set of corresponding points is determined.

[0011] The technical solution of this invention provides a method for extracting corresponding points, comprising: determining a video image and a 3D model based on image data and model data corresponding to a target region, and aligning the 3D model and the video image; determining an image cache based on the 3D model aligned with the video image, determining a first feature point set based on the image cache, determining a second feature point set based on the video image, and matching the first feature point set and the second feature point set to obtain an initial point pair set; removing noise from the initial point pair set to obtain a target point pair set, wherein the target point pair set includes at least one set of target point pairs, and each target point pair includes a first target feature point and a second target feature point that correspond to each other; determining the 3D coordinates of the first target feature point in the 3D model, determining the 2D coordinates of the second target feature point in the video image, and determining a corresponding point set based on the 3D coordinates of the first target feature point and the 2D coordinates of the second target feature point. The above technical solution, after acquiring the image data and model data corresponding to the target area, can further determine the video image corresponding to the image data and the 3D model corresponding to the model data. Then, it can adjust the 3D model to align it with the video image and determine the image cache when the 3D model is aligned with the video image. Furthermore, it can match the first set of feature points extracted from the image cache with the second set of feature points extracted from the video image to determine an initial point pair set. The initial point pair set includes at least one initial point pair, where each initial point pair includes corresponding first and second initial feature points. Due to extraction or matching errors, the initial point pair set may contain initial point pairs consisting of mismatched first and second initial feature points. Therefore, noise removal can be performed on the initial point pair set to delete the initial point pairs that do not match the first and second initial feature points, resulting in a target point pair set. The target point pair set includes at least one target point pair, which includes corresponding first and second target feature points. This allows the determination of the three-dimensional coordinates of the first target feature point in the three-dimensional model and the two-dimensional coordinates of the second target feature point in the video image, thus obtaining a set of corresponding points. This achieves automatic acquisition of the set of corresponding points, improving the efficiency of acquiring the set of corresponding points. Moreover, since the target point pair set removes the initial point pairs that do not match the first and second initial feature points, the accuracy of the corresponding point pairs determined based on the target point pair set is higher.

[0012] Furthermore, based on the image data and model data corresponding to the target region, the video images and 3D models are determined respectively, including:

[0013] Load the image data and model data corresponding to the target region into the rendering engine;

[0014] The rendering engine determines the video image corresponding to the image data and the 3D model corresponding to the model data.

[0015] Furthermore, aligning the 3D model and the video image includes:

[0016] The 3D model and the video image are aligned based on adjustment instructions triggered by the user in the rendering engine.

[0017] Further, a first feature point set is determined based on the image cache, a second feature point set is determined based on the video image, and an initial point pair set is obtained by matching the first feature point set and the second feature point set, including:

[0018] The first set of feature points is extracted from the image cache based on the feature extraction algorithm, and the second set of feature points is extracted from the video image;

[0019] The first feature point set and the second feature point set are matched based on the feature matching algorithm to obtain the initial point pair set.

[0020] Further, determining the video image corresponding to the image data and the 3D model corresponding to the model data based on the rendering engine includes:

[0021] The rendering engine displays the video image corresponding to the image data and the 3D model corresponding to the model data on two parallel interfaces.

[0022] Further, noise removal is performed on the initial set of point pairs to obtain the target set of point pairs, including:

[0023] In the rendering engine, the connecting lines of each group of first initial feature points and second initial feature points contained in the initial point pair set are determined, and the angle between the connecting lines and the preset standard line is determined.

[0024] Each of the included angles is placed into a histogram divided by angle. The target point pair set is determined based on the first initial feature point and the second initial feature point corresponding to each of the angles in the rectangle containing the most angles. The histogram includes at least one rectangle divided by angle.

[0025] Furthermore, the method also includes:

[0026] Depth buffer and camera parameters are determined based on a 3D model aligned with the video image;

[0027] Accordingly, determining the three-dimensional coordinates of the first target feature point in the three-dimensional model includes:

[0028] The three-dimensional coordinates of the first target feature point in the three-dimensional model are determined based on the depth cache and the camera parameters.

[0029] Secondly, embodiments of the present invention also provide a device for extracting corresponding points, comprising:

[0030] The first determining module is used to determine the video image and the 3D model based on the image data and model data corresponding to the target area, and to align the 3D model and the video image.

[0031] The extraction module is used to determine an image cache based on a 3D model aligned with the video image, determine a first feature point set based on the image cache, determine a second feature point set based on the video image, and match the first feature point set and the second feature point set to obtain an initial point pair set.

[0032] The noise removal module is used to remove noise from the initial set of point pairs to obtain a set of target point pairs, wherein the set of target point pairs includes at least one set of target point pairs, and the target point pairs include corresponding first target feature points and second target feature points.

[0033] The execution module is used to determine the three-dimensional coordinates of the first target feature point in the three-dimensional model, determine the two-dimensional coordinates of the second target feature point in the video image, and determine a set of corresponding points based on the three-dimensional coordinates of the first target feature point and the two-dimensional coordinates of the second target feature point.

[0034] Thirdly, embodiments of the present invention also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the same-name point extraction method as described in any of the first aspects.

[0035] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the same-point extraction method as described in any of the first aspects.

[0036] Fifthly, this application provides a computer program product including computer instructions that, when executed on a computer, cause the computer to perform the same-point extraction method as provided in the first aspect.

[0037] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the point extraction device, or it may be packaged separately from the processor of the point extraction device; this application does not impose any limitations on this.

[0038] The descriptions of the second, third, fourth, and fifth aspects in this application can be referred to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth, and fifth aspects can be referred to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0039] In this application, the name of the aforementioned corresponding point extraction device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.

[0040] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart of a method for extracting corresponding points provided in an embodiment of the present invention;

[0043] Figure 2 A flowchart of another method for extracting corresponding points provided in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of a corresponding point extraction device provided in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0046] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0047] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0048] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0049] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0050] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc. Moreover, embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0051] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0052] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0053] Figure 1 This is a flowchart illustrating a method for extracting corresponding points according to an embodiment of the present invention. This embodiment is applicable to situations requiring the rapid acquisition of a large number of accurate corresponding points. The method can be executed by a corresponding point extraction device, such as... Figure 1 As shown, the specific steps include the following:

[0054] Step 110: Determine the video image and 3D model based on the image data and model data corresponding to the target area, and align the 3D model and the video image.

[0055] Specifically, the target area can be photographed using the camera sensors included in the drone, obtaining corresponding photographic data. This photographic data can then be processed to obtain model data. The model data can then be loaded into a 3D rendering engine and displayed to obtain a 3D model of the target area. Alternatively, the target area can be photographed using a PTZ camera (a camera with omnidirectional gimbal movement and variable focal length, Pan / Tilt / Zoom), obtaining corresponding image data. This image data can then be loaded into a 3D rendering engine and displayed to obtain a video image of the target area.

[0056] In a 3D rendering engine, the viewpoint and size of the video image generated are fixed, while the viewpoint and size of the generated 3D model are adjustable. Therefore, after determining the video image and 3D model for the target area, the viewpoint and / or size of the 3D model can be adjusted until the size and viewpoint of the 3D model are nearly identical to those of the video image, thus aligning the 3D model and the video image.

[0057] In this embodiment of the invention, model data corresponding to the target area acquired by the camera sensors of the UAV and image data corresponding to the target area acquired by the PTZ camera are loaded into the 3D rendering engine. The 3D rendering engine can determine the 3D model corresponding to the target area based on the model data and determine the video image corresponding to the target area based on the image data. This allows for the alignment of the 3D model and the video image, facilitating the identification of corresponding points in the 3D model and the video image.

[0058] Step 120: After determining the image cache based on the 3D model aligned with the video image, determine the first feature point set according to the image cache, determine the second feature point set according to the video image, and match the first feature point set and the second feature point set to obtain an initial point pair set.

[0059] Specifically, after aligning the 3D model and the video image, the RGB cache of the 3D model aligned with the video image can be designated as the image cache. Next, first feature points can be extracted from the image cache, and a first feature point set can be determined based on the extracted first feature points. Second feature points can be extracted from the video image, and a second feature point set can be determined based on the extracted second feature points. Furthermore, the first and second feature point sets can be matched to obtain an initial set of point pairs. The initial set of point pairs includes at least one set of initial point pairs, and each initial point pair can include corresponding first and second initial feature points.

[0060] In this embodiment of the invention, after determining the image cache of the 3D model aligned with the video image, a first feature point can be extracted from the image cache and a first feature point set can be constructed. A second feature point set can be extracted from the video image and a second feature point set can be constructed. The first feature point set and the second feature point set are then matched to obtain an initial point pair set including at least one set of initial point pairs.

[0061] Step 130: Remove noise from the initial set of point pairs to obtain the target set of point pairs.

[0062] The target point pair set includes at least one set of target point pairs, wherein each target point pair includes a first target feature point and a second target feature point that correspond to each other.

[0063] Due to algorithm errors, the initial point pair set obtained by matching the first feature point set and the second feature point set may contain mismatched initial point pairs. Therefore, it is necessary to delete the mismatched initial point pairs.

[0064] Specifically, the matching degree of the first initial feature point and the second initial feature point contained in each initial point pair can be determined, and the initial point pairs consisting of the first initial feature point and the second initial feature point with a smaller matching degree can be deleted from the initial point pair set to achieve noise removal of the initial point pair set and obtain the target point pair set.

[0065] In this embodiment of the invention, the matching degree of the first initial feature point and the second initial feature point can be used to determine whether the first initial feature point and the second initial feature point match. When it is determined that the first initial feature point and the second initial feature point do not match, the initial point pair formed by the first initial feature point and the second initial feature point is deleted, thereby removing noise from the initial point pair set and obtaining the target point pair set.

[0066] Step 140: Determine the three-dimensional coordinates of the first target feature point in the three-dimensional model, determine the two-dimensional coordinates of the second target feature point in the video image, and determine a set of corresponding points based on the three-dimensional coordinates of the first target feature point and the two-dimensional coordinates of the second target feature point.

[0067] Specifically, when determining the second feature point set based on the video image, the two-dimensional coordinates of each second feature point contained in the second feature point set in the video image can be determined. When determining the first feature point set based on the image buffer, the pixel coordinates of each first feature point contained in the first feature point set in the image buffer can be determined. Therefore, further transformation of the pixel coordinates is required to obtain the three-dimensional coordinates of the first feature points in the three-dimensional model.

[0068] Specifically, when determining the image cache based on a 3D model aligned with the video image, the depth cache and camera parameters can also be determined. Then, the pixel coordinates of the first target feature point in the image cache can be transformed according to the depth cache and camera parameters to obtain the 3D coordinates of the first target feature point in the 3D model.

[0069] In this embodiment of the invention, after determining the set of target point pairs, the three-dimensional coordinates of the first target feature point in the three-dimensional model and the two-dimensional coordinates of the second target feature point in the video image can be further determined for each target point pair in the set of target point pairs.

[0070] The method for extracting corresponding points provided in this invention includes: determining a video image and a 3D model based on image data and model data corresponding to a target region, and aligning the 3D model and the video image; determining an image cache based on the 3D model aligned with the video image, determining a first set of feature points based on the image cache, determining a second set of feature points based on the video image, and matching the first set of feature points and the second set of feature points to obtain an initial set of point pairs; removing noise from the initial set of point pairs to obtain a set of target point pairs, wherein the set of target point pairs includes at least one set of target point pairs, and each target point pair includes a first target feature point and a second target feature point that correspond to each other; determining the 3D coordinates of the first target feature point in the 3D model, determining the 2D coordinates of the second target feature point in the video image, and determining a set of corresponding points based on the 3D coordinates of the first target feature point and the 2D coordinates of the second target feature point. The above technical solution, after acquiring the image data and model data corresponding to the target area, can further determine the video image corresponding to the image data and the 3D model corresponding to the model data. Then, it can adjust the 3D model to align it with the video image and determine the image cache when the 3D model is aligned with the video image. Furthermore, it can match the first set of feature points extracted from the image cache with the second set of feature points extracted from the video image to determine an initial point pair set. The initial point pair set includes at least one initial point pair, where each initial point pair includes corresponding first and second initial feature points. Due to extraction or matching errors, the initial point pair set may contain initial point pairs consisting of mismatched first and second initial feature points. Therefore, noise removal can be performed on the initial point pair set to delete the initial point pairs that do not match the first and second initial feature points, resulting in a target point pair set. The target point pair set includes at least one target point pair, which includes corresponding first and second target feature points. This allows the determination of the three-dimensional coordinates of the first target feature point in the three-dimensional model and the two-dimensional coordinates of the second target feature point in the video image, thus obtaining a set of corresponding points. This achieves automatic acquisition of the set of corresponding points, improving the efficiency of acquiring the set of corresponding points. Moreover, since the target point pair set removes the initial point pairs that do not match the first and second initial feature points, the accuracy of the corresponding point pairs determined based on the target point pair set is higher.

[0071] Figure 2 This is a flowchart illustrating another method for extracting corresponding points according to an embodiment of the present invention. This embodiment is a specific modification based on the above embodiments. Figure 2 As shown, in this embodiment, the method may further include:

[0072] Step 210: Determine the video image and 3D model based on the image data and model data corresponding to the target area.

[0073] In one embodiment, step 210 may specifically include:

[0074] The image data and model data corresponding to the target region are loaded into the rendering engine; the video image corresponding to the image data and the 3D model corresponding to the model data are determined based on the rendering engine.

[0075] In one implementation, determining the video image corresponding to the image data and the 3D model corresponding to the model data based on the rendering engine includes:

[0076] The rendering engine displays the video image corresponding to the image data and the 3D model corresponding to the model data on two parallel interfaces.

[0077] Specifically, after processing the photographic data corresponding to the target area acquired by the camera sensors of the drone, model data can be obtained. This model data can then be loaded into the 3D rendering engine and displayed on the first interface of the 3D rendering engine, resulting in a 3D model of the target area. Similarly, image data corresponding to the target area acquired by the PTZ camera is loaded into the 3D rendering engine and displayed on a second interface parallel to the first interface, resulting in a video image of the target area.

[0078] In this embodiment of the invention, the 3D rendering engine can determine the 3D model corresponding to the model data of the target area acquired by the camera sensors of the UAV, and can also determine the video image corresponding to the image data of the target area acquired by the PTZ camera. Of course, the 3D model and the video image can also be displayed on the first and second interfaces in parallel in the 3D rendering engine.

[0079] Step 220: Align the 3D model and the video image.

[0080] In one implementation, step 220 may specifically include:

[0081] The 3D model and the video image are aligned based on adjustment instructions triggered by the user in the rendering engine.

[0082] The adjustment command can be used to adjust the size and viewpoint of a 3D model.

[0083] Specifically, the viewpoint and size of the video image displayed in the second interface of the 3D rendering engine are fixed, while the viewpoint and size of the 3D model displayed in the first interface are adjustable. After receiving an adjustment command triggered by the user, the 3D rendering engine places the first interface displaying the 3D model on top of the second interface displaying the video image, and adjusts the viewpoint and / or size of the 3D model according to the adjustment command until the size and viewpoint of the 3D model are nearly identical to the size and viewpoint of the video image, thus aligning the 3D model and the video image.

[0084] In this embodiment of the invention, in the 3D rendering engine, based on the adjustment command triggered by the user, the viewpoint and / or size of the three-dimensional model displayed in the first interface of the 3D rendering engine can be adjusted so that the viewpoint and size of the three-dimensional model are approximately consistent with the viewpoint and size of the video image, and when the viewpoint and size of the three-dimensional model are approximately consistent with the viewpoint and size of the video image, the alignment of the three-dimensional model and the video image is determined.

[0085] Step 230: Determine the image buffer, depth buffer, and camera parameters based on the 3D model aligned with the video image.

[0086] Specifically, after aligning the 3D model and the video image, the 3D window color of the 3D model aligned with the video image, i.e., the RGB cache, or image cache, can be determined in the first interface of the 3D rendering engine. Simultaneously, the depth cache of each point in this image cache can be determined; the depth cache indicates the distance of each point in the image cache from the camera. Of course, the parameters of the camera that retrieves this image cache can also be determined, such as the camera's pose angle and position.

[0087] In this embodiment of the invention, after determining the three-dimensional model aligned with the video image, the image cache, depth cache, and camera parameters of the three-dimensional model at this time can be further determined to provide a data basis for determining the first target feature point.

[0088] Step 240: Extract a first set of feature points from the image cache, extract a second set of feature points from the video image, and match the first set of feature points and the second set of feature points to obtain an initial set of point pairs.

[0089] In one implementation, step 240 may specifically include:

[0090] The first set of feature points is extracted from the image cache based on the feature extraction algorithm, and the second set of feature points is extracted from the video image; the first set of feature points and the second set of feature points are matched based on the feature matching algorithm to obtain the initial set of point pairs.

[0091] Specifically, a first set of feature points can be extracted from an image cache based on the SIFT feature extraction operator, and a second set of feature points can be extracted from a video image. Alternatively, the first and second set of feature points can be matched based on the SIFT feature matching operator to obtain an initial set of point pairs.

[0092] In this embodiment of the invention, after determining the image cache of the 3D model aligned with the video image, a first feature point can be extracted from the image cache and a first feature point set can be constructed. A second feature point set can be extracted from the video image and a second feature point set can be constructed. The first feature point set and the second feature point set are then matched to obtain an initial point pair set including at least one set of initial point pairs.

[0093] Step 250: Remove noise from the initial set of point pairs to obtain the target set of point pairs.

[0094] The target point pair set includes at least one set of target point pairs, and each target point pair includes a first target feature point and a second target feature point that correspond to each other.

[0095] In one implementation, step 250 may specifically include:

[0096] In the rendering engine, the connecting lines of each group of first initial feature points and second initial feature points contained in the initial point pair set are determined, and the angle between the connecting lines and the preset standard line is determined; each angle is placed in a histogram divided by angle, and the target point pair set is determined according to the first initial feature points and second initial feature points corresponding to each angle in the rectangle containing the most angles, wherein the histogram includes at least one rectangle divided by angle.

[0097] The preset standard line can be the bottom edge of the first interface and the second interface.

[0098] Specifically, the 3D rendering engine can display the video image and the image cache corresponding to the 3D model in two parallel interfaces respectively. In order to determine whether the first initial feature point and the second initial feature point contained in each initial point pair in the initial point pair set correspond, the connecting line between the first initial feature point and the second initial feature point contained in each initial point pair can be determined first, and the angle between the connecting line and the bottom edge of the display interface can be determined. The angle distribution is statistically analyzed, and the initial point pairs formed by the first initial feature point and the second initial feature point with a small matching degree are deleted from the initial point pair set according to the statistical information. This achieves noise removal of the initial point pair set and obtains the target point pair set.

[0099] Specifically, each angle can be placed into a histogram interval divided by angle. If the previously determined angle is not in the histogram interval with the most points, it is determined that the first initial feature point and the second initial feature point corresponding to that angle do not match. Then, the initial point pair formed by the first initial feature point and the second initial feature point corresponding to that angle can be deleted to remove noise from the set of initial point pairs.

[0100] In practical applications, firstly, the 360° is divided into 72 equal parts, with each 5° portion serving as a rectangular frame for a histogram, thus establishing the histogram. Secondly, the rectangular frames to which the previously determined angles belong are identified, and the target point pair set is determined based on the first and second initial feature points corresponding to each angle in the rectangular frame containing the most angles. Simultaneously, the first and second initial feature points corresponding to each angle in other rectangular frames are removed, achieving noise removal from the initial point pair set.

[0101] In this embodiment of the invention, the connecting line between the first initial feature point and the second initial feature point contained in each initial point pair in the 3D rendering engine is determined, and the angle between the connecting line and the bottom edge of the display interface is determined. Then, by determining the rectangle corresponding to the angle in the histogram, the rectangle containing the most angles is determined, and the target point pair set is determined according to the first initial feature point and the second initial feature point corresponding to each angle in the rectangle containing the most angles. This achieves noise removal of the initial point pair set and obtains the target point pair set.

[0102] Step 260: Determine the three-dimensional coordinates of the first target feature point in the three-dimensional model, determine the two-dimensional coordinates of the second target feature point in the video image, and determine a set of corresponding points based on the three-dimensional coordinates of the first target feature point and the two-dimensional coordinates of the second target feature point.

[0103] In one embodiment, determining the three-dimensional coordinates of the first target feature point in the three-dimensional model includes:

[0104] The three-dimensional coordinates of the first target feature point in the three-dimensional model are determined based on the depth cache and the camera parameters.

[0105] As described in Embodiment 1 above, when determining the second feature point set based on the video image, the two-dimensional coordinates of each second feature point contained in the second feature point set in the video image can be determined. Therefore, the two-dimensional coordinates of the second target feature point in the video image can be determined. When determining the first feature point set based on the image cache, the pixel coordinates of each first feature point contained in the first feature point set in the image cache can be determined. Therefore, the pixel coordinates of the first target feature point in the image cache can be determined, and then the pixel coordinates of the first target feature point in the image cache can be transformed according to the depth cache and camera parameters to obtain the three-dimensional coordinates of the first target feature point in the three-dimensional model.

[0106] In this embodiment of the invention, after determining the target point set, the three-dimensional coordinates of the first target feature point in the three-dimensional model and the two-dimensional coordinates of the second target feature point in the video image can be further determined for each target point pair in the target point set.

[0107] The method for extracting corresponding points provided in this invention includes: determining a video image and a 3D model based on image data and model data corresponding to a target region, respectively; aligning the 3D model and the video image; determining an image cache, a depth cache, and camera parameters based on the 3D model aligned with the video image; extracting a first set of feature points from the image cache, extracting a second set of feature points from the video image, and matching the first set of feature points and the second set of feature points to obtain an initial set of point pairs; removing noise from the initial set of point pairs to obtain a target set of point pairs; determining the 3D coordinates of the first target feature point in the 3D model, determining the 2D coordinates of the second target feature point in the video image, and determining a set of corresponding points based on the 3D coordinates of the first target feature point and the 2D coordinates of the second target feature point. The above technical solution, after acquiring the image data and model data corresponding to the target region, can further determine the video image corresponding to the image data and the 3D model corresponding to the model data. Then, the 3D model can be adjusted to align with the video image, and the image cache, depth cache, and camera parameters when the 3D model is aligned with the video image can be determined. Furthermore, the first feature point set extracted from the image cache and the second feature point set extracted from the video image can be matched to determine an initial point pair set. The initial point pair set includes at least one initial point pair, which includes corresponding first and second initial feature points. Due to extraction or matching errors, the initial point pair set may contain mismatched initial point pairs consisting of first and second initial feature points. Therefore, noise removal can be performed on the initial point pair set to delete the mismatched initial point pairs, resulting in the target point pair set. The punctuation pair set includes at least one set of target point pairs, each containing a corresponding first target feature point and a second target feature point. When determining the first feature point based on the image cache, the pixel coordinates of the first feature point in the image cache can be determined, thus determining the pixel coordinates of the first target feature point in the image cache. Therefore, coordinate transformation can be performed on the pixel coordinates of the first target feature point in the image cache based on the depth cache and camera parameters to obtain the three-dimensional coordinates of the first target feature point in the three-dimensional model. The two-dimensional coordinates of the second target feature point in the video image can also be determined. Then, based on the first target feature point and its three-dimensional coordinates, and the second target feature point and its two-dimensional coordinates, a set of corresponding points is determined, realizing automatic acquisition of the set of corresponding points and improving the efficiency of acquiring the set of corresponding points. Moreover, since the initial point pairs consisting of mismatched first and second initial feature points are removed from the target point pair set, the accuracy of the corresponding point pairs determined based on the target point pair set is higher.

[0108] Figure 3This is a schematic diagram of a corresponding point extraction device provided in an embodiment of the present invention. This device is suitable for situations requiring the rapid acquisition of a large number of accurate corresponding points. The device can be implemented through software and / or hardware and is generally integrated into a computer device.

[0109] like Figure 3 As shown, the device includes:

[0110] The first determining module 310 is used to determine the video image and the three-dimensional model according to the image data and model data corresponding to the target area, and to align the three-dimensional model and the video image.

[0111] The extraction module 320 is used to determine an image cache based on a 3D model aligned with the video image, determine a first feature point set based on the image cache, determine a second feature point set based on the video image, and match the first feature point set and the second feature point set to obtain an initial point pair set.

[0112] The noise removal module 330 is used to remove noise from the initial point pair set to obtain a target point pair set, wherein the target point pair set includes at least one set of target point pairs, and the target point pairs include corresponding first target feature points and second target feature points.

[0113] The execution module 340 is used to determine the three-dimensional coordinates of the first target feature point in the three-dimensional model, determine the two-dimensional coordinates of the second target feature point in the video image, and determine a set of corresponding points based on the three-dimensional coordinates of the first target feature point and the two-dimensional coordinates of the second target feature point.

[0114] The corresponding point extraction device provided in this embodiment determines a video image and a 3D model based on image data and model data corresponding to the target region, respectively, and aligns the 3D model and the video image; after determining an image cache based on the 3D model aligned with the video image, a first feature point set is determined based on the image cache, and a second feature point set is determined based on the video image; the first feature point set and the second feature point set are matched to obtain an initial point pair set; noise is removed from the initial point pair set to obtain a target point pair set, wherein the target point pair set includes at least one set of target point pairs, and the target point pairs include corresponding first target feature points and second target feature points; the 3D coordinates of the first target feature points in the 3D model are determined, the 2D coordinates of the second target feature points in the video image are determined, and a corresponding point set is determined based on the 3D coordinates of the first target feature points and the 2D coordinates of the second target feature points. The above technical solution, after acquiring the image data and model data corresponding to the target area, can further determine the video image corresponding to the image data and the 3D model corresponding to the model data. Then, it can adjust the 3D model to align it with the video image and determine the image cache when the 3D model is aligned with the video image. Furthermore, it can match the first set of feature points extracted from the image cache with the second set of feature points extracted from the video image to determine an initial point pair set. The initial point pair set includes at least one initial point pair, where each initial point pair includes corresponding first and second initial feature points. Due to extraction or matching errors, the initial point pair set may contain initial point pairs consisting of mismatched first and second initial feature points. Therefore, noise removal can be performed on the initial point pair set to delete the initial point pairs that do not match the first and second initial feature points, resulting in a target point pair set. The target point pair set includes at least one target point pair, which includes corresponding first and second target feature points. This allows the determination of the three-dimensional coordinates of the first target feature point in the three-dimensional model and the two-dimensional coordinates of the second target feature point in the video image, thus obtaining a set of corresponding points. This achieves automatic acquisition of the set of corresponding points, improving the efficiency of acquiring the set of corresponding points. Moreover, since the target point pair set removes the initial point pairs that do not match the first and second initial feature points, the accuracy of the corresponding point pairs determined based on the target point pair set is higher.

[0115] Based on the above embodiments, the first determining module 310 is specifically used for:

[0116] Load the image data and model data corresponding to the target region into the rendering engine;

[0117] Based on the rendering engine, the video image corresponding to the image data and the 3D model corresponding to the model data are determined;

[0118] The 3D model and the video image are aligned based on adjustment instructions triggered by the user in the rendering engine.

[0119] In one implementation, determining the video image corresponding to the image data and the 3D model corresponding to the model data based on the rendering engine includes:

[0120] The rendering engine displays the video image corresponding to the image data and the 3D model corresponding to the model data on two parallel interfaces.

[0121] Based on the above embodiments, the extraction module 320 is specifically used for:

[0122] Image buffering is determined based on a 3D model aligned with the size and viewpoint of the video image;

[0123] The first set of feature points is extracted from the image cache based on the feature extraction algorithm, and the second set of feature points is extracted from the video image;

[0124] The first feature point set and the second feature point set are matched based on the feature matching algorithm to obtain the initial point pair set.

[0125] Based on the above embodiments, the elimination module 330 is specifically used for:

[0126] In the rendering engine, the connecting lines of each group of first initial feature points and second initial feature points contained in the initial point pair set are determined, and the angle between the connecting lines and the preset standard line is determined.

[0127] Each of the included angles is placed into a histogram divided by angle. The target point pair set is determined based on the first initial feature point and the second initial feature point corresponding to each of the angles in the rectangle containing the most angles. The histogram includes at least one rectangle divided by angle.

[0128] Based on the above embodiments, the device further includes:

[0129] The second determining module is used to determine the depth buffer and camera parameters based on a 3D model aligned with the video image;

[0130] Accordingly, execution module 340 is specifically used for:

[0131] The three-dimensional coordinates of the first target feature point in the three-dimensional model are determined based on the depth cache and the camera parameters.

[0132] Determine the two-dimensional coordinates of the second target feature point in the video image;

[0133] A set of points with the same name is determined based on the three-dimensional coordinates of the first target feature point and the two-dimensional coordinates of the second target feature point.

[0134] The corresponding point extraction device provided in the embodiments of the present invention can execute the corresponding point extraction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0135] It is worth noting that in the embodiments of the above-mentioned homonym extraction device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0136] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 4 A block diagram of an exemplary computer device 4 suitable for implementing embodiments of the present invention is shown. Figure 4 The computer device 4 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0137] like Figure 4 As shown, the computer device 4 is represented in the form of a general-purpose computing electronic device. The components of the computer device 4 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0138] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0139] Computer device 4 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 4, including volatile and non-volatile media, removable and non-removable media.

[0140] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 4 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0141] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0142] Computer device 4 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with computer device 4, and / or with any device that enables computer device 4 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, computer device 4 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 20. Figure 4 As shown, network adapter 20 communicates with other modules of computer device 4 via bus 18. It should be understood that, although... Figure 4 Not shown, it can be used in conjunction with computer device 4 with other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0143] Processing unit 16 executes various functional applications and page displays by running programs stored in system memory 28, such as implementing the same-name point extraction method provided in the embodiments of the present invention, which includes:

[0144] Based on the image data and model data corresponding to the target area, determine the video image and the 3D model respectively, and align the 3D model and the video image;

[0145] After determining the image cache based on the 3D model aligned with the video image, a first feature point set is determined based on the image cache, and a second feature point set is determined based on the video image. The first feature point set and the second feature point set are then matched to obtain an initial set of point pairs.

[0146] The initial set of point pairs is subjected to noise removal to obtain a set of target point pairs, wherein the set of target point pairs includes at least one set of target point pairs, and the target point pairs include corresponding first target feature points and second target feature points;

[0147] The three-dimensional coordinates of the first target feature point in the three-dimensional model are determined, and the two-dimensional coordinates of the second target feature point in the video image are determined. Based on the three-dimensional coordinates of the first target feature point and the two-dimensional coordinates of the second target feature point, a set of corresponding points is determined.

[0148] Of course, those skilled in the art will understand that the processor can also implement the technical solution of the corresponding point extraction method provided in any embodiment of the present invention.

[0149] This invention provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements, for example, the same-name point extraction method provided in this invention, which includes:

[0150] Based on the image data and model data corresponding to the target area, determine the video image and the 3D model respectively, and align the 3D model and the video image;

[0151] After determining the image cache based on the 3D model aligned with the video image, a first feature point set is determined based on the image cache, and a second feature point set is determined based on the video image. The first feature point set and the second feature point set are then matched to obtain an initial set of point pairs.

[0152] The initial set of point pairs is subjected to noise removal to obtain a set of target point pairs, wherein the set of target point pairs includes at least one set of target point pairs, and the target point pairs include corresponding first target feature points and second target feature points;

[0153] The three-dimensional coordinates of the first target feature point in the three-dimensional model are determined, and the two-dimensional coordinates of the second target feature point in the video image are determined. Based on the three-dimensional coordinates of the first target feature point and the two-dimensional coordinates of the second target feature point, a set of corresponding points is determined.

[0154] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0155] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0156] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0157] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0158] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0159] Furthermore, the acquisition, storage, use, and processing of data in the technical solution of this invention all comply with the relevant provisions of national laws and regulations.

[0160] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for extracting corresponding points, characterized in that, include: Load the image and model data corresponding to the target region into the rendering engine; Based on the rendering engine, the video image corresponding to the image data and the 3D model corresponding to the model data are determined; Align the 3D model and the video image based on the adjustment instructions triggered by the user in the rendering engine; Image buffer, depth buffer, and camera parameters are determined based on a 3D model aligned with the video image; wherein the image buffer is an RGB buffer of the 3D model aligned with the video image; and the depth buffer is the distance of each point in the image buffer from the camera. A first feature point set is determined based on the image cache, a second feature point set is determined based on the video image, and the first feature point set and the second feature point set are matched to obtain an initial point pair set; The initial set of point pairs is subjected to noise removal to obtain a set of target point pairs, wherein the set of target point pairs includes at least one set of target point pairs, and the target point pairs include corresponding first target feature points and second target feature points; Based on the depth cache and the camera parameters, the three-dimensional coordinates of the first target feature point in the three-dimensional model are determined, the two-dimensional coordinates of the second target feature point in the video image are determined, and a set of corresponding points is determined based on the three-dimensional coordinates of the first target feature point and the two-dimensional coordinates of the second target feature point.

2. The method for extracting corresponding points according to claim 1, characterized in that, A first feature point set is determined based on the image cache, a second feature point set is determined based on the video image, and an initial point pair set is obtained by matching the first feature point set and the second feature point set, including: The first set of feature points is extracted from the image cache based on the feature extraction algorithm, and the second set of feature points is extracted from the video image; The first feature point set and the second feature point set are matched based on the feature matching algorithm to obtain the initial point pair set.

3. The method for extracting corresponding points according to claim 1, characterized in that, Determining the video image corresponding to the image data and the 3D model corresponding to the model data based on the rendering engine includes: The rendering engine displays the video image corresponding to the image data and the 3D model corresponding to the model data on two parallel interfaces.

4. The method for extracting corresponding points according to claim 3, characterized in that, The initial set of point pairs is subjected to noise removal to obtain the target set of point pairs, including: In the rendering engine, the connecting lines of each group of first initial feature points and second initial feature points contained in the initial point pair set are determined, and the angle between the connecting lines and the preset standard line is determined. Each of the included angles is placed into a histogram divided by angle. The target point pair set is determined based on the first initial feature point and the second initial feature point corresponding to each of the angles in the rectangle containing the most angles. The histogram includes at least one rectangle divided by angle.

5. A device for extracting homonymous points, characterized in that, include: The first determining module is used to load the image data and model data corresponding to the target area into the rendering engine; Based on the rendering engine, the video image corresponding to the image data and the 3D model corresponding to the model data are determined; Align the 3D model and the video image based on the adjustment instructions triggered by the user in the rendering engine; The second determining module is used to determine image cache, depth cache, and camera parameters based on a 3D model aligned with the video image; wherein, the image cache is an RGB cache of the 3D model aligned with the video image; and the depth cache is the distance of each point in the image cache from the camera. The extraction module is used to determine a first feature point set based on the image cache, determine a second feature point set based on the video image, and match the first feature point set and the second feature point set to obtain an initial point pair set; The noise removal module is used to remove noise from the initial set of point pairs to obtain a set of target point pairs, wherein the set of target point pairs includes at least one set of target point pairs, and the target point pairs include corresponding first target feature points and second target feature points. An execution module is configured to determine the three-dimensional coordinates of the first target feature point in the three-dimensional model based on the depth cache and the camera parameters, determine the two-dimensional coordinates of the second target feature point in the video image, and determine a set of corresponding points based on the three-dimensional coordinates of the first target feature point and the two-dimensional coordinates of the second target feature point.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the same-name point extraction method as described in any one of claims 1-4.

7. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the corresponding point extraction method as described in any one of claims 1-4.

Images