A method for matching points of the same name and a hardware module for matching points of the same name
In the binocular vision detection system, by utilizing the distance relationship between the feature points and the pole lines in the feature area, the intersection of the connecting line and the pole lines on both sides of the pole lines and the pole lines are calculated as the point of the same name, which solves the problem of large calculation amount and low efficiency in the prior art, and achieves efficient matching of the same name point.
Patent Information
- Application Number
- CN202411947090.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-04-04
AI Technical Summary
In the prior art, when using polar line constraint relationships to match points of the same name, the calculation is large, time-consuming, and high-performance hardware equipment is required, resulting in low efficiency.
In the binocular vision detection system, the point closest to the two sides of the pole line is found by using the horizontal or longitudinal distance of the feature points in the feature area to the pole line, and the intersection point of its connecting line and the pole line is calculated as the point of the same name.
It greatly reduces the amount of calculation and improves the computing speed, and can achieve efficient matching of points of the same name under low hardware resources.
Smart Images
Figure CN119762819A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to a homonymous point matching method and a homonymous point matching hardware module. Background Art
[0002] Homophone point matching usually refers to finding the pixel position representing the same physical scene point in two images. It is a basic link in the field of photogrammetry and machine vision. The accuracy and efficiency of its matching directly affect the performance of image processing in practical applications.
[0003] According to the epipolar constraint relationship, that is, the same-name points of feature points on one image exist on the corresponding epipolar line of another image, the epipolar constraint method can be used to match the same-name points. However, in actual situations, due to image noise, imperfect feature extraction and matching, the same-name points may not fall completely on the epipolar line. Therefore, the point with the closest Euclidean distance to the epipolar line is usually found near the epipolar line as the matching point.
[0004] In the prior art, epipolar constraints are used to search for points of the same name. Although the search space is reduced from the entire two-dimensional image to the one-dimensional epipolar line, which reduces some of the calculations, the calculation of the Euclidean distance is too large, the calculation takes a long time, and requires high-quality hardware equipment to support it, resulting in low efficiency in matching points of the same name. Summary of the invention
[0005] In order to solve the above technical problems, the present invention provides a homonymous point matching method and a homonymous point matching hardware module, which are suitable for homonymous point matching in a binocular vision detection system, can greatly reduce the amount of calculation and improve the calculation speed.
[0006] To this end, the technical solution of the present invention is as follows:
[0007] A method for matching homonymous points, wherein the homonymous points are imaging points of the same point in space respectively taken by two cameras in a binocular vision detection system, and the imaging points are subject to epipolar constraints; the images obtained by the two cameras taking images of the same object to be detected are respectively referred to as a first image and a second image;
[0008] The steps include:
[0009] S1. Perform image segmentation on the first image and the second image respectively to obtain a plurality of mutually corresponding feature regions; at the same time, perform feature point extraction on the first image and the second image, and mark the extracted feature points according to the feature regions where they are located;
[0010] S2, a certain point to be matched with the same name in the first image is recorded as point A, the image coordinates of point A are obtained, the feature region where it is located is marked as feature region A, and the corresponding region of feature region A is searched in the second image, recorded as feature region A';
[0011] S3, using the relationship between the two cameras in the binocular vision detection system, obtaining the epipolar line II corresponding to point A on the second image;
[0012] Taking the A' feature area as the calculation area, obtain the lateral distance or longitudinal distance between each feature point in the A' feature area and the polar line II in the image coordinate system of the second image; take a point closest to the polar line II on both sides, respectively, and record them as point P and point Q; the intersection of the line connecting point P and point Q and the polar line II is the point of the same name as point A.
[0013] Further, the method of obtaining the lateral distance or longitudinal distance of each feature point in the A' feature area from the epipolar line II in the image coordinate system of the second image in step S3 is:
[0014] ① Obtain the image coordinates of each feature point in the A' feature area and the equation of the epipolar line II;
[0015] ② When calculating the horizontal distance, substitute the longitudinal coordinate value of the feature point into the equation of the polar line II, calculate the horizontal coordinate of the corresponding point on the polar line, and then calculate |the horizontal coordinate of the feature point - the horizontal coordinate of the corresponding point on the polar line|, which is the horizontal distance;
[0016] When calculating the longitudinal distance, substitute the horizontal coordinate value of the feature point into the equation of the polar line II, and calculate the longitudinal coordinate of the corresponding point on the polar line. The result is |longitudinal coordinate of the feature point - longitudinal coordinate of the corresponding point on the polar line|, which is the longitudinal distance.
[0017] Further, let the coordinates of the intersection of the line connecting point P and point Q obtained in step S3 and the polar line II be (X, Y), and let the equation of the polar line II be: Ax+By+C=0;
[0018] The calculation method of X and Y is:
[0019]
[0020] Among them, x1 and x2 are the horizontal coordinates of point P and point Q respectively; m is the longitudinal distance between point Q and polar line II; n is the longitudinal distance between point P and polar line II;
[0021] or:
[0022] Among them, y1 and y2 are the ordinates of point P and point Q respectively; m is the lateral distance between point Q and polar line II; n is the lateral distance between point P and polar line II.
[0023] A hardware module for matching homonymous points is applied to a binocular vision detection system. The images obtained by two cameras respectively capturing images of the same object to be detected are referred to as a first image and a second image, and the image where the homonymous points to be matched are referred to as the first image. There are epipolar constraints on the imaging points of the two cameras at the same point in space.
[0024] It includes image segmentation module, feature point extraction module, feature point area search module and homonymous point calculation module;
[0025] The image segmentation module is used to segment the first image and the second image into a plurality of mutually corresponding feature regions, and transmit the regions to the feature point extraction module and the feature point region search module;
[0026] The feature point extraction module extracts feature points from the first image and the second image respectively, and marks each feature point and the feature area where it is located;
[0027] The feature point region search module is used to search the feature region where the to-be-matched homonymous points are located in the second image, and transmit the result to the homonymous point calculation module;
[0028] The homonymous point calculation module includes an epipolar line calculation module, a distance calculation module and an intersection calculation module which are electrically connected; the epipolar line calculation module is used to obtain the epipolar line equation corresponding to the homonymous point to be matched in the second image, and can substitute the horizontal coordinate or vertical coordinate of each feature point in the feature area corresponding to the homonymous point to be matched on the second image into the epipolar line equation to obtain the vertical coordinate or horizontal coordinate of the corresponding point on the epipolar line; the distance calculation module is used to calculate |the horizontal coordinate of the feature point-the horizontal coordinate of the corresponding point on the epipolar line|, or the vertical coordinate of the feature point-the vertical coordinate of the corresponding point on the epipolar line|, and respectively select the points with the smallest values on both sides of the epipolar line, and transmit the results to the intersection calculation module; the intersection calculation module is used to connect the points with the smallest values on both sides of the received epipolar line, and obtain the intersection of the line connecting the two points and the epipolar line, and the intersection is the homonymous point of the homonymous point to be matched.
[0029] Further, the epipolar equation obtained by the epipolar calculation module is Ax+By+C=0;
[0030] The points with the smallest values on both sides of the polar line obtained by the distance calculation module are recorded as point P and point Q;
[0031] The intersection calculation module obtains the coordinates of the intersection as (X, Y), and the method for calculating X, Y by the intersection calculation module is:
[0032]
[0033] Wherein, x1 and x2 are the horizontal coordinates of point P and point Q respectively; m is the longitudinal distance between point Q and the polar line; n is the longitudinal distance between point P and the polar line; the data of m and n are obtained from the polar line calculation module;
[0034] or:
[0035] Among them, y1 and y2 are the ordinates of point P and point Q respectively; m is the lateral distance between point Q and the polar line; n is the lateral distance between point P and the polar line; the data of m and n are obtained from the polar line calculation module.
[0036] The technical solution provided by the present invention is preferably applicable to FPGA, which can reduce more than 75% of DSP resources in the calculation process of same-name point matching in a binocular vision detection system. Combined with the parallel computing characteristics of FPGA, the calculation speed of an FPGA chip with 500 hardware DSP resources can be increased by more than 4 times. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 It is a structural diagram of the hardware module for matching the same-name points of the present invention. DETAILED DESCRIPTION
[0038] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0039] A method for matching homonymous points, wherein the homonymous points are imaging points of the same point in space respectively taken by two cameras in a binocular vision detection system, and the imaging points have epipolar constraints; the images obtained by the two cameras taking images of the same object to be detected are respectively referred to as a first image and a second image;
[0040] The steps include:
[0041] S1. Perform image segmentation on the first image and the second image respectively to obtain a plurality of mutually corresponding feature regions; at the same time, perform feature point extraction on the first image and the second image, and mark the extracted feature points according to the feature regions where they are located;
[0042] S2, a certain point to be matched with the same name in the first image is recorded as point A, the image coordinates of point A are obtained, the feature region where it is located is marked as feature region A, and the corresponding region of feature region A is searched in the second image, recorded as feature region A';
[0043] S3, using the relationship between the two cameras in the binocular vision detection system, obtaining the epipolar line II corresponding to point A on the second image;
[0044] Taking the A' feature area as the calculation area, obtain the lateral distance or longitudinal distance between each feature point in the A' feature area and the polar line II in the image coordinate system of the second image; take a point closest to the polar line II on both sides, respectively, and record them as point P and point Q; the intersection of the line connecting point P and point Q and the polar line II is the point of the same name as point A.
[0045] In specific implementation, the method of obtaining the horizontal distance or vertical distance between each feature point in the A' feature area and the polar line II in the image coordinate system of the second image in step S3 is:
[0046] ① Obtain the image coordinates of each feature point in the A' feature area and the equation of the epipolar line II;
[0047] ② When calculating the horizontal distance, substitute the longitudinal coordinate value of the feature point into the equation of the polar line II, calculate the horizontal coordinate of the corresponding point on the polar line, and then calculate |the horizontal coordinate of the feature point - the horizontal coordinate of the corresponding point on the polar line|, which is the horizontal distance;
[0048] When calculating the longitudinal distance, substitute the horizontal coordinate value of the feature point into the equation of the polar line II, and calculate the longitudinal coordinate of the corresponding point on the polar line. The result is |longitudinal coordinate of the feature point - longitudinal coordinate of the corresponding point on the polar line|, which is the longitudinal distance.
[0049] The equation of polar line II is recorded as Ax+By+C=0. The conventional method is to calculate the distance from a point outside the line to the straight line as Then the minimum DSP resources consumed in the calculation process are 27, specifically: 4 multiplication IP cores (A×x, A×A, B×B, the product of the numerator and the reciprocal of the square root), 2 addition IP cores (A 2 +B 2 , Ax+By), 1 multiplication and addition IP core (By+C), 1 square root reciprocal IP core, and 1 absolute value IP core. (Each multiplication IP core consumes 3 DSP resources; the addition / subtraction IP core consumes 2 DSP resources, the square root reciprocal IP core consumes 9 DSP resources; the multiplication and addition / multiplication and subtraction IP cores consume 2 DSP resources, the absolute value IP core consumes 0 DSP resources; the division IP core consumes 0 DSP resources.)
[0050] The technical solution provided by the present application only needs to calculate the horizontal distance or vertical distance from the point (x0, y0) to the polar line II, and the formula used can be or The calculation process consumes at least 4 DSP resources, including 1 multiplication and subtraction IP core (-C-BY or -C-AX), 1 subtraction IP core (x0- or y0-), 1 division IP core ((-C-BY) ÷ A or (-C-AX) ÷ B), and 1 absolute value IP core.
[0051] In specific implementation, the coordinates of the intersection of the line connecting point P and point Q obtained in step S3 and the polar line II are assumed to be (X, Y), and the equation of the polar line II is recorded as: Ax+By+C=0;
[0052] The calculation method of X and Y is:
[0053]
[0054] Among them, x1 and x2 are the horizontal coordinates of point P and point Q respectively; m is the longitudinal distance between point Q and polar line II; n is the longitudinal distance between point P and polar line II;
[0055] or:
[0056] Among them, y1 and y2 are the ordinates of point P and point Q respectively; m is the lateral distance between point Q and polar line II; n is the lateral distance between point P and polar line II.
[0057] In the above process of obtaining the intersection point, the applicable DSP resource consumption is at least 12, including 2 addition IP cores, 1 multiplication and subtraction IP core (-C-BY or -C-AX), 2 division IP cores, and 2 multiplication IP cores.
[0058] The method of calculating the intersection point (x, y) by conventional methods is to assume that the polar line equation is known to be A1X+B1Y+C1=0;
[0059] The equation of the fitted straight line based on the two closest points is A2X+B2Y+C2=0, where:
[0060] A2 = y2-y1; B2 = x1-x2; C2 = x1 y2-x2y1; (x1, y1) are the coordinates of one nearest point, (x2, y2) are the coordinates of the other nearest point;
[0061] The coordinates of the intersection point are:
[0062] x=B2C1-B1C2A1B2-A2B1; y=A1C2-A2C1A1B2-A2B1
[0063] The calculation process consumes at least 44 DSP resources, including 6 subtraction IP cores, 10 multiplication IP cores, 2 division IP cores, and 1 addition IP core.
[0064] In summary, the existing method for obtaining the intersection requires 27+44=71 DSP resources, while the method provided by the present invention only requires 4+12=16 DSP resources, saving a total of 55 DSP resources, that is, the DSP resource consumption is reduced by 77%; due to the parallel characteristics of FPGA, the improvement of computing speed is directly related to the total number of DSPs allocated by the algorithm and the DSP resources consumed by a single channel operation, and the relationship is as follows: assuming that the number of hardware DSP resources in a certain FPGA chip is 500 (the DSP resources of common FPGA chips range from hundreds to thousands), according to the original scheme 500 / 71=7.04, there are at most 7 parallel channels; while using the method provided by the present invention, 500 / 16=31.25, there are at most 31 parallel channels. In summary, its computing speed will be 4.43 times that of the original scheme, and the more DSP resources are allocated, the more obvious this improvement will be.
[0065] A hardware module for matching homonymous points is applied to a binocular vision detection system. The images obtained by two cameras respectively capturing images of the same object to be detected are referred to as a first image and a second image, and the image where the homonymous points to be matched are referred to as the first image. There are epipolar constraints on the imaging points of the two cameras at the same point in space.
[0066] It includes image segmentation module, feature point extraction module, feature point area search module and homonymous point calculation module;
[0067] The image segmentation module is used to segment the first image and the second image into a plurality of mutually corresponding feature regions, and transmit the regions to the feature point extraction module and the feature point region search module;
[0068] The feature point extraction module extracts feature points from the first image and the second image respectively, and marks each feature point and the feature area where it is located;
[0069] The feature point region search module is used to search the feature region where the to-be-matched homonymous points are located in the second image, and transmit the result to the homonymous point calculation module;
[0070] The homonymous point calculation module includes an epipolar line calculation module, a distance calculation module and an intersection calculation module which are electrically connected; the epipolar line calculation module is used to obtain the epipolar line equation corresponding to the homonymous point to be matched in the second image, and can substitute the horizontal coordinate or vertical coordinate of each feature point in the feature area corresponding to the homonymous point to be matched on the second image into the epipolar line equation to obtain the vertical coordinate or horizontal coordinate of the corresponding point on the epipolar line; the distance calculation module is used to calculate |the horizontal coordinate of the feature point-the horizontal coordinate of the corresponding point on the epipolar line|, or the vertical coordinate of the feature point-the vertical coordinate of the corresponding point on the epipolar line|, and respectively select the points with the smallest values on both sides of the epipolar line, and transmit the results to the intersection calculation module; the intersection calculation module is used to connect the points with the smallest values on both sides of the received epipolar line, and obtain the intersection of the line connecting the two points and the epipolar line, and the intersection is the homonymous point of the homonymous point to be matched.
[0071] Further, the polar line equation obtained by the polar line calculation module is Ax+By+C=0;
[0072] The points with the smallest values on both sides of the polar line obtained by the distance calculation module are recorded as point P and point Q;
[0073] The intersection calculation module obtains the coordinates of the intersection as (X, Y). The method for calculating X and Y by the intersection calculation module is:
[0074]
[0075] Among them, x1 and x2 are the horizontal coordinates of point P and point Q respectively; m is the longitudinal distance between point Q and the polar line; n is the longitudinal distance between point P and the polar line; the data of m and n are obtained from the polar line calculation module;
[0076] or:
[0077] Among them, y1 and y2 are the ordinates of point P and point Q respectively; m is the lateral distance between point Q and the polar line; n is the lateral distance between point P and the polar line; the data of m and n are obtained from the polar line calculation module.
[0078] The hardware module for matching homonymous points has obvious advantages when performing parallel operations in FPGA. Under the same hardware equipment conditions, the number of DSP resources is reduced by 78% when the existing method for obtaining homonymous points mentioned above is used. Assuming that the number of hardware DSP resources in the FPGA chip is 500, the operation speed is improved by 4.43 compared with the existing technology. The higher the total number of DSP resources, the more obvious the speed improvement of this method is.
[0079] The foregoing description of the specific exemplary embodiments of the present invention is presented for the purpose of illustration and description. The foregoing description is not intended to be exhaustive, nor is it intended to limit the present invention to the precise form disclosed, and it is apparent that many changes and variations are possible in light of the above teachings. The exemplary embodiments are selected and described to explain the specific principles of the present invention and its practical application, so that other technicians in the field can realize and utilize the various exemplary embodiments of the present invention and its different selected forms and modified forms. The scope of the present invention is intended to be limited by the appended claims and their equivalents.
Claims
1. A method for matching homonymous points, wherein the homonymous points are imaging points of the same point in space respectively taken by two cameras in a binocular vision detection system, and the imaging points are subject to epipolar constraints; the images obtained by the two cameras taking images of the same object to be detected are respectively referred to as the first image and the second image; It is characterized in that The steps include: S1. Perform image segmentation on the first image and the second image respectively to obtain a plurality of mutually corresponding feature regions; at the same time, perform feature point extraction on the first image and the second image, and mark the extracted feature points according to the feature regions where they are located; S2, a certain point to be matched with the same name in the first image is recorded as point A, the image coordinates of point A are obtained, the feature region where it is located is marked as feature region A, and the corresponding region of feature region A is searched in the second image, recorded as feature region A'; S3, using the relationship between the two cameras in the binocular vision detection system, obtaining the epipolar line II corresponding to point A on the second image; Taking the A' feature area as the calculation area, obtain the lateral distance or longitudinal distance between each feature point in the A' feature area and the polar line II in the image coordinate system of the second image; take a point closest to the polar line II on both sides, respectively, and record them as point P and point Q; the intersection of the line connecting point P and point Q and the polar line II is the point of the same name as point A.
2. A method for matching points of the same name as claimed in claim 1, characterized in that: Step S3 is a method for obtaining the lateral distance or longitudinal distance of each feature point in the A' feature area from the polar line II in the image coordinate system of the second image by: ① Obtain the image coordinates of each feature point in the A' feature area and the equation of the epipolar line II; ② When calculating the horizontal distance, substitute the longitudinal coordinate value of the feature point into the equation of the polar line II, calculate the horizontal coordinate of the corresponding point on the polar line, and then calculate |the horizontal coordinate of the feature point - the horizontal coordinate of the corresponding point on the polar line|, which is the horizontal distance; When calculating the longitudinal distance, substitute the horizontal coordinate value of the feature point into the equation of the polar line II, and calculate the longitudinal coordinate of the corresponding point on the polar line. The result is |longitudinal coordinate of the feature point - longitudinal coordinate of the corresponding point on the polar line|, which is the longitudinal distance.
3. A method for matching points of the same name as claimed in claim 1, characterized in that: Assume that the coordinates of the intersection of the line connecting point P and point Q obtained in step S3 and the polar line II are (X, Y), and the equation of the polar line II is: Ax+By+C=0; The calculation method of X and Y is: Among them, x1 and x2 are the horizontal coordinates of point P and point Q respectively; m is the longitudinal distance between point Q and polar line II; n is the longitudinal distance between point P and polar line II; or: Among them, y1 and y2 are the ordinates of point P and point Q respectively; m is the lateral distance between point Q and polar line II; n is the lateral distance between point P and polar line II.
4. A hardware module for matching homonymous points, applied to a binocular vision detection system, wherein the images obtained by two cameras for the same object to be detected are respectively referred to as the first image and the second image, and the image where the homonymous points to be matched are referred to as the first image; the imaging points of the two cameras for the same point in space have epipolar constraints; Features: It includes image segmentation module, feature point extraction module, feature point area search module and homonymous point calculation module; The image segmentation module is used to segment the first image and the second image into a plurality of mutually corresponding feature regions, and transmit the regions to the feature point extraction module and the feature point region search module; The feature point extraction module extracts feature points from the first image and the second image respectively, and marks each feature point and the feature area where it is located; The feature point region search module is used to search the feature region where the to-be-matched homonymous points are located in the second image, and transmit the result to the homonymous point calculation module; The same-name point calculation module includes an epipolar line calculation module, a distance calculation module and an intersection calculation module which are electrically connected; The epipolar line calculation module is used to obtain the epipolar line equation corresponding to the to-be-matched homonymous point in the second image, and can substitute the horizontal coordinate or vertical coordinate of each feature point in the feature area corresponding to the to-be-matched homonymous point on the second image into the epipolar line equation to obtain the vertical coordinate or horizontal coordinate of the corresponding point on the epipolar line; the distance calculation module is used to calculate |the horizontal coordinate of the feature point-the horizontal coordinate of the corresponding point on the epipolar line|, or |the vertical coordinate of the feature point-the vertical coordinate of the corresponding point on the epipolar line|, and respectively select the points with the smallest values on both sides of the epipolar line, and transmit the results to the intersection calculation module; the intersection calculation module is used to connect the points with the smallest values on both sides of the received epipolar line, and obtain the intersection of the line connecting the two points and the epipolar line, and the intersection is the homonymous point of the to-be-matched homonymous point.
5. The hardware module for matching homonymous points as claimed in claim 4, characterized in that: The polar line equation obtained by the polar line calculation module is Ax+By+C=0; The points with the smallest values on both sides of the polar line obtained by the distance calculation module are recorded as point P and point Q; The intersection calculation module obtains the coordinates of the intersection as (X, Y), and the method for calculating X, Y by the intersection calculation module is: Wherein, x1 and x2 are the horizontal coordinates of point P and point Q respectively; m is the longitudinal distance between point Q and the polar line; n is the longitudinal distance between point P and the polar line; the data of m and n are obtained from the polar line calculation module; or: Among them, y1 and y2 are the ordinates of point P and point Q respectively; m is the lateral distance between point Q and the polar line; n is the lateral distance between point P and the polar line; the data of m and n are obtained from the polar line calculation module.