Homonymous image point acquisition method, electronic equipment and storage medium

Abstract

The invention relates to the technical field of image matching, and provides a homonymy image point acquisition method, electronic equipment and a storage medium, and the method comprises the steps: determining a first image point suitable for representing a target object point in a first image, setting a depth range adaptive to the target object point, according to the first image point and the depth range, generating a corresponding target line segment in each epipolar line for at least one second image having a relative pose relationship with the first image, so that a redundant line segment suitable for being complementary with the target line segment as a whole epipolar line is abandoned, and defining a buffer area in the corresponding second image for the at least one target line segment, according to the method, the second image point suitable for representing the target object point is searched in the at least one buffer area, compared with a whole image or a whole epipolar line, the buffer area can accommodate the situation that the second image point is adjacent to the target line segment and the situation that the second image point is located on the target line segment, and the efficiency, the accuracy and the robustness of obtaining the same-name image points can be better considered.

Description

Technical Field

[0001] This invention relates to the field of image matching technology, and more specifically, to a method for acquiring corresponding image points, an electronic device, and a storage medium. Background Technology

[0002] Corresponding image points refer to the image points of the same object point in different images. Efficiently and accurately finding corresponding image points can have a crucial impact on many fields such as digital photogrammetry and computer vision.

[0003] The relative pose relationship between the two images is known. If a certain object point corresponds to an image point in one image as the first image point, the epipolar line where the epipolar plane where the first image point is located intersects with the other image can be calculated. The second image point corresponding to the aforementioned object point can be found on this epipolar line. Compared with the two-dimensional space of the entire image, the entire epipolar line defines the one-dimensional space in the image, which helps to improve the efficiency and accuracy of finding the same image point.

[0004] However, in reality, there are redundant parts along the entire epipolar line. In addition, the image points that need to be found may be near the epipolar line rather than on the epipolar line, resulting in poor performance of existing methods. Summary of the Invention

[0005] The present invention aims to at least partially solve the technical problems in related technologies and provide a method for acquiring corresponding image points, an electronic device, and a storage medium.

[0006] In a first aspect, the present invention provides a method for acquiring image points with the same name, comprising: In the first image, a first image point suitable for representing the target object point is determined, and a depth range adapted to the target object point is set; Based on the first image point and the depth range, target line segments in each epipolar line are generated in at least one second image that has a relative pose relationship with the first image. For at least one of the target line segments, a buffer zone is defined in the corresponding second image, and a second image point suitable for representing the target object point is searched in at least one of the buffer zones.

[0007] Optionally, setting a depth range adapted to the target point includes: The depth value corresponding to the first image point is obtained from the depth map matched with the first image, and the depth range is defined based on the depth value corresponding to the first image point and a preset error allowable range.

[0008] Optionally, generating target line segments in each epipolar line of at least one second image that has a relative pose relationship with the first image based on the first image point and the depth range includes: Determine the projection center for the first image; A ray suitable for emanating from the projection center and passing through the first image point is used as a basic projection line, and a first line segment matching the depth range is defined for the basic projection line; Based on the relative pose relationship between each second image and the first image, the first line segment is transformed into the corresponding second line segment, and the second line segment is projected onto the corresponding second image to form the corresponding target line segment.

[0009] Optionally, the basic projection line is denoted as S. m A: ,in, Indicates from the image space coordinate system S m -x m y m z m To the auxiliary coordinate system S in image space m -X m Y m Z m The rotation matrix, Indicates along z m The negative direction of the coordinate axes is relative to the projection center S. m Depth of distance Indicates from the projection center S m The vertical distance to the first image, , and This indicates that the first image point is located in the image space coordinate system S. m -x m y m z m coordinate data, , and This indicates that the first image point is located in the auxiliary coordinate system S in image space. m -X m Y m Z m Coordinate data.

[0010] The maximum and minimum depth values ​​within the depth range are respectively taken as Determine the basic projection line S m Two points in A, denoted as A''''''''''''''''''''''"'"""' ... d- A d+ With two points A d- A d+ Define a straight line segment for the endpoints, and the straight line segment is the first line segment.

[0011] The relationship between the second line segment and the first image point is as follows: ,in, , and This indicates that any point in the second line segment lies in the image space coordinate system S. n -x n y n z n coordinate data, express The inverse matrix, Indicates from the image space coordinate system S n -x n y n z n To the auxiliary coordinate system S in image space n -X n Y n Z n The rotation matrix, The value is taken within the aforementioned depth range. , and Indicates the auxiliary coordinate system S in image space m -X m Y m Z m To the auxiliary coordinate system S in image space n -X n Y n Z n Translation data.

[0012] Any point in the target line segment is represented as follows: ,in, Indicates from the projection center S n The vertical distance to the second image.

[0013] Optionally, defining a buffer zone in the corresponding second image for at least one of the target line segments includes: For any one of the target line segments, a translation direction suitable for being parallel to the corresponding second image is set; Parallelogram frames are generated on both sides of the corresponding target line segment according to the translation direction and the preset translation distance; The buffer zone is defined as the area that is suitable to be enclosed by the parallelogram frame and is located in the corresponding second image.

[0014] Optionally, generating parallelogram frames on both sides of the corresponding target line segment according to the translation direction and a preset translation distance includes: A translation vector is calculated based on the translation direction and the translation distance; Another translation vector is calculated using the direction opposite to the translation direction and the translation distance; Each of the aforementioned translation vectors is used to translate from the two endpoints of the corresponding target line segment to the corresponding boundary point; The four boundary points, which are suitable for distribution on both sides of the corresponding target line segment, are connected to form the parallelogram frame.

[0015] Optionally, searching for a second image point suitable for representing the target object point in at least one of the buffers includes: Determine the neighborhood of the first pixel in the first image based on the first pixel point; Detect a second pixel neighborhood that matches the first pixel neighborhood in each of the buffers; The distance between the center point in the neighborhood of each second pixel and the corresponding target line segment is measured. Determine whether the measured distance values ​​of each point are greater than a preset distance threshold. If so, discard the corresponding second pixel neighborhood; otherwise, use the corresponding center point as the second image point.

[0016] Secondly, the present invention provides a method for acquiring image points with the same name, including: In the first image, a first image point suitable for representing the target object point is determined, and a depth range adapted to the target object point is set; Based on the first image point and the depth range, target line segments in each epipolar line are generated for multiple second images that have a relative pose relationship with the first image. For each target line segment, a buffer zone is defined in the corresponding second image, and a second image point suitable to be a homonym of the first image point is searched in each buffer zone; For multiple second image points belonging to multiple second images respectively, calculate the corresponding forward intersection points in groups of two, and identify the corresponding multiple second image points based on at least one of the forward intersection points.

[0017] Optionally, identifying a plurality of corresponding second image points based on at least one of the forward intersection points includes: Identify projection points in the first image that correspond to each of the aforementioned forward intersection points; Error measurement is performed on each of the projection points based on the first image point; Determine whether the error of each measured projection point exceeds the preset point error threshold. If so, remove the corresponding forward intersection point; otherwise, retain the corresponding forward intersection point. Each second image point corresponding to each of the retained forward intersection points is taken as the correct image point of the same name.

[0018] Thirdly, the present invention provides an electronic device, comprising: A memory that stores computer programs; At least one processor is adapted to implement any of the above-described methods for acquiring image points with the same name when executing the computer program.

[0019] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed, implements any of the above-described methods for acquiring image points with the same name.

[0020] Using the aforementioned method for acquiring corresponding image points, electronic devices, and computer-readable storage media, the relative pose relationships between the first image and at least one second image are known. The first image point in the first image corresponds to the target object point. The depth range adapted to the target object point is used as a constraint condition. For each second image intersecting with the corresponding epipolar line, only the target line segment needs to be formed, so that redundant line segments that are complementary to the target line segment to form a whole epipolar line can be discarded. Furthermore, a buffer zone is obtained by extending from the target line segment to its vicinity. Compared to searching for the second image point corresponding to the target object point in the whole image or the whole epipolar line, it is more appropriate to use the local space of the buffer zone in the second image as the two-dimensional space for searching the second image point. The buffer zone can accommodate the cases where the second image point is adjacent to the target line segment and the cases where it is on the target line segment, which helps to better balance the efficiency, accuracy, and robustness of acquiring corresponding image points. Furthermore, when multiple second image points belong to multiple second images, the forward intersection method can be used to eliminate inferior second image points and retain superior ones, which helps to improve the accuracy of corresponding image points. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a method for obtaining image points with the same name according to an embodiment of the present invention; Figure 2 , Figure 3 and Figure 4 This is a geometric schematic diagram of two images being relatively oriented through consecutive image pairs, according to an embodiment of the present invention. Figure 5 This is a geometrical diagram of the image plane coordinate system and the pixel coordinate system on the same image according to an embodiment of the present invention. Figure 6 This is a geometrical diagram illustrating the relationship between the depth range and the first image point and the target line segment in an embodiment of the present invention. Figure 7 This is a geometric schematic diagram showing the entire core line divided into three segments according to an embodiment of the present invention; Figure 8 and Figure 9 These are geometric schematic diagrams of an embodiment of the present invention, showing the buffer zones defined on both sides of a target line segment. Figure 10This is a flowchart illustrating another method for obtaining image points with the same name according to an embodiment of the present invention; Figure 11 This is a geometrical diagram of a forward intersection point projected onto a first image according to an embodiment of the present invention. Detailed Implementation

[0022] Embodiments of the present invention will now be described in detail with reference to the accompanying drawings. When referring to the drawings, unless otherwise indicated, the same reference numerals in different drawings denote the same or similar elements. It should be noted that the embodiments described in the following exemplary embodiments do not represent all embodiments of the present invention. They are merely examples of apparatuses and methods consistent with some aspects of the present invention disclosed in the claims, and the scope of the present invention is not limited thereto. Features in the various embodiments of the present invention can be combined with each other without contradiction.

[0023] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "more" means greater than or equal to 2, such as two or three, unless otherwise explicitly specified.

[0024] See Figure 1 The present invention provides a method for obtaining image points with the same name, which includes steps S110 to S130.

[0025] S110: Determine a first image point suitable for representing the target object point in the first image, and set a depth range adapted to the target object point.

[0026] For example, the target point can be a point located on, for example, a building or a road line. The geographic coordinates of the target point can be converted into preset coordinates in an image space coordinate system adapted to the first image. These preset coordinates can be used to automatically identify the first image point in the first image, or, in response to input to a graphical user interface (e.g., a click operation), to identify the first image point in the first image.

[0027] For example, the minimum and maximum heights of the area where the target point is located relative to the geoid can be estimated in advance, and the range between the minimum and maximum heights can be used as the depth range. For example, the undulation of the road surface area where a target point is located can be observed in advance, with a minimum height of 42 meters and a maximum height of 44 meters.

[0028] S120: Generate target line segments in each epipolar line from at least one second image that has a relative pose relationship with the first image, based on the first image point and depth range.

[0029] For example, the first image can be any one of the multi-view images, and any second image can be another image that belongs to the multi-view images and is not the first image. The multi-view images can refer to multiple images of a real scene taken from multiple perspectives. The relative pose relationship between the multi-view images can be known in advance through relative orientation, such as relative orientation of consecutive image pairs or relative orientation of individual image pairs.

[0030] For a better concrete example, see Figure 2 , Figure 3 and Figure 4 Two projection centers S m S n And a certain object point A is distributed on the same core surface, from the projection center S m Ray S emitted toward object point A m A suitable for image I m Image point a in m From the projection center S n Ray S emitted toward object point A n A suitable for image I n Image point a in n Two images I n I m The relative positions and attitude relationships between them can be obtained through the following formulas 1, 2 and 3.

[0031] Formula 1: ,in, , and Indicates the projection center S n Located in the auxiliary coordinate system S in image space m -X m Y m Z m coordinate data, , and Representing image point a m Located in the auxiliary coordinate system S in image space m -X m Y m Z m Coordinate data, such as the spatial auxiliary coordinate system S m -X m Y m Z m Suitable for use with the auxiliary coordinate system S in image space after translation n -X n Y n Z n coincide, , and Representing image point a n Located in the auxiliary coordinate system S in image space n -X n Y n Z n Coordinate data.

[0032] Formula 2: ,in, Indicates from the image space coordinate system S m -x m y m z m To the auxiliary coordinate system S in image space m -X m Y m Z m The rotation matrix, like the spatial coordinate system S m -x m y m z m With Image I m match, , and Representing image point a m Located in image space coordinate system S m -x m y m z m coordinate data, Indicates from the projection center S m To Image I m The vertical distance (also known as the principal distance).

[0033] Formula 3: ,in, Indicates from the image space coordinate system S n -x n y n z n To the auxiliary coordinate system S in image space n -X n Y n Z n The rotation matrix, like the spatial coordinate system S n -x n y n z n With Image I n match, , and Representing image point a n Located in image space coordinate system S n -x n y n z n coordinate data, Indicates from the projection center S n To Image I n The vertical distance.

[0034] See formulas 4, 5, and 6 below. Figures 2 to 4 If two images I n I m The relative positions and orientations between them are known, based on image point a m The corresponding kernel surface and image I can be calculated. n The entire intersecting nuclear line.

[0035] Formula 4: ,in, , and This indicates that object point A is located in the auxiliary coordinate system S in image space. m -X m Y m Z m coordinate data, , and This indicates that object point A is located in the image space coordinate system S. m -x m y m z m coordinate data, Indicates from the projection center S m Along z m The straight-line distance (also known as depth) between the negative direction of the coordinate axis and object point A.

[0036] Formula 5: ,in, , and It also indicates that from the auxiliary coordinate system S in image space m -X m Y m Z m To the auxiliary coordinate system S in image space n -X n Y n Z n Translation data, , and This indicates that object point A is located in the image space coordinate system S. n -x n y n z n The coordinate data, from the projection center S n Along z n The straight-line distance between the negative directions of the coordinate axes and object point A can be expressed as: .

[0037] Formula 6: Formula 6 can be obtained by combining Formula 5 and Formula 4. express The inverse matrix, that is Indicates the auxiliary coordinate system S in image space n -X n Y n Z n To the image space coordinate system S n -x n y n z n The rotation matrix.

[0038] Accordingly, any point in the epipolar line can be represented as: .

[0039] S130: Define a buffer zone in the corresponding second image for at least one target line segment, and search for a second image point suitable for representing the target object point in at least one buffer zone, so that the second image point and the first image point are image points with the same name.

[0040] For example, see Formula 7 below and Figure 5 In any of the images mentioned above, the two coordinate axes of the image plane coordinate system are the x-axis and the y-axis, and the origin of the image plane coordinate system is at the principal point o. The two coordinate axes of the pixel coordinate system are the u-axis and the v-axis, and the origin of the pixel coordinate system is at the upper left corner of the image. The positive direction of the u-axis is the same as the positive direction of the x-axis, and the positive direction of the v-axis is opposite to the positive direction of the y-axis.

[0041] Formula 7: , where u o v o This represents the coordinate data of the principal point o in the pixel coordinate system, d x d y It represents the physical size of any pixel in the image plane coordinate system.

[0042] Using the same-name image point acquisition method of the present invention, the relative pose relationship between the first image and at least one second image is known. The first image point in the first image corresponds to the target object point. The depth range adapted to the target object point is used as a constraint condition. For the epipolar lines intersecting each second image with the corresponding epipolar plane, only target line segments need to be formed, so that redundant line segments that are suitable to complement the target line segments to form a whole epipolar line can be discarded. In addition, a buffer zone is obtained by extending from the target line segment to its vicinity. Compared with searching for the second image point corresponding to the target object point in the whole image or the whole epipolar line, it is more appropriate to use the local space of the buffer zone in the second image as the two-dimensional space for searching the second image point. The buffer zone can accommodate the situation where the second image point is adjacent to the target line segment and the situation where it is on the target line segment, which helps to better balance the efficiency, accuracy and robustness of acquiring the same-name image point.

[0043] Optionally, in S110, setting a depth range adapted to the target object point includes: obtaining the depth value corresponding to the first image point from the depth map matched with the first image; defining the depth range based on the depth value corresponding to the first image point and a preset error allowable range; if the position of the first image point changes, the depth range also changes adaptively. Compared with keeping the depth range unchanged, this helps to improve the accuracy of the depth range adapting to the target object point and is more in line with the actual situation.

[0044] For example, the depth map matching the first image can be generated in advance. The depth map generation method can employ a monocular depth estimation algorithm based on deep learning technology or / and a disparity estimation algorithm based on multi-view geometry, etc.

[0045] For example, see Figure 6 The allowable error range can be expressed as ±E or [-E, +E], where E represents the absolute error limit. The depth range can be expressed as d1±E or [d1-E, d1+E], where d1 represents the depth value of the first image point in the preset depth map. The first image point can be denoted as a. m For example, E represents 2.5 meters and d1 represents 10.5 meters.

[0046] For example, see Figure 6 The minimum depth value within the depth range can be denoted as d. min d min =d1-E, where the minimum depth value in the depth range can be denoted as d. max d max =d1+E, it should be noted that d max -d1 and d1-d min They can be unequal and the difference can be small, for example: 0 < |d max +d min -2d1|<1.

[0047] Optionally, S120 includes: determining a projection center for the first image, using a ray suitable for emanating from the projection center and passing through the first image point as a basic projection line, defining a first line segment that matches the depth range for the basic projection line, transforming the first line segment into a corresponding second line segment according to the relative pose relationship between each second image and the first image, and projecting each second line segment into the corresponding second image to form corresponding target line segments.

[0048] For example, see Figure 6 The first image can be denoted as I. m , with the first image I m The corresponding projection center can be denoted as S. m The basic projection line is denoted as S. m A, Basic projection line S m A can be obtained through Formula 4 mentioned above, by taking the formula from Formula 4... By taking the maximum and minimum depth values ​​within the depth range respectively, the basic projection line S can be determined. m Two points in A are denoted as A0 and A1 respectively. d- A d+ With two points A d- A d+ Connecting the endpoints yields a straight line segment, which is the first line segment. This segment can be obtained by applying formula 5 mentioned above to both points A. d- A d+ Transform the line to obtain the two endpoints of the corresponding second line segment, which is also set as a straight line segment.

[0049] For example, any point in the second line segment and the first image point a m The relationship between them can be represented using Formula 6 mentioned above, see [link / reference]. Figure 6 Following the principle of central projection, two points distributed in the corresponding second image can be obtained using the two endpoints of any second line segment. The straight line segment connecting these two points is taken as the corresponding target line segment. The second image can be denoted as I. n , with the second image I n The corresponding projection center can be denoted as S. n .

[0050] For example, see Figure 7 The target line segment can be denoted as NL. rs , and the target line segment NL rs The redundant segments that complement each other to form the entire core line can be denoted as NL. es In this embodiment of the invention, a portion of the entire core line is extracted as the target segment, effectively preventing the formation of segment NL. es This helps to simply, accurately, and reliably reduce the one-dimensional space of the entire core line.

[0051] Optionally, in S130, defining a buffer zone in the corresponding second image for at least one target line segment includes: setting a translation direction suitable for parallel to the corresponding second image for any target line segment, generating parallelogram frames on both sides of the corresponding target line segment according to the translation direction and a preset translation distance, and using the area suitable for being surrounded by the parallelogram frames and in the corresponding second image as a buffer zone.

[0052] For example, the two endpoints of any target line segment can be used as two reference points. The coordinate data of the two reference points in the corresponding second image can be used to represent the reference vector from one reference point to another. The reference vector is used to calculate a unit vector that forms a certain angle with it. The direction indicated by the unit vector is the translation direction.

[0053] For example, see Figure 8 The red directed line segment can represent the reference vector, θ can represent the preset included angle, and θ can belong to [60°, 90°]. The parallelogram box is completely within the second image. See [link / reference]. Figure 9 The parallelogram frame intersects the boundary of the second image at two points, and the buffer zone refers to the two-dimensional space where the parallelogram and the second image intersect.

[0054] Optionally, generating a parallelogram frame on both sides of the corresponding target line segment according to the translation direction and a preset translation distance includes: calculating a translation vector according to the translation direction and translation distance, calculating another translation vector according to the opposite direction and translation distance, translating from the two endpoints of the corresponding target line segment to the corresponding boundary points according to each translation vector, and connecting the four boundary points suitable for distribution on both sides of the corresponding target line segment to form a parallelogram frame. This helps to prevent the buffer from being too large or too small, and has simplicity, accuracy and reliability.

[0055] For example, see Figure 8 The first translation vector is obtained by multiplying the unit vector mentioned above by the preset translation distance. The second translation vector can be obtained by inverting the first translation vector, or by multiplying the unit vector by the preset translation distance. The two boundary points are obtained by translating from the two reference points mentioned above according to the first translation vector. The other two boundary points are obtained by translating from the two reference points according to the second translation vector. The four boundary points are joined together as four vertices to form a parallelogram. For example, the parallelogram frame is rectangular due to the constraint that the included angle θ is 90°.

[0056] For example, the preset translation distance can be the value of the pixel pitch multiplied by the first magnification. The pixel pitch can be the Euclidean distance between the centers of two adjacent pixels in the same image. The Euclidean distance can be calculated using the xy coordinate data or uv coordinate data of the two points. The first magnification can be in the range [8, 11], for example, 8, 9, or 11.

[0057] Optionally, in S130, searching for a second image point suitable for representing the target object point in at least one buffer includes: determining a first pixel neighborhood in the first image based on the first image point; detecting a second pixel neighborhood that matches the first pixel neighborhood in each buffer; measuring the distance between the center point in each second pixel neighborhood and the corresponding target line segment; determining whether the measured distance value of each point-line is greater than a preset distance threshold; if so, discarding the corresponding second pixel neighborhood to prevent the center point in the corresponding second pixel neighborhood from being used as the second image point; otherwise, using the center point in the corresponding second pixel neighborhood as the second image point, which is beneficial for balancing the efficiency, success rate, accuracy, reliability and robustness of obtaining the same image point.

[0058] Regarding the first pixel and any buffer zone in the second image, a strategy of "coarse detection" followed by "precise detection" is adopted. Using the first pixel neighborhood that belongs to the first image and contains the second pixel, the second pixel neighborhood that may contain a pixel with the same name in any buffer zone is quickly found. The center point in any second pixel neighborhood can be referred to as a candidate pixel. If the corresponding point-to-line distance value is greater than the distance threshold, it indicates that the candidate pixel deviates too much from the target line segment that is in the same second image, and the candidate pixel does not meet the requirement of being a pixel with the same name as the first pixel. If the corresponding point-to-line distance value is less than or equal to the distance threshold, it indicates that the candidate pixel is near or contained in the target line segment, and the candidate pixel is suitable to be a pixel with the same name as the first pixel.

[0059] For example, in the first image, the pixel containing the first image point is taken as the center pixel, and multiple pixels distributed near the center pixel are taken as neighboring pixels according to a preset neighborhood specification data. The center pixel and the corresponding multiple neighboring pixels form the first pixel neighborhood. The neighborhood specification data can represent 4 neighborhoods, 8 neighborhoods, or 24 neighborhoods, etc. The second pixel neighborhood with the highest similarity to the first pixel neighborhood can be searched in any buffer using a preset image matching model. The image matching model can adopt a gray-scale-based matching algorithm or a deep learning-based matching algorithm, etc.

[0060] For example, the xy coordinates of the center point in any second pixel neighborhood and the two endpoints of the corresponding target line segment can be obtained respectively. The corresponding straight line equation can be established based on the xy coordinates of the two endpoints. The shortest distance between the center point and the target line segment can be calculated based on the straight line equation and the xy coordinates of the center point. The shortest distance is the corresponding point-to-line distance value. It should be understood that the shortest distance can also be obtained based on the uv coordinates of the center point and the uv coordinates of the two endpoints.

[0061] For example, the preset distance threshold may refer to the value of the pixel spacing mentioned above multiplied by the second multiplier, which may be in the range of [1, 4], such as 1, 3.2, or 4.

[0062] See Figure 10 Another embodiment of the present invention provides a method for obtaining image points with the same name, which includes steps S210 to S240.

[0063] S210: Determine a first image point suitable for representing the target object point in the first image, and set a depth range adapted to the target object point.

[0064] S220: Generate target line segments in each epipolar line for multiple second images that have a relative pose relationship with the first image, based on the first image point and depth range.

[0065] S230: Define buffer zones for each target line segment in the corresponding second image, and search for a second image point suitable for representing the target object point in each buffer zone.

[0066] S240: For multiple second image points belonging to multiple second images respectively, calculate the corresponding forward intersection points in groups of two, and identify the corresponding multiple second image points based on at least one forward intersection point.

[0067] For example, see Figure 11 For N-1 second images, C(N-1,2) candidate image pairs can be formed by following an unordered combination method, where N is a positive integer greater than 2. Each second image contains a corresponding second image point. Forward intersection calculation is performed on the two second image points in each candidate image pair.

[0068] For example, N equals 5, and the 5 images have different perspectives from each other. The first image is the first image, and the remaining four images are the second images. By following the unordered combination method, 6 forward intersection points can be obtained.

[0069] Using the same image point acquisition method of the present invention, the specific implementation and beneficial effects of S210 to S230 are the same as those of the method mentioned above, and will not be repeated here. Furthermore, by using the forward intersection method, multiple second image points belonging to multiple second images can be eliminated and retained as good as bad, which helps to improve the accuracy of the same image points.

[0070] Optionally, in S240, identifying multiple second image points based on at least one forward intersection point includes: identifying projection points corresponding to each forward intersection point in the first image; measuring the error of each projection point based on the first image point; determining whether the measured error of each projection point exceeds a preset point error threshold; if so, removing the corresponding forward intersection point; otherwise, retaining the corresponding forward intersection point, until the last forward intersection point is removed or retained; and using the second image points corresponding to each retained forward intersection point as correct corresponding image points, which helps to easily and reliably reduce the error rate of corresponding image points.

[0071] For example, see Figure 11 The first pixel in the first image can be denoted as a1, the second pixel in the first second image can be denoted as a2, and the second pixel in the last second image can be denoted as a... N Regarding the second image point a2 and the second image point a N The corresponding forward intersection point A is calculated. 2N ′, the intersection point A ahead 2N Projected onto the first image, so that the corresponding projection point a 2N It was thus located.

[0072] For example, the projection point error can refer to the Euclidean distance between the corresponding projection point and the first image point. For instance, the corresponding Euclidean distance can be calculated using the xy coordinates of the first image point and the xy coordinates of the projection point, or the corresponding Euclidean distance can be calculated using the uv coordinates of the first image point and the uv coordinates of the projection point.

[0073] For example, the preset point error threshold can be the value of the pixel spacing multiplied by the third multiplier, as mentioned above. The third multiplier can be in the range of [1, 2], such as 1, 1.5, or 2.

[0074] If the error of any projection point is greater than the point error threshold, it indicates that the corresponding projection point deviates too much from the first image point, and the corresponding forward intersection point does not match the target object point; if the error of any projection point is less than or equal to the point error threshold, it indicates that the corresponding projection point is close to or coincides with the first image point, and the corresponding forward intersection point matches the target object point.

[0075] For example, the first image can be denoted as I1, the four second images can be denoted as I2, I3, I4, and I5 respectively, the six candidate image pairs can be denoted as (I2, I3), (I2, I4), (I2, I5), (I3, I4), (I3, I5), and (I4, I5), and the six forward intersection points corresponding one-to-one with the six candidate image pairs can be denoted as a. 23 ′、a 24 ′、a 25 ′、a 34 ′、a 35 ′、a 45 ′, where 3 forward intersection points a were removed. 23 ′、a 34 ′、a 35 ′, retaining 3 forward intersection points a 24 ′、a 25 ′、a 45 ′, and then select the intersection point a with the three points in front. 24 ′、a 25 ′、a 45 The three candidate image pairs (I2, I4), (I2, I5), and (I4, I5) are used to eliminate the second image I3, so that the three second image points belonging to the three second images I2, I4, and I5 respectively are used as the final determined image points with the same name.

[0076] Another embodiment of the present invention includes an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements at least one of the aforementioned methods for acquiring corresponding image points. The processor can be connected to the memory via a universal serial bus. It is understood that the aforementioned electronic device may be a server or a terminal device.

[0077] Another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements at least one of the above-mentioned methods for acquiring image points with the same name.

[0078] Generally, a computer program for implementing the method of the present invention can be carried by any combination of one or more computer-readable storage media, except for the signal itself in temporary propagation.

[0079] Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0080] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. In particular, Python, suitable for neural network computation, and platform frameworks based on TensorFlow, PyTorch, etc., can be used. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet using an Internet service provider).

[0081] For details regarding the aforementioned electronic devices and computer-readable storage media, please refer to the implementation details and beneficial effects of the above-mentioned method for acquiring identical image points, which will not be repeated here.

[0082] Although embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for obtaining image points with the same name, characterized in that, include: In the first image, a first image point suitable for representing the target object point is determined, and a depth range adapted to the target object point is set; Based on the first image point and the depth range, target line segments in each epipolar line are generated in at least one second image that has a relative pose relationship with the first image. For at least one of the target line segments, a buffer zone is defined in the corresponding second image, and a second image point suitable for representing the target object point is searched in at least one of the buffer zones.

2. The method for obtaining corresponding image points according to claim 1, characterized in that, Setting a depth range adapted to the target point includes: The depth value corresponding to the first image point is obtained from the depth map matched with the first image, and the depth range is defined based on the depth value corresponding to the first image point and a preset error allowable range.

3. The method for obtaining corresponding image points according to claim 1, characterized in that, Based on the first image point and the depth range, generating corresponding target line segments in each epipolar line of at least one second image that has a relative pose relationship with the first image includes: Determine the projection center for the first image; A ray suitable for emanating from the projection center and passing through the first image point is used as a basic projection line, and a first line segment matching the depth range is defined for the basic projection line; Based on the relative pose relationship between each second image and the first image, the first line segment is transformed into the corresponding second line segment, and the second line segment is projected onto the corresponding second image to form the corresponding target line segment.

4. The method for obtaining corresponding image points according to claim 1, characterized in that, Defining a buffer zone in the corresponding second image for at least one of the target line segments includes: For any one of the target line segments, a translation direction suitable for being parallel to the corresponding second image is set; Parallelogram frames are generated on both sides of the corresponding target line segment according to the translation direction and the preset translation distance; The buffer zone is defined as the area that is suitable to be enclosed by the parallelogram frame and is located in the corresponding second image.

5. The method for acquiring corresponding image points according to claim 4, characterized in that, Generating parallelogram frames on both sides of the corresponding target line segment according to the translation direction and the preset translation distance includes: A translation vector is calculated based on the translation direction and the translation distance; Another translation vector is calculated using the direction opposite to the translation direction and the translation distance; Each of the aforementioned translation vectors is used to translate from the two endpoints of the corresponding target line segment to the corresponding boundary point; The four boundary points, which are suitable for distribution on both sides of the corresponding target line segment, are connected to form the parallelogram frame.

6. The method for obtaining corresponding image points according to any one of claims 1 to 5, characterized in that, Searching for a second image point suitable for representing the target object point in at least one of the buffers includes: Determine the neighborhood of the first pixel in the first image based on the first pixel point; Detect a second pixel neighborhood that matches the first pixel neighborhood in each of the buffers; The distance between the center point in the neighborhood of each second pixel and the corresponding target line segment is measured. Determine whether the measured distance values ​​of each point are greater than a preset distance threshold. If so, discard the corresponding second pixel neighborhood; otherwise, use the corresponding center point as the second image point.

7. A method for obtaining image points with the same name, characterized in that, include: In the first image, a first image point suitable for representing the target object point is determined, and a depth range adapted to the target object point is set; Based on the first image point and the depth range, target line segments in each epipolar line are generated for multiple second images that have a relative pose relationship with the first image. For each target line segment, a buffer zone is defined in the corresponding second image, and a second image point suitable for representing the target object point is found in each buffer zone; For multiple second image points belonging to multiple second images respectively, calculate the corresponding forward intersection points in groups of two, and identify the corresponding multiple second image points based on at least one of the forward intersection points.

8. The method for obtaining corresponding image points according to claim 7, characterized in that, Identifying a plurality of corresponding second image points based on at least one of the aforementioned forward intersection points includes: Identify projection points in the first image that correspond to each of the aforementioned forward intersection points; Error measurement is performed on each of the projection points based on the first image point; Determine whether the error of each measured projection point exceeds the preset point error threshold. If so, remove the corresponding forward intersection point; otherwise, retain the corresponding forward intersection point. Each second image point corresponding to each of the retained forward intersection points is taken as the correct image point of the same name.

9. An electronic device, characterized in that, include: A memory that stores computer programs; At least one processor adapted to implement the same-name image point acquisition method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed, it implements the method for obtaining image points with the same name as described in any one of claims 1 to 8.

Images