A gun-ball coordinate conversion method suitable for panoramic stitched images

Abstract

The application discloses a gun-ball coordinate conversion method suitable for panoramic spliced images, which comprises the following steps: selecting a plurality of marking points closest to a to-be-converted point P on the panoramic spliced image; screening specific triangles; determining edge length density distortion; and using a three-point conversion algorithm to obtain an accurate gun-ball conversion result. The gun-ball coordinate conversion problem under the panoramic spliced video is solved, the accuracy is high, the algorithm is reliable, and the gun-ball coordinate conversion method does not depend on camera parameters and splicing parameters, and has high practical value.

Description

Technical Field

[0001] This invention belongs to the field of intelligent visual surveillance, specifically relating to a gun-sphere coordinate transformation method suitable for panoramic stitched images. Background Technology

[0002] The bullet-and-PTZ (PTZ) camera linkage technology, where a bullet camera captures panoramic images and a PTZ camera supplements the details of the bullet camera's view, is becoming increasingly widely used in the field of intelligent visual surveillance. However, because panoramic stitching requires various transformations of the original bullet camera images, the stitched panoramic image contains a large amount of distortion. This distortion differs from the optical distortion of traditional cameras and cannot be effectively corrected optically. This results in unsatisfactory conversion accuracy of the bullet-and-PTZ coordinates in the panoramic stitched image.

[0003] Traditional gun-to-ball linkage schemes involve randomly selecting three marked points on the gun image to determine the gun's image plane, and then using trigonometric relationships to calculate the deflection and elevation angles of the point to be switched on the PTZ camera. However, when this scheme is applied to stitched images, the distortion of the stitched image itself causes the image plane determined by the randomly selected three points to differ significantly from the actual image plane where the point to be switched is located.

[0004] Existing technologies mainly focus on the conversion between a bullet camera conforming to optical distortion and a wide-angle camera conforming to radial distortion. The main principle is to compensate for the image using optical principles before performing bullet-to-ball linkage processing. However, in addition to traditional optical distortion, there is also artificial distortion generated during the image stitching process, resulting in insufficient conversion accuracy between pixel coordinates and PTZ coordinates of panoramic stitched images when using existing technologies. Summary of the Invention

[0005] To address the shortcomings of the existing technology, the present invention aims to provide a method for converting PTZ coordinates between panoramic stitched images and PTZ coordinates of a PTZ camera, thereby solving the problem of insufficient conversion accuracy between pixel coordinates of panoramic stitched images and PTZ coordinates of a PTZ camera in the existing technology.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] The present invention provides a method for gun-sphere coordinate transformation applicable to panoramic stitched images, comprising the following steps:

[0008] 1) Select multiple annotation points on the panoramic stitched image that are closest to the point P to be converted, and use sets S0 and S1 to record the multiple annotation points and the vectors between the multiple annotation points and the point P to be converted, respectively;

[0009] 2) Perform logical judgments on the above sets S0 and S1, and select triangles that take any 3 panoramic stitched image annotation points in set S1 as vertices and can contain the point P to be converted; add the triangles that meet the conditions to set S2, and set S2 contains no repeated triangles.

[0010] 3) Determine whether a new triangle has appeared in set S2. If a new triangle has appeared, add the new triangle to set S3.

[0011] 4) Perform panoramic image side length density distortion determination on the triangles in set S3. If the triangle determination is successful, the iteration ends and proceeds to step 5); if the triangle determination fails, perform panoramic image side length density distortion determination on the next triangle.

[0012] 5) Input the three vertices of the triangle obtained in step 4) into the three-point conversion algorithm to obtain the accurate gun-ball conversion result.

[0013] Further, step 1) specifically includes:

[0014] The kd-tree algorithm is used to select the i nearest annotation points to the point P to be converted on the panoramic stitched image. The initial value of i is 3, and the points are B0, B1, B2, ..., B1 from nearest to farthest. i Label points B0, B1, B2...B i Add to set S0; label points B0, B1, B2...B i The vectors between the point P to be transformed are denoted as follows: Add these vectors to set S1.

[0015] Furthermore, step 2) specifically includes:

[0016] The last vector in set S1 with vector The included angle between them is used as a constraint condition. The initial value of j is 0, and j takes the values ​​0, 1, 2, ..., i-1. The marked points in set S0 are judged in turn, and all marked points in set S0 that meet the constraint condition are selected.

[0017] The specific constraint is: if the labeled points in set S0 are located in vector... with vector If the point is within the included angle between the two points, it is considered to meet the restriction conditions; otherwise, it does not meet the restriction conditions.

[0018] Add the triangle formed by all the labeled points in set S0 that meet the constraints and the two feature points corresponding to the two sets of vectors constituting the constraints in set S0 to set S2. Each triangle in set S2 can include the point P to be transformed.

[0019] Furthermore, step 3) specifically includes:

[0020] If a new triangle appears in set S2, the new triangle is added to set S3, and the k-th triangle in set S3 is output. The initial value of k is 0, and the maximum value of k is the number of triangles in set S3.

[0021] If no new triangle appears in set S2 or the value of k is greater than the number of triangles in set S3, then set S3 is cleared, and the process returns to step 2) to define the vectors constituting the constraints. Change the value of index j to j+1;

[0022] If vector Once all constraints formed by the remaining vectors in set S1 have been verified, and j+1>=i, return to step 1), and change the number of selected labeled points to i+1.

[0023] Furthermore, step 4) specifically includes: if all triangles in set S3 have been judged, then return to step 1) and increase the number of annotation points on the panoramic stitched image that are closest to the point P to be converted in step 1).

[0024] Furthermore, the method for determining the side length density distortion of the panoramic stitched image in step 4) is as follows:

[0025]

[0026] Among them, (x B1 ,y B1 ), (x B2 ,y B2 ), (x B3 ,y B3 ) represent the pixel coordinates of the three vertices of the newly added triangle on the panoramic stitched image, respectively. B1 ,t B1 ), (p B2 ,t B2 ), (p B3 ,t B3 ) represents the PTZ coordinates of the three vertices of the newly added triangle, t x ,t y For dynamic thresholds;

[0027] If the triangle satisfies formula (1), the iteration ends, and the triangle is used as the output to enter step 5);

[0028] If the triangle does not satisfy formula (1), return to step 3) and change the value of k in step 3) to k+1.

[0029] Further, the three-point conversion algorithm in step 5) is as follows: Let the three vertices of the triangle output in step 4) be A, B, and C; the pixel coordinates of the panoramic stitched image corresponding to points A, B, and C are respectively (x... A ,y A ),(x B ,y B ),(x C ,y C The PTZ coordinates of the PTZ camera are respectively

[0030] Where T is the image plane where the point P to be converted is located, point O is the location of the PTZ camera, which is also the origin of the PTZ coordinates of the PTZ camera; O′ is the mapping of point O on the image plane T;

[0031] Then the distance from point A to point B is AB, the distance from point B to point C is BC, and the distance from point C to point A is AC:

[0032]

[0033] In the spherical coordinate system corresponding to the PTZ coordinate system (origin is O, and the horizontal and vertical angles are the same as the yaw and pitch angles), the coordinates of points A, B, and C are transformed by the following formula (5):

[0034]

[0035] Let there be three points A', B', and C' in a spherical coordinate system, and vectors... For vectors The unit vector; and the vector For vectors The unit vector; and the vector For vectors , unit vector;

[0036] Then there are vectors unit vector for

[0037] vector unit vector for

[0038] vector unit vector for

[0039] Then there are vectors with vector The angle between them is equal to the unit vector With unit vector The included angle;

[0040] vector with vector The angle between them is equal to the unit vector With unit vector The included angle;

[0041] vector with vector The angle between them is equal to the unit vector With unit vector The included angle;

[0042] The distance from A' to B' is A'B', the distance from B' to C' is B'C', and the distance from C' to A' is A'C':

[0043]

[0044] By the Law of Cosines, Since OA' and OB' have a length of 1, the distance between A' and B' can be determined by formula (3), and ∠AOB can be calculated. Similarly, ∠BOC and ∠AOC can be calculated.

[0045] Assume AB=m, AC=n, BC=l, ∠AOB=α, ∠BOC=β, ∠AOC=γ, OA=x, OB=y, OC=z; then:

[0046] m 2 =x 2 +y 2 -2xycosα

[0047] n 2 =x 2 +z 2 -2xycosβ (7)

[0048] l 2 =y 2 +z 2 -2xycosγ

[0049] Where m, n, l, cosα, cosβ, cosγ are known, the lengths of OA, OB, and OC can be obtained from formula (4), and the spherical coordinates of O' can be further determined.

[0050] For point P to be converted, the corresponding pixel coordinates of the panoramic stitched image are (u,v). The lengths of PA and PO′ can be obtained from the distance formula between the two points, and thus the length of OP can be calculated. At this time, ∠AOP and ∠O'OP can be obtained by using trigonometric functions.

[0051]

[0052] Let the PTZ coordinates of the PTZ camera corresponding to P be... Then there are vectors unit vector for vector unit vector for vector unit vector for vector with vector The included angle ∠AOP equals and The included angle, vector with vector The included angle is equal to and The included angle;

[0053] Therefore, the lengths of A'P' and O”P' can be determined:

[0054]

[0055] Furthermore, according to the Law of Cosines:

[0056]

[0057] Substituting formula (8) into formula (10), we get:

[0058]

[0059] Combining formulas (9) and (11), we get:

[0060]

[0061]

[0062] Among them, besides the PTZ coordinates corresponding to the point P to be transformed All other quantities are known; solving the above equations will determine a unique set of PTZ coordinates corresponding to P.

[0063] The beneficial effects of this invention are:

[0064] This invention significantly improves the reliability of selecting annotation points for gun-sphere coordinate transformation in panoramic images by filtering specific triangles and determining side length density distortion, resulting in high accuracy of the gun-sphere coordinate transformation results. Furthermore, this invention is independent of camera parameters and image stitching parameters, making it highly valuable for practical applications. Attached Figure Description

[0065] Figure 1 This is a schematic diagram of the method of the present invention.

[0066] Figure 2A panoramic diagram showing the combination of multiple image planes.

[0067] Figure 3 This is a schematic diagram of the horizontal and vertical components of the pixel coordinates of three points.

[0068] Figure 4 This is a schematic diagram of the PTZ coordinate deflection angle and pitch angle of three points.

[0069] Figure 5 This is a diagram illustrating the correspondence between gun and ball conversion.

[0070] Figure 6 Example of filtering criteria Figure 1 .

[0071] Figure 7 Example of filtering criteria Figure 2 . Detailed Implementation

[0072] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0073] Reference Figure 1 As shown, the present invention provides a gun-sphere coordinate transformation method suitable for panoramic image stitching, comprising the following steps:

[0074] 1) Select multiple annotation points on the panoramic stitched image that are closest to the point P to be converted, and use sets S0 and S1 to record the multiple annotation points and the vectors between the multiple annotation points and the point P to be converted, respectively;

[0075] The kd-tree algorithm is used to select the i nearest annotation points to the point P to be converted on the panoramic stitched image. The initial value of i is 3, and the points are B0, B1, B2, ..., B1 from nearest to farthest. i Label points B0, B1, B2...B i Add to set S0; label points B0, B1, B2...B i The vectors between the point P to be transformed are denoted as follows: Add these vectors to set S1.

[0076] 2) Perform logical judgments on the above sets S0 and S1, and select triangles that take any 3 panoramic stitching image annotation points in set S1 as vertices and can contain the point P to be converted; add the triangles that meet the conditions to set S2, and set S2 has no duplicate triangles (if a newly added triangle to set S2 is the same as a triangle previously added to set S2, it will not be added).

[0077] The last vector in set S1 with vector The included angle between them is used as a constraint condition. The initial value of j is 0, and j takes the values ​​0, 1, 2, ..., i-1. The marked points in set S0 are judged in turn, and all marked points in set S0 that meet the constraint condition are selected.

[0078] The specific constraint is: if the labeled points in set S0 are located in vector... with vector If the point is within the included angle between the two points, it is considered to meet the constraint conditions; otherwise, it does not meet the constraint conditions, as follows: Figure 6 As shown, point B1 meets the filtering criteria of vectors B0P and B2P, and ΔB0B1B2 contains point P, as follows. Figure 7 As shown, B1 does not meet the filtering conditions of vectors B0P and B2P, and ΔB0B1B2 does not contain point P.

[0079] The set S0 contains all labeled points that meet the constraints, and the two feature points (e.g., vectors) corresponding to the two sets of vectors constituting the constraints in set S0. The triangle formed by the corresponding point B0 is added to set S2 (if set S2 already contains the new triangle, it is not added again). Each triangle in set S2 can include the point P to be transformed.

[0080] 3) Determine whether a new triangle has appeared in set S2. If a new triangle has appeared, add the new triangle to set S3.

[0081] If a new triangle appears in set S2, the new triangle is added to set S3, and the k-th triangle in set S3 is output. The initial value of k is 0, and the maximum value of k is the number of triangles in set S3.

[0082] If no new triangle appears in set S2 or the value of k is greater than the number of triangles in set S3, then set S3 is cleared, and the process returns to step 2) to define the vectors constituting the constraints. Change the value of index j to j+1;

[0083] If vector Once all constraints formed by the remaining vectors in set S1 have been verified (j+1>=i), return to step 1), and change the number of selected labeled points to i+1.

[0084] 4) Perform panoramic image side length density distortion determination on the triangles in set S3. If the triangle determination is successful, the iteration ends and proceeds to step 5); if the triangle determination fails, perform panoramic image side length density distortion determination on the next triangle.

[0085] If all triangles in set S3 have been judged, return to step 1) and increase the number of annotation points on the panoramic stitched image that are closest to the point P to be converted, as selected in step 1).

[0086] The method for determining the side length density distortion of the panoramic stitched image in step 4) is as follows:

[0087]

[0088] Among them, (x B1 ,y B1 ), (x B2 ,y B2 ), (x B3 ,y B3 ) represent the pixel coordinates of the three vertices of the newly added triangle on the panoramic stitched image, respectively. B1 ,t B1 ), (p B2 ,t B2 ), (p B3 ,t B3 ) represents the PTZ coordinates of the three vertices of the newly added triangle, t x ,t y For dynamic thresholds;

[0089] If the triangle satisfies formula (1), the iteration ends, and the triangle is used as the output to enter step 5);

[0090] If the triangle does not satisfy formula (1), return to step 3) and change the value of k in step 3) to k+1.

[0091] The distorted image from a panoramic stitch can be understood as a combination of multiple sufficiently small image planes, as shown in the reference. Figure 2 As shown;

[0092] All pixels within the triangular region formed by three sufficiently close pixels P1, P2, and P3 in the panoramic stitched image exhibit the same type of distortion and lie on the image plane determined by the spatial coordinates corresponding to points P1, P2, and P3.

[0093] Similarly, if there is distortion in the region formed by any three points in a panoramic stitched image, then the image plane containing that region can be understood as being formed by the combination of two or more image planes in space.

[0094] Select three labeled points A1, A2, and A3 on the panoramic stitched image, whose corresponding panoramic pixel coordinates are (x, y, y) and (x, y, y) respectively. A1 ,y A1 ), (x A2 ,y A2 ), (x A3 ,y A3The corresponding PTZ coordinates of the PTZ camera are (p A1 ,t A1 ), (p A2 ,t A2 ), (p A3 ,t A3 ), ΔA1A2A3 can contain the point P to be converted, and A1, A2, A3 are relatively close to P; if all pixels inside ΔA1A2A3 in the panoramic stitched image follow the same distortion, then the point P to be converted can be considered to be on the image plane determined by the spatial coordinates corresponding to points A1, A2, A3.

[0095] Since the PTZ coordinates of a single pixel in a panoramic stitched image do not have distortion issues, the PTZ coordinates of a pixel in a panoramic image are an important basis for determining whether all pixels within ΔA1A2A3 in a panoramic stitched image conform to the same type of distortion.

[0096] By comparing the side length and angular density (pixels / angle) of ΔA1A2A3, a judgment can be made on the distortion within the triangular region; refer to Figure 3 , Figure 4 As shown;

[0097] Since the distance between the three annotation points A1, A2, and A3 is relatively close, it can be assumed that if the three annotation points A1, A2, and A3 conform to the following formula (2), then all pixels in the ΔA1A2A3 region of the panoramic stitched image will follow the same distortion, and the image plane determined by the spatial coordinates of points A1, A2, and A3 can represent the image plane of the panoramic image region where the point P to be converted is located.

[0098]

[0099] Right now:

[0100]

[0101] t x t y A constant that approaches 0

[0102] (x) B1 ,y B1 ), (x B2 ,y B2 ), (x B3 ,y B3 ), (p B1 ,t B1 ), (p B2 ,t B2 ), (p B3 ,t B3 Substituting each into formula (3) will yield formula (1).

[0103] 5) Input the three vertices of the triangle obtained in step 4) into the three-point conversion algorithm to obtain the accurate gun-ball conversion result;

[0104] The three-point conversion algorithm is as follows: Let the three vertices of the triangle output in step 4) be A, B, and C; the pixel coordinates of the panoramic stitched image corresponding to points A, B, and C are respectively (x, y ... A ,y A ),(x B ,y B ),(x C ,y C The PTZ coordinates of the PTZ camera are respectively like Figure 5 As shown;

[0105] Where T is the image plane where the point P to be converted is located, point O is the location of the PTZ camera, which is also the origin of the PTZ coordinates of the PTZ camera; O′ is the mapping of point O on the image plane T;

[0106] Then the distance from point A to point B is AB, the distance from point B to point C is BC, and the distance from point C to point A is AC:

[0107]

[0108] In the spherical coordinate system corresponding to the PTZ coordinate system (origin is O, and the horizontal and vertical angles are the same as the yaw and pitch angles), the coordinates of points A, B, and C are transformed by the following formula (5):

[0109] x=rcosφcosθ

[0110] y=rcosφsinθ (5)

[0111] z = rsinφ

[0112] Let there be three points A', B', and C' in a spherical coordinate system, and vectors... For vectors The unit vector; and the vector For vectors The unit vector; and the vector For vectors , unit vector;

[0113] Then there are vectors unit vector for

[0114] vector unit vector for

[0115] vector unit vector for

[0116] Then there are vectors with vector The angle between them is equal to the unit vector With unit vector The included angle;

[0117] vector with vector The angle between them is equal to the unit vector With unit vector The included angle;

[0118] vector with vector The angle between them is equal to the unit vector With unit vector The included angle;

[0119] Distance from A' to B': Distance from B' to C': Distance from C' to A': Distance from C' to A':

[0120]

[0121] By the Law of Cosines, Since OA' and OB' have a length of 1, the distance between A' and B' can be determined by formula (3), and ∠AOB can be calculated. Similarly, ∠BOC and ∠AOC can be calculated.

[0122] Assume AB=m, AC=n, BC=l, ∠AOB=α, ∠BOC=β, ∠AOC=γ, OA=x, OB=y, OC=z; then:

[0123] m 2 =x 2 +y 2 -2xycosα

[0124] n 2 =x 2 +z 2 -2xycosβ (7)

[0125] l 2 =y 2 +z 2 -2xycosγ

[0126] Where m, n, l, cosα, cosβ, cosγ are known, the lengths of OA, OB, and OC can be obtained from formula (4), and the spherical coordinates of O' can be further determined.

[0127] For point P to be converted, the corresponding pixel coordinates of the panoramic stitched image are (u,v). The lengths of PA and PO′ can be obtained from the distance formula between the two points, and thus the length of OP can be calculated. At this time, ∠AOP and ∠O'OP can be obtained by using trigonometric functions.

[0128]

[0129] Let the PTZ coordinates of the PTZ camera corresponding to P be... Then there are vectors unit vector for vector unit vector for vector unit vector for vector with vector The included angle ∠AOP equals and The included angle, vector with vector The included angle is equal to and The included angle;

[0130] Therefore, the lengths of A'P' and O”P' can be determined:

[0131]

[0132] Furthermore, according to the Law of Cosines:

[0133]

[0134] Substituting formula (8) into formula (10), we get:

[0135]

[0136] Combining formulas (9) and (11), we get:

[0137]

[0138]

[0139] Among them, besides the PTZ coordinates corresponding to the point P to be transformed All other quantities are known; solving the above equations will determine a unique set of PTZ coordinates corresponding to P.

[0140] This invention has many specific applications. The above description is only a preferred embodiment of this invention. It should be noted that for those skilled in the art, several improvements can be made without departing from the principle of this invention, and these improvements should also be considered within the scope of protection of this invention.

Claims

1. A method for gun-sphere coordinate transformation suitable for panoramic stitched images, characterized in that, The steps are as follows: 1) Select multiple annotation points on the panoramic stitched image that are closest to the point P to be converted, and use sets S0 and S1 to record the multiple annotation points and the vectors between the multiple annotation points and the point P to be converted, respectively; 2) Perform logical judgments on the above sets S0 and S1, and select triangles that take any 3 panoramic stitched image annotation points in set S1 as vertices and can contain the point P to be converted; add the triangles that meet the conditions to set S2, and set S2 contains no repeated triangles. 3) Determine whether a new triangle has appeared in set S2. If a new triangle has appeared, add the new triangle to set S3. 4) Perform panoramic image side length density distortion determination on the triangles in set S3. If the triangle determination is successful, the iteration ends and proceeds to step 5); if the triangle determination fails, perform panoramic image side length density distortion determination on the next triangle. 5) Input the three vertices of the triangle obtained in step 4) into the three-point conversion algorithm to obtain the accurate gun-ball conversion result.

2. The gun-sphere coordinate transformation method for panoramic stitched images according to claim 1, characterized in that, Step 1) specifically includes: using the kd-tree algorithm to select the i closest annotation points to the point P to be converted on the panoramic stitched image, with the initial value of i being 3, and the points from closest to farthest being... Mark the points Add to set S0; label point The vectors between the point P to be transformed are denoted as follows: Add these vectors to set S1.

3. The gun-sphere coordinate transformation method for panoramic stitched images according to claim 2, characterized in that, Step 2) specifically includes: The last vector in set S1 with vector The included angle between them is used as a constraint condition. The initial value of j is 0, and j takes the values ​​0, 1, 2, ..., i-1. The marked points in set S0 are judged in turn, and all marked points in set S0 that meet the constraint condition are selected. Point O is the location of the PTZ camera. The specific constraint is: if the labeled points in set S0 are located in vector... with vector If the point is within the included angle between the two points, it is considered to meet the restriction conditions; otherwise, it does not meet the restriction conditions. Add the triangle formed by all the labeled points in set S0 that meet the constraints and the two feature points corresponding to the two sets of vectors constituting the constraints in set S0 to set S2. Each triangle in set S2 can include the point P to be transformed.

4. The gun-sphere coordinate transformation method for panoramic stitched images according to claim 3, characterized in that, Step 3) specifically includes: If a new triangle appears in set S2, the new triangle is added to set S3, and the k-th triangle in set S3 is output. The initial value of k is 0, and the maximum value of k is the number of triangles in set S3. If no new triangle appears in set S2 or the value of k is greater than the number of triangles in set S3, then clear set S3 and return to step 2) to define the vectors constituting the constraints. Change the value of index j to j+1; If vector Once all constraints formed by the remaining vectors in set S1 have been verified, and j+1>=i, return to step 1), and change the number of selected labeled points to i+1.

5. The gun-sphere coordinate transformation method for panoramic stitched images according to claim 4, characterized in that, Step 4) specifically includes: if all triangles in set S3 have been judged, then return to step 1) and increase the number of annotation points on the panoramic stitched image that are closest to the point P to be converted in step 1).

6. The gun-sphere coordinate transformation method for panoramic stitched images according to claim 4, characterized in that, The method for determining the side length density distortion of the panoramic stitched image in step 4) is as follows: （1）； in, These are the pixel coordinates of the three vertices of the newly added triangle on the panoramic stitched image. To add the PTZ coordinates of the three vertices of the new triangle, For dynamic thresholds; If the triangle satisfies formula (1), the iteration ends, and the triangle is used as the output to proceed to step 5). If the triangle does not satisfy formula (1), then return to step 3) and change the value of k in step 3) to k+1.

Images