A multi-scale measurement method based on three-dimensional spherical coordinate system and two-axis pan-tilt head

Abstract

The present invention discloses a multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis pan-tilt head, belonging to the field of computer vision measurement technology. The method collects experimental data, establishes a magnification and field of view offset relationship model, establishes a three-dimensional spherical coordinate system, obtains a target frame of an identified target object, and calibrates the identified target frame to the three-dimensional spherical coordinate system at a large scale. After three-dimensional space registration of all target objects, the center viewing angle of the pan-tilt head is moved to the center of the target frame of the first target object. After the pan-tilt head center and the target frame center are aligned, the magnification is continuously increased to ensure that the minimum bounding box of the target object always remains within the camera's field of view, and the image of the target object is maximized. The method automatically selects an appropriate magnification and accurately adjusts the pan-tilt head viewing angle based on the angular range of the target frame and the field of view offset model of the pan-tilt head, ensuring that the pan-tilt head can accurately cover the target area at different magnifications, thereby completing high-precision target object measurement.

Description

Technical Field

[0001] The present invention relates to the field of computer vision measurement, and in particular to a multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis pan-tilt platform. Background Art

[0002] In actual applications, when the pan-tilt system performs high-precision target measurement and field of view adjustment, it is usually restricted by objective factors such as equipment performance, shooting angle, environmental conditions and field of view range. These factors may lead to inaccurate field of view range and affect measurement accuracy. Especially in dynamic scenes, how to accurately adjust the pan-tilt viewing angle and automatically adapt to the size and position of the target object at different magnifications remains a technical problem. At present, the calculation of field of view offset and magnification adjustment in the existing technology usually rely on manual settings or fixed formulas, and lacks efficient modeling and precise calculation of the relationship between the field of view range and the target frame angle. In a complex three-dimensional space environment, how to automatically adjust the pan-tilt viewing angle to ensure accurate capture of the full picture of the target object still poses a great challenge. The innovation of the present invention lies in combining the cubic spline interpolation algorithm and the three-dimensional spherical coordinate system to accurately establish a magnification and field of view offset relationship model, so as to realize automatic adjustment of the viewing angle range of the pan-tilt system at different magnifications. This method can automatically select the most appropriate magnification and accurately adjust the pan-tilt viewing angle in complex environments and dynamic scenes, thereby effectively overcoming the field of view offset and angle calculation problems in traditional technologies, improving measurement accuracy, reducing human errors, and promoting the intelligent and automated development of pan-tilt control and measurement. Summary of the Invention

[0003] The purpose of the present invention is to provide a multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis pan-tilt head to solve the various problems arising in the multi-scale measurement process of target objects proposed in the above background technology.

[0004] To achieve the above object, the present invention adopts the following technical solution: a multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis pan-tilt head, comprising the following steps:

[0005] Step 1: Collect experimental data and establish a cubic spline interpolation model of magnification and field of view;

[0006] Step 2: Establish a three-dimensional spherical coordinate system, with the center of the gimbal as the origin of the spherical coordinate system, and rotate the gimbal horizontally by an angle of ∆ i h is mapped to longitude in spherical coordinates, and the vertical rotation angle ∆ i v is mapped to the latitude in the spherical coordinate system, and the radius r of the spherical coordinate system is defined as a fixed value;

[0007] Step 3: In the 3D spherical coordinate system established in Step 2, obtain the target frame of the identified target object, and calibrate the multiple identified target frames to the 3D spherical coordinate system at a large scale;

[0008] Step 4: Move the gimbal's center of view to the center of the target frame of the first target object;

[0009] Step 5: After aligning the gimbal center and the target frame center, use the cubic spline interpolation model to calculate the viewing angle range at different magnifications. Continuously increase the magnification to ensure that the minimum bounding box of the target object always remains within the camera's field of view and maximize the magnification of the target object's image.

[0010] Furthermore, the steps for establishing the relational model in Step 1 are as follows:

[0011] 1) Set the target point to the lower left corner of the FOV of the two-axis gimbal at the current magnification setting, and use the center point of the gimbal as the initial position. Use the gimbal's built-in sensor system to record the horizontal angle change ∆ when the gimbal rotates from the center point to the target point. i h and vertical angle change ∆ i v;

[0012] 2) Multiple experiments were conducted at magnifications of 1×–30× to collect horizontal and vertical angle change data under different magnification conditions;

[0013] 3) Using the cubic spline interpolation algorithm, the horizontal and vertical angle changes at different magnifications are fitted to construct a model of vertical and horizontal magnification and field of view.

[0014] Furthermore, in Step 3, the multiple identified target frames are calibrated to the three-dimensional spherical coordinate system at a large scale. The specific steps are as follows:

[0015] 1) Get the coordinates of the upper left corner, width and height of the target box, as well as the horizontal and vertical viewing angles of the camera;

[0016] 2) Calculate the proportional relationship between the upper left corner coordinates of the target frame and the image width and height, and calculate the horizontal and vertical viewing angles of the target frame based on the camera viewing angle range. The horizontal viewing angle range is determined by the camera's horizontal viewing angle range [u min ,u max ] indicates that the vertical viewing angle range is represented by the vertical viewing angle range of the camera [v min ,v max ]express;

[0017] 3) Based on the proportional relationship between the target frame’s position in the image and the image width and height, the target frame is mapped from the image coordinate system to the camera’s viewing angle and calibrated in a 3D spherical coordinate system.

[0018] 4) Under large-scale magnification, the horizontal movement of the gimbal identifies all target objects and calibrates them all in a three-dimensional spherical coordinate system;

[0019] 5) Perform non-maximum suppression algorithm processing on multiple spatially calibrated target boxes in a three-dimensional spherical coordinate system.

[0020] Furthermore, in Step 4, the center of the gimbal's viewing angle is moved to the center of the target frame of the first target object. The specific steps are as follows:

[0021] 1) The magnification of the gimbal at large scale is m. Based on the constructed cubic spline interpolation model, the field of view at the current magnification is obtained;

[0022] 2) Get the horizontal and vertical positions of the target frame relative to the gimbal center. The horizontal position is expressed as the relative offset between the target frame center and the gimbal center in the horizontal direction, and the vertical position is expressed as the relative offset between the target frame center and the gimbal center in the vertical direction.

[0023] Furthermore, the magnification adjustment is calculated based on the viewing angle range of the three-dimensional space spherical coordinate system.

[0024] Beneficial effects of the present invention: The present invention accurately solves the problem of adjusting the field of view range of the pan-tilt head at different magnifications through a multi-scale measurement method based on a three-dimensional spherical coordinate system, combined with a cubic spline interpolation algorithm. This method can automatically select a suitable magnification and accurately adjust the pan-tilt head viewing angle according to the angular range of the target frame and the field of view offset model of the pan-tilt head, thereby ensuring that the pan-tilt head can accurately cover the target area at different magnifications and complete high-precision target object measurement. The recognition accuracy and positioning accuracy of the target object are further improved through the YOLO algorithm and three-dimensional space registration. In practical applications, the present invention effectively solves the problem of unstable pan-tilt head field of view adjustment and measurement accuracy in multi-target and dynamic scenes. This technology not only improves the intelligence and automation level of pan-tilt head control and measurement, but also significantly improves work efficiency and reduces human errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of establishing a three-dimensional spherical coordinate system;

[0026] Figure 2 It is a schematic diagram of three-dimensional space coordinate calibration. DETAILED DESCRIPTION

[0027] Step 1: Collect experimental data and establish a relationship model between magnification and field of view offset. The horizontal and vertical angle changes required for the gimbal to move from the center point to the target point were calculated. The target point was selected and set to the lower left corner of the field of view at the gimbal's current magnification, and the center point of the gimbal was set as the initial position. The gimbal's built-in sensor recorded the changes in horizontal and vertical angles each time the gimbal rotated from the center point to the target point. Multiple experiments were conducted at magnifications of 1-30, and the angle change values ​​collected at magnification settings of 1×–30× are shown in Table 1:

[0028] Table 1: Horizontal and vertical angle changes at magnifications of 1-30

[0029] x Dthh ∆θv 1 27.960592899399483 21.96487661273073 2 25.860877574093585 20.47795674049567 3 23.875457116817692 19.038768718011703 4 21.999565040047642 17.64898506490078 5 20.228691424408602 16.310157635670645 6 18.558582918675015 15.0237176197148 7 16.985242739770644 13.79097554131253 8 15.504930672768555 12.613121259628898 9 14.114163070891106 11.491223968714742 10 12.809712855509972 10.42623219750667 11 11.588609516146118 9.41897380982708 12 10.448139110469821 8.470156004384128 13 9.385844264300651 7.580365314771765 14 8.3995241716075 6.750067609469699 15 7.487234594508539 5.979608091843435 16 6.647287863271259 5.269211300144232 17 5.878252876312452 4.618981107509143 18 5.178955100198202 4.028900721960991 19 4.54847656964391 3.4988326864083596 20 3.9861558875142755 3.0285188786456487 21 3.491588224823296 2.617580511352987 22 3.064625320734269 2.2655181320963074 23 2.7053754825598126 1.9717116233273195 24 2.4142035857618396 1.7354202023834908 25 2.191731073951548 1.5557824214880824 26 2.0388359588894573 1.4318161677501244 27 1.9566528204853952 1.3624186631644228 28 1.9465728067984784 1.3463664646115596 29 2.0102436340371312 1.3823154638578963 30 2.1495695865590854 1.4688008875555667

[0030] Step 2: Establish a three-dimensional spherical coordinate system, such as Figure 1 As shown, the center of the gimbal is taken as the origin of the spherical coordinate system, the horizontal rotation angle θh of the gimbal is mapped to the longitude in the spherical coordinate system, and the vertical rotation angle θv is mapped to the latitude in the spherical coordinate system.

[0031] Step 3: The collected data is then interpolated with cubic spline to construct the model. The horizontal angle change data is , the vertical angle change data is The horizontal angle and vertical angle data are fitted separately by cubic spline interpolation, and each interval Construct a cubic polynomial to represent the curve fitting in each interval. Horizontal angle cubic spline interpolation model: in, is the coefficient to be determined.

[0032] Vertical angle cubic spline interpolation model: in, is the coefficient to be determined.

[0033] For each data point and , list the values ​​of the polynomials separately: , for each adjacent interval, list the continuity conditions for the first-order derivative: , for each adjacent interval, list the continuity conditions for the second-order derivative: , for the boundary points and , list the conditions for the second-order derivative to be zero: , through the above conditions, we can get a linear equation system, the matrix form is: Use Gaussian elimination method to solve the system of equations. Constructing the augmented matrix , The elimination process transforms the matrix A The matrix is ​​transformed into an upper triangular matrix, and the elements of the lower triangular part are eliminated. The back substitution process starts from the last row, and each unknown number is calculated step by step, and finally the solution vector c is obtained, which is the coefficient of the cubic spline. These coefficients define the smooth curve of the fitted data, and the horizontal angle cubic spline interpolation model is constructed as S h (x) The vertical angle cubic spline interpolation model is S v (x), which can be used to predict the horizontal and vertical angle changes corresponding to a given x magnification.

[0034] Use the yolov8 algorithm to train the target object recognition and apply the algorithm to the gimbal. The pixel coordinates of the target box in the gimbal's field of view are: x min ,y min The coordinates w and h of the upper left corner of the target frame represent the width and height of the target frame, and the camera viewing angle range is composed of the horizontal viewing angle i h and vertical viewing angle i v is defined as the camera's field of view in the horizontal and vertical directions. The width of the image is W img , the height is H img , through the relationship between the coordinates of the target frame and the image size, it can be mapped to the camera's viewing angle range. The coordinates of the upper left corner of the target frame The ratio of the position on the image width to the image width is , from which the horizontal viewing angle range of the target frame ∆ can be calculated i h: This formula relates the position of the target frame in the image to the camera's viewing angle, and thus derives the horizontal viewing angle of the target frame in three-dimensional space. Similarly, the vertical viewing angle of the target frame is ∆ i v is calculated using the following formula: Among them, y min is the position of the upper edge of the target box in the image, H img is the height of the image, i v is the vertical viewing angle of the camera.

[0035] The position of the target frame is mapped from the image coordinate system to the camera's viewing angle range, and these viewing angle ranges are further mapped to three-dimensional space to ensure the accurate positioning of the target frame in three-dimensional space. The horizontal angle range of the target frame in three-dimensional space i h_3D Calculated by the following formula: In the above formula i h is the horizontal viewing angle of the camera, is the horizontal position ratio of the upper left corner of the target frame in the image, and the product of the two is the horizontal angle range of the target frame in three-dimensional space. The vertical angle range of the target frame i v_3D It can be calculated by the following formula: This formula combines the vertical position of the target frame in the image with the vertical viewing angle of the camera to calculate the vertical viewing angle of the target frame in three-dimensional space.

[0036] After mapping the target frame to three-dimensional coordinates, the next step is to adjust the horizontal angle of the gimbal. At a magnification of m, the gimbal is moved horizontally in the left and right directions. Within a large scale range, the position information of all target identification objects is marked. As the gimbal angle changes, all identified objects will be mapped to the gimbal's three-dimensional spherical coordinate system. To reduce the possibility of repeated markings during multiple inspections, a non-maximum suppression algorithm is applied to the three-dimensional spherical coordinate system, and an appropriate threshold is set to remove redundant detection results, ensuring the uniqueness and accuracy of each steel coil in three-dimensional space. i h_max : represents the horizontal angle coordinate of the right edge of the bounding box, i h_min Represents the horizontal angle coordinate of the left edge of the bounding box, i v_max Represents the vertical angle coordinate of the top edge of the bounding box, i v_min The vertical angle coordinate representing the bottom edge of the bounding box. i h_max1 and i h_min1 Indicates that the target frame 1 is at the right edge and the left edge at the horizontal angle coordinate, i v_max1 and i v_min1 Indicates that the target frame 1 is at the vertical angle coordinates of the upper and lower edges, i h_max2 and i h_min2 Indicates that the target frame 2 is at the right edge and the left edge in the horizontal angle coordinates, i v_max2 and i v_min2 Indicates the vertical angle coordinates of the upper and lower edges of target box 2. Represents the overlapping width of two bounding boxes on the horizontal axis, and its calculation formula is: . Indicates the overlapping height of two bounding boxes on the vertical axis. Its calculation formula is: Intersection area It is the product of overlap width and overlap height. Its calculation formula is: if or If it is 0, it means there is no intersection and the intersection area is 0. The area of ​​the first bounding box area 1 is the product of its width and height, and the formula is: , the area of ​​the second bounding box area2 The product of its width and height is: , IOU represents the ratio of the overlapping area of ​​two bounding boxes to their union area, and its calculation formula is: The larger the IOU value, the higher the degree of overlap between the two bounding boxes. When it is close to 1, it means that the two boxes almost completely overlap. The smaller the IOU value, the lower the degree of overlap. When it is close to 0, it means that the two boxes have almost no overlap. Set the threshold to 0.8 to eliminate those bounding boxes with more overlap and retain only the bounding boxes with higher confidence.

[0037] Step 4: To achieve small-scale recognition of each target object, the gimbal's viewing angle needs to be precisely adjusted. Move the gimbal's central viewing angle to the center of the first target frame. The gimbal's magnification is set to m. At this magnification, the gimbal's image resolution has a width of w and a height of h. The relative position of the identified target frame is represented by its horizontal position x and vertical position y from the gimbal's center. x and y represent the horizontal and vertical offsets of the target frame relative to the gimbal's center, respectively. The horizontal movement v of the gimbal is given by the following formula: In the horizontal direction, the gimbal needs to move according to the ratio of the target frame offset to the image width in order to adjust to the target position. In the vertical direction, the gimbal movement n is given by the following formula: ,The vertical adjustment of the gimbal also needs to move according to the ratio of the vertical offset of the target frame and the height of the image.

[0038] Step 5: After the gimbal center moves to the center of the target frame, it can maintain the complete presentation of the target frame and maximize the magnification effect of the target object. The value of the magnification m must be specifically limited to m min =1 and m max =30. The angle range corresponding to the target frame obtained by the initial detection of the gimbal is [ i h_min, i h_max ]and[ i v_min, i v_max ], i h_min and i h_max are the minimum and maximum angles of the target box in the horizontal direction; iv_min and i v_max are the minimum and maximum angles of the target frame in the vertical direction. In order to accurately determine the optimal magnification m_best that meets the target frame angle requirements, the preset minimum magnification m is used. min With the maximum magnification m max The system initializes the search interval to [m lo ,m hi ], where m lo =m min ,m hi= m max ; Continue to narrow the candidate interval through the following steps until convergence: Calculate the midpoint magnification m_mid=(m lo +m hi ) / 2, and using the pre-established cubic spline interpolation model S h (x) and S v (x) respectively obtain the horizontal viewing angle range under the magnification [ i h_min (m_mid) , i h_max (m_mid)] and vertical viewing angle range [ i v_min (m_mid) , i v_max (m_mid)]. Then, compare the above interpolation results with the angle requirements of the target frame in the horizontal and vertical directions [ i h_min, i h_max ]and[ i v_min, i v_max ]:If satisfied ,as well as This indicates that the current magnification m_mid can achieve complete coverage of the target frame, so the lower bound m lo Update to m_mid to continue shrinking to a higher rate range in subsequent iterations; otherwise, update the upper bound m hi The above shrinking process continues until the interval width is m_mid to exclude excessive magnification values. Less than the preset accuracy threshold of 0.5 or the number of iterations reaches the upper limit of 20 times. Finally, m_best is taken as m lo , that is, to obtain the maximum magnification, ensure that the target frame is always in the field of view of the gimbal, and then perform the zoom operation to perform small-scale recognition of the target.

[0039] This invention automatically selects the optimal magnification and precisely controls the pan / tilt viewing angle in complex environments and dynamic scenes, effectively overcoming the field of view offset and angle calculation errors found in traditional technologies. This technology significantly improves measurement accuracy, reduces errors caused by human intervention, and promotes the development of pan / tilt control and measurement technology towards intelligent and automated development.

[0040] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformation made by using the contents of the present invention specification under the inventive concept of the present invention, or direct / indirect application in other related technical fields is included in the patent protection scope of the present invention.

Claims

1. A multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis pan-tilt head, characterized in that: The following steps are involved: Step 1: Collect experimental data and establish a cubic spline interpolation model of magnification and field of view offset; Step 2: Establish a three-dimensional spherical coordinate system, with the center of the gimbal as the origin of the spherical coordinate system, and rotate the gimbal horizontally by an angle of ∆ θ h is mapped to longitude in spherical coordinates, and the vertical rotation angle ∆ θ v is mapped to the latitude in the spherical coordinate system, and the radius r of the spherical coordinate system is defined as a fixed value; Step 3: In the 3D spherical coordinate system established in Step 2, obtain the target frame of the identified target object, and calibrate the multiple identified target frames to the 3D spherical coordinate system at a large scale; Step 4: Move the gimbal's center of view to the center of the target frame of the first target object; Step 5: After aligning the gimbal center and the target frame center, use the cubic spline interpolation model to calculate the viewing angle range at different magnifications. Continuously increase the magnification to ensure that the minimum bounding box of the target object always remains within the camera's field of view and maximize the magnification of the target object's image.

2. The multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis pan-tilt head according to claim 1, characterized in that: The steps for establishing the relational model in Step 1 are as follows: 1) Set the target point to the lower left corner of the FOV of the two-axis gimbal at the current magnification setting, and use the center point of the gimbal as the initial position. Use the gimbal's built-in sensor system to record the horizontal angle change ∆ when the gimbal rotates from the center point to the target point. θ h and vertical angle change ∆ θ v; 2) Multiple experiments were conducted at magnifications of 1×–30× to collect horizontal and vertical angle change data under different magnification conditions; 3) Using the cubic spline interpolation algorithm, the horizontal and vertical angle changes at different magnifications are fitted to construct a model of vertical and horizontal magnification and field of view.

3. The multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis pan-tilt head according to claim 1, characterized in that: In Step 3, the specific steps for calibrating the identified multiple target frames to the three-dimensional spherical coordinate system at a large scale are as follows: 1) Get the coordinates of the upper left corner, width and height of the target box, as well as the horizontal and vertical viewing angles of the camera; 2) Calculate the proportional relationship between the upper left corner coordinates of the target frame and the image width and height, and calculate the horizontal and vertical viewing angles of the target frame based on the camera viewing angle range. The horizontal viewing angle range is determined by the camera's horizontal viewing angle range [u min ,u max ] indicates that the vertical viewing angle range is represented by the vertical viewing angle range of the camera [v min ,v max ]express; 3) Based on the proportional relationship between the target frame’s position in the image and the image width and height, the target frame is mapped from the image coordinate system to the camera’s viewing angle and calibrated in a 3D spherical coordinate system. 4) Under large-scale magnification, the horizontal movement of the gimbal identifies all target objects and calibrates them all in a three-dimensional spherical coordinate system; 5) Perform non-maximum suppression algorithm processing on multiple spatially calibrated target boxes in a three-dimensional spherical coordinate system.

4. The multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis pan-tilt head according to claim 1, characterized in that: In Step 4, move the center of the gimbal's view to the center of the target frame of the first target object. The specific steps are as follows: 1) The magnification of the gimbal at large scale is m. Based on the constructed cubic spline interpolation model, the field of view at the current magnification is obtained; 2) Get the horizontal and vertical positions of the target frame relative to the gimbal center. The horizontal position is expressed as the relative offset between the target frame center and the gimbal center in the horizontal direction, and the vertical position is expressed as the relative offset between the target frame center and the gimbal center in the vertical direction.

5. The multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis pan-tilt head according to claim 1, characterized in that: The magnification adjustment is calculated based on the viewing angle range of the three-dimensional spherical coordinate system.

Images