Multi-scale measurement method based on three-dimensional space spherical coordinate system and two-axis holder

Abstract

The invention discloses a multi-scale measurement method based on a three-dimensional space spherical coordinate system and a two-axis holder, and belongs to the technical field of computer vision measurement. Experimental data are collected, and a multiplying power and view field deviation relation model is established; establishing a three-dimensional space spherical coordinate system; obtaining a target frame of the recognized target object, and calibrating the recognized target frame to a three-dimensional space spherical coordinate system under a large scale; after three-dimensional space registration is carried out on all the target objects, the center view angle of the holder is moved to the center of the target frame of the first target object; after the center of the holder is aligned with the center of the target frame, the magnification is continuously amplified to ensure that the minimum external frame of the target object is always kept in the visual field of the camera, and the image of the target object is amplified to the maximum extent. According to the method, the appropriate multiplying power is automatically selected and the viewing angle of the holder is accurately adjusted according to the angle range of the target frame and the view field offset model of the holder, so that the holder can accurately cover the target area under different multiplying power, and high-precision target object measurement is completed.

Description

Technical Field

[0001] The present invention relates to the field of computer vision measurement, and specifically refers to a multi-scale measurement method based on a three-dimensional space spherical coordinate system and a two-axis pan-tilt head. Background Art

[0002] In practical applications, when the pan-tilt head system performs high-precision target measurement and field of view adjustment, it is usually restricted by objective factors such as device performance, shooting angle, environmental conditions, and field of view range. These factors may lead to inaccurate field of view range and affect measurement accuracy. Especially in dynamic scenes, how to accurately adjust the pan-tilt head angle and automatically adapt to the size and position of the target object at different magnifications is still a technical problem. Currently, in the prior art, the calculation of field of view offset and magnification adjustment usually rely on manual settings or fixed formulas, lacking efficient modeling and accurate calculation of the relationship between the field of view range and the target frame angle. In a complex three-dimensional space environment, how to automatically adjust the pan-tilt head angle to ensure accurate capture of the entire target object still poses a great challenge. The innovation of the present invention lies in combining the cubic spline interpolation algorithm and the three-dimensional space spherical coordinate system to accurately establish a relationship model between magnification and field of view offset, realizing automatic adjustment of the viewing angle range of the pan-tilt head system at different magnifications. This method can automatically select the most appropriate magnification and accurately adjust the pan-tilt head angle in complex environments and dynamic scenes, thereby effectively overcoming the problems of field of view offset and angle calculation in traditional technologies, improving measurement accuracy, reducing human errors, and promoting the intelligent and automated development of pan-tilt head control and measurement. Summary of the Invention

[0003] The purpose of the present invention is to provide a multi-scale measurement method based on a three-dimensional space spherical coordinate system and a two-axis pan-tilt head to solve various problems occurring in the multi-scale measurement process of target objects as mentioned in the above background art.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions: A multi-scale measurement method based on a three-dimensional space spherical coordinate system and a two-axis pan-tilt head, comprising the following steps: Step1: Collect experimental data and establish a relationship model between magnification and field of view offset; Step2: Establish a three-dimensional space spherical coordinate system; Step3: Under the three-dimensional space spherical coordinate system established in Step2, obtain the target frame of the recognized target object, and calibrate multiple recognized target frames to the three-dimensional space spherical coordinate system at a large-scale magnification; Step4: Move the central viewing angle of the pan-tilt head to the center of the target frame of the first target object; Step 5: After the center of the pan-tilt and the center of the target box are aligned, calculate the viewing angle range at different magnifications through a cubic spline interpolation model. Continuously increase the magnification to ensure that the minimum bounding box of the target object always remains within the camera's field of view and maximize the magnified image of the target object.

[0005] Furthermore, the steps for establishing the relationship model in Step 1 are as follows: 1) Set the target point to the bottom leftmost position in the field of view of the two-axis pan-tilt at the current magnification setting, and use the center point of the pan-tilt as the initial position. Utilize the built-in sensor system of the pan-tilt to record the horizontal angle change ∆ θ h and the vertical angle change ∆ θ v when the pan-tilt rotates from the center point to the target point; 2) Conduct multiple experiments at magnification settings from 1× to 30×, and collect horizontal and vertical angle change data under different magnification conditions; 3) Adopt the cubic spline interpolation algorithm to fit the horizontal and vertical angle changes at different magnifications, and construct a model of vertical and horizontal magnification versus the field of view.

[0006] Furthermore, the method for establishing the three-dimensional spherical coordinate system in Step 2 is as follows: Take the center of the pan-tilt as the origin of the spherical coordinate system, map the horizontal rotation angle ∆ θ h of the pan-tilt to the longitude in the spherical coordinate system, map the vertical rotation angle ∆ θ v to the latitude in the spherical coordinate system, and define the radius r of the spherical coordinate system as a fixed value.

[0007] Furthermore, the specific steps for calibrating multiple identified target boxes to the three-dimensional spherical coordinate system at a large-scale magnification in Step 3 are as follows: 1) Obtain the upper left coordinate, width, and height of the target box, as well as the horizontal and vertical viewing angle ranges of the camera; 2) Calculate the proportional relationship between the upper left coordinate of the target box and the total width and height of the image, and based on the camera viewing angle range, calculate the viewing angle ranges of the target box in the horizontal and vertical directions. The horizontal viewing angle range is represented by the camera's horizontal viewing angle range [u min , u max , and the vertical viewing angle range is represented by the camera's vertical viewing angle range [v min , v max ; 3) According to the proportional relationship between the position of the target box in the image and the width and height of the image, map the target box from the image coordinate system to the camera's viewing angle range and calibrate it to the three-dimensional spherical coordinate system; 4) At a large-scale magnification, perform horizontal movement of the pan-tilt to identify all target objects and calibrate them all to the three-dimensional spherical coordinate system; 5) Perform non-maximum suppression algorithm processing on multiple spatially calibrated target boxes in a three-dimensional spherical coordinate system.

[0008] Further, in Step 4, move the central view of the pan-tilt to the center of the target box of the first target object. The specific steps are as follows: 1) The magnification at the large scale of the pan-tilt is m. Based on the constructed cubic spline interpolation model, obtain the field of view range at the current magnification. 2) Obtain the horizontal position and vertical position of the target box relative to the center of the pan-tilt. The horizontal position is expressed as the relative offset between the center of the target box and the center of the pan-tilt in the horizontal direction, and the vertical position is expressed as the relative offset between the center of the target box and the center of the pan-tilt in the vertical direction.

[0009] Further, the magnification adjustment is calculated based on the view angle range of the three-dimensional spherical coordinate system.

[0010] Advantages of the present invention: Through the multi-scale measurement method based on the three-dimensional spherical coordinate system and combined with the cubic spline interpolation algorithm, the present invention accurately solves the problem of adjusting the field of view range of the pan-tilt at different magnifications. This method can automatically select the appropriate magnification and accurately adjust the view angle of the pan-tilt according to the angle range of the target box and the field of view offset model of the pan-tilt, so as to ensure that the pan-tilt can accurately cover the target area at different magnifications and complete high-precision target object measurement. Through the YOLO algorithm and three-dimensional space registration, the recognition accuracy and positioning accuracy of the target object are further improved. In practical applications, the present invention effectively solves the problem of unstable field of view adjustment and measurement accuracy of the pan-tilt in multi-target and dynamic scenarios. This technology not only improves the intelligence and automation level of pan-tilt control and measurement, but also significantly improves work efficiency and reduces human errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a schematic diagram of establishing a three-dimensional spherical coordinate system; Figure 2 is a schematic diagram of performing three-dimensional space coordinate calibration. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] Step 1: Collect experimental data and establish a relationship model between magnification and field of view offset. The horizontal and vertical angle changes required for the pan-tilt to move from the center point to the target point. Select the target point and set it as the bottom left corner in the field of view at the current magnification of the pan-tilt, and set the center point of the pan-tilt as the initial position. Through the sensors built in the pan-tilt, record the horizontal angle and vertical angle changes each time the pan-tilt rotates from the center point to the target point. Multiple experiments were carried out at magnifications from 1 to 30, and the angle change values at magnification settings from 1× to 30× were collected as shown in Table 1:

[0013] Table 1: Table of horizontal and vertical angle change values at magnifications from 1 to 30 x ∆θh ∆θv 1 27.960592899399483 21.96487661273073 2 25.860877574093585 20.47795674049567 3 23.875457116817692 19.038768718011703 4 21.999565040047642 17.64898506490078 5 20.228691424408602 16.310157635670645 6 18.558582918675015 15.0237176197148 7 16.985242739770644 13.79097554131253 8 15.504930672768555 12.613121259628898 9 14.114163070891106 11.491223968714742 10 12.809712855509972 10.42623219750667 11 11.588609516146118 9.41897380982708 12 10.448139110469821 8.470156004384128 13 9.385844264300651 7.580365314771765 14 8.3995241716075 6.750067609469699 15 7.487234594508539 5.979608091843435 16 6.647287863271259 5.269211300144232 17 5.878252876312452 4.618981107509143 18 5.178955100198202 4.028900721960991 19 4.54847656964391 3.4988326864083596 20 3.9861558875142755 3.0285188786456487 21 3.491588224823296 2.617580511352987 22 3.064625320734269 2.2655181320963074 23 2.7053754825598126 1.9717116233273195 24 2.4142035857618396 1.7354202023834908 25 2.191731073951548 1.5557824214880824 26 2.0388359588894573 1.4318161677501244 27 1.9566528204853952 1.3624186631644228 28 1.9465728067984784 1.3463664646115596 29 2.0102436340371312 1.3823154638578963 30 2.1495695865590854 1.4688008875555667

[0014] Step 2: Establish a three-dimensional spherical coordinate system. As shown, take the center of the pan-tilt as the origin of the spherical coordinate system, map the horizontal rotation angle θh of the pan-tilt to the longitude in the spherical coordinate system, and map the vertical rotation angle θv to the latitude in the spherical coordinate system. Figure 1 As shown, take the center of the pan-tilt as the origin of the spherical coordinate system, map the horizontal rotation angle θh of the pan-tilt to the longitude in the spherical coordinate system, and map the vertical rotation angle θv to the latitude in the spherical coordinate system.

[0015] Step 3: The collected data is then subjected to cubic spline interpolation to construct the model. The horizontal angle change data is , and the vertical angle change data is The data of the horizontal angle and the vertical angle are respectively fitted by cubic spline interpolation. For each interval construct a cubic polynomial to represent the curve fitting within each interval. Cubic spline interpolation model for the horizontal angle: where are the coefficients to be determined.

[0016] Cubic spline interpolation model for the vertical angle: where are the coefficients to be determined.

[0017] For each data point and , list the values of the polynomial respectively: , for each adjacent interval, list the continuity condition of the first derivative: , for each adjacent interval, list the continuity condition of the second derivative: , for the boundary points and , list the condition that the second derivative is zero: , through the above conditions, a linear equation system can be obtained, in matrix form: Use the Gaussian elimination method to solve the equation system. Construct the augmented matrix from the equation system , . In the elimination process, the matrix A is transformed into an upper triangular matrix by row transformation, and the elements in the lower triangular part are eliminated. In the back substitution process, starting from the last row, each unknown is calculated step by step, and finally the solution vector c, that is, the coefficients of the cubic spline, is obtained. These coefficients define the smooth curve fitting the data, and the cubic spline interpolation model for the horizontal angle is constructed as S h (x), and the cubic spline interpolation model for the vertical angle is S v (x), which can be used to predict the horizontal and vertical angle changes corresponding to a given x magnification.

[0018] Use the yolov8 algorithm to train the recognition of target objects and apply the algorithm to the pan-tilt head. The pixel coordinates of the target box in the field of view of the pan-tilt head are: x min , y min represent the upper left corner coordinates of the target box, and w, h represent the width and height of the target box. The camera viewing angle range is defined by the horizontal viewing angle θ h and the vertical viewing angle θ v, which represent the viewing angles of the camera in the horizontal and vertical directions respectively. The width of the image is W img , and the height is H img . Through the relationship between the coordinates of the target box and the image size, it can be mapped to the camera viewing angle range. The proportion of the position of the upper left corner coordinates of the target box in the image width to the image width is , from which the viewing angle range ∆ θ h in the horizontal direction of the target box can be calculated: This formula associates the position of the target box in the image with the camera viewing angle range, and then obtains the horizontal viewing angle range of the target box in three-dimensional space. Similarly, the viewing angle range ∆ θ v in the vertical direction of the target box is calculated by the following formula: where y min is the position of the upper edge of the target box in the image, H img is the height of the image, θ v is the vertical viewing angle range of the camera.

[0019] The position of the target box is mapped from the image coordinate system to the camera viewing angle range, and these viewing angle ranges are further mapped to three-dimensional space to ensure the precise positioning of the target box in three-dimensional space. The horizontal direction angle range θ h_3D of the target box in three-dimensional space is calculated by the following formula: In the above formula θ h is the horizontal viewing angle range of the camera, is the proportion of the horizontal position of the upper left corner of the target box in the image, and the product of the two is the horizontal direction angle range of the target box in three-dimensional space. The vertical direction angle range θ v_3D of the target box can be calculated by the following formula: This formula combines the vertical position of the target box in the image with the vertical viewing angle range of the camera to calculate the vertical viewing angle range of the target box in three-dimensional space.

[0020] After mapping the target bounding box to three-dimensional coordinates, the next step is to adjust the horizontal angle of the pan-tilt head. At magnification m, the pan-tilt head is horizontally angled in both the left and right directions. Within a relatively large scale range, the position information of all target recognition objects is marked. As the angle of the pan-tilt head changes, the recognized objects will all be mapped into the three-dimensional spherical coordinate system of the pan-tilt head. To reduce possible duplicate markings during multiple detections, the non-maximum suppression algorithm is applied in the three-dimensional spherical coordinate system, and an appropriate threshold is set to remove redundant detection results, ensuring the uniqueness and accuracy of each steel coil in three-dimensional space. θ h_max : Represents the horizontal angle coordinate of the right edge of the bounding box, θ h_min Represents the horizontal angle coordinate of the left edge of the bounding box, θ v_max Represents the vertical angle coordinate of the upper edge of the bounding box, θ v_min Represents the vertical angle coordinate of the lower edge of the bounding box. θ h_max1 and θ h_min1 Represent the horizontal angle coordinates of the right and left edges of target bounding box 1, θ v_max1 and θ v_min1 Represent the vertical angle coordinates of the upper and lower edges of target bounding box 1, θ h_max2 and θ h_min2 Represent the horizontal angle coordinates of the right and left edges of target bounding box 2, θ v_max2 and θ v_min2 Represent the vertical angle coordinates of the upper and lower edges of target bounding box 2. Represents the overlapping width of two bounding boxes on the horizontal axis, and its calculation formula is: . Represents the overlapping height of two bounding boxes on the vertical axis. Its calculation formula is: Intersection area That is, the product of the overlapping width and the overlapping height. Its calculation formula is: If or is 0, it means there is no intersection and the intersection area is 0. The area area of the first bounding box is the product of its width and height, and the formula is: , and the area area2 of the second bounding box is the product of its width and height, and the formula is: , IOU represents the ratio of the overlapping area of two bounding boxes to their union area, and its calculation formula is: The larger the IOU value, the higher the overlapping degree of the two bounding boxes. When it is close to 1, it means that the two boxes almost completely overlap; while the smaller the IOU value, the lower the overlapping degree. When it is close to 0, it means that the two boxes hardly overlap. Set the threshold to 0.8, eliminate those bounding boxes with more overlap, and only retain the bounding boxes with higher confidence.

[0021] Step4: To achieve the small-scale recognition of each target object, the viewing angle of the pan-tilt needs to be precisely adjusted. Move the central viewing angle of the pan-tilt to the center of the first target box. The magnification of the pan-tilt is set to m. At this magnification, the width of the image resolution of the pan-tilt is w, and the height is h. The relative position of the recognized target box is represented by its horizontal position x and vertical position y from the center of the pan-tilt. x and y respectively represent the horizontal and vertical offsets of the target box relative to the center of the pan-tilt. The movement amount v of the pan-tilt in the horizontal direction is given by the following formula: , in the horizontal direction, the pan-tilt needs to move according to the ratio of the offset of the target box to the width of the image to adjust to the target position. In the vertical direction, the movement amount n of the pan-tilt is given by the following formula: , the adjustment of the pan-tilt in the vertical direction also needs to move according to the ratio of the vertical offset of the target box to the height of the image.

[0022] Step5: After the center of the pan-tilt moves to the center point of the target box, to maintain the complete presentation of the target box and maximize the magnification effect on the target object, the value of the magnification m needs to be specifically limited to the range between m min = 1 and m max = 30. The angle range θ h_min, θ h_max and θ v_min, θ v_max obtained by the initial detection of the pan-tilt, θ h_min and θ h_max are respectively the minimum and maximum angles of the target box in the horizontal direction; θ v_min and θ v_max are respectively the minimum and maximum angles of the target box in the vertical direction. To accurately determine the optimal magnification m_best that meets the angle requirements of the target box, an iterative search strategy based on binary search is adopted between the preset minimum magnification m min and the maximum magnification m max . The system initializes the search interval as [mlo , m hi , where m lo = m min , m hi= m max ; Continuously narrow the candidate interval until convergence through the following steps: Calculate the midpoint magnification m_mid = (m lo + m hi ) / 2 based on the current interval endpoints, and use the pre-established cubic spline interpolation models S h (x) and S v (x) to obtain the horizontal viewing angle range θ h_min (m_mid) , θ h_max (m_mid)] and the vertical viewing angle range θ v_min (m_mid) , θ v_max (m_mid)] at this magnification. Subsequently, compare the above interpolation results with the angle requirements of the target box in the horizontal and vertical directions θ h_min, θ h_max and θ v_min, θ v_max : If , and , it indicates that the complete coverage of the target box can be achieved at the current magnification m_mid. Therefore, update the lower bound m lo to m_mid to continue shrinking towards the higher magnification interval in subsequent iterations; otherwise, update the upper bound m hi to m_mid to exclude excessive magnification values. The above shrinking process continues until the interval width is less than the preset precision threshold of 0.5 or the number of iterations reaches the limit of 20 times. Finally, take m_best as m lo , that is, obtain the maximum magnification to ensure that the target box is always within the pan-tilt field of view, and then perform the zoom operation for small-scale recognition of the target.

[0023] The present invention can automatically select the optimal magnification and precisely control the pan-tilt viewing angle in complex environments and dynamic scenes, effectively overcoming the problems of field-of-view deviation and angle calculation error existing in traditional technologies. This technology significantly improves the measurement accuracy, reduces the error caused by human intervention, and promotes the development of pan-tilt control and measurement technologies towards intelligence and automation.

[0024] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural transformation made by using the content of the specification of the present invention under the inventive concept of the present invention, or any direct / indirect application in other related technical fields shall be included within the patent protection scope of the present invention.

Claims

1. A multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis gimbal, characterized in that: The following steps are involved: Step 1: Collect experimental data and establish a relationship model between magnification and field of view offset; Step 2: Establish a three-dimensional spherical coordinate system; Step 3: In the three-dimensional spherical coordinate system established in Step 2, the target frame of the identified target object is obtained, and the multiple identified target frames are calibrated to the three-dimensional spherical coordinate system at a large scale; Step 4: Move the center viewing angle of the gimbal to the center of the target frame of the first target object; Step 5: After the center of the gimbal is aligned with the center of the target frame, the viewing angle range at different magnifications is calculated using the cubic spline interpolation model. The magnification is continuously increased to ensure that the minimum bounding box of the target object always remains within the camera's field of view and to maximize the magnification of the target object's image.

2. The multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis pan-tilt head according to claim 1, characterized in that: The steps for establishing the relationship model in Step 1 are as follows: 1) Set the target point to the lower left corner of the field of view of the two-axis gimbal at the current magnification setting, and use the center point of the gimbal as the initial position. Use the built-in sensor system of the gimbal to record the horizontal angle change ∆ when the gimbal rotates from the center point to the target point θ h and vertical angle change ∆ θ v; 2) Multiple experiments were conducted at 1×–30× magnification settings to collect horizontal and vertical angle change data under different magnification conditions; 3) The cubic spline interpolation algorithm is used to fit the horizontal angle changes and vertical angle changes under different magnifications, and a model of vertical and horizontal magnification and field of view is constructed.

3. The multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis gimbal according to claim 1, characterized in that: The method of establishing the three-dimensional spherical coordinate system in Step 2 is: take the center of the gimbal as the origin of the spherical coordinate system, and rotate the gimbal horizontally by an angle ∆ θ h is mapped to longitude in spherical coordinates, and the vertical rotation angle ∆ θ v is mapped to the latitude in the spherical coordinate system, and the radius r of the spherical coordinate system is defined as a fixed value.

4. The multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis pan-tilt head according to claim 1, characterized in that: In Step 3, the specific steps of calibrating the identified multiple target frames to the three-dimensional space spherical coordinate system at a large scale are as follows: 1) Get the coordinates of the upper left corner, width and height of the target box, as well as the horizontal and vertical viewing angles of the camera; 2) Calculate the ratio between the upper left corner coordinates of the target frame and the total width and height of the image, and calculate the horizontal and vertical viewing angles of the target frame based on the camera viewing angle. The horizontal viewing angle is determined by the camera’s horizontal viewing angle [u min ,u max ] indicates that the vertical viewing angle range is determined by the vertical viewing angle range of the camera [v min ,v max ]express; 3) According to the proportional relationship between the position of the target frame in the image and the image width and height, the target frame is mapped from the image coordinate system to the camera's viewing angle range and calibrated in the three-dimensional spherical coordinate system; 4) Under large-scale magnification, the horizontal movement of the gimbal identifies all target objects and calibrates them all in the three-dimensional spherical coordinate system; 5) Perform non-maximum suppression algorithm processing on multiple spatially calibrated target boxes in a three-dimensional spherical coordinate system.

5. The multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis pan-tilt head according to claim 1, characterized in that: In Step 4, move the center viewing angle of the gimbal to the center of the target frame of the first target object. The specific steps are as follows: 1) The magnification of the gimbal at large scale is m. Based on the constructed cubic spline interpolation model, the field of view at the current magnification is obtained; 2) Get the horizontal and vertical positions of the target frame relative to the center of the gimbal. The horizontal position is expressed as the relative offset between the center of the target frame and the center of the gimbal in the horizontal direction, and the vertical position is expressed as the relative offset between the center of the target frame and the center of the gimbal in the vertical direction.

6. The multi-scale measurement method based on a three-dimensional spherical coordinate system and a two-axis gimbal according to claim 1, characterized in that: The magnification adjustment is calculated based on the viewing angle range of the three-dimensional space spherical coordinate system.

Images