Dual-camera target tracking method and device, storage medium and electronic equipment

Abstract

The application discloses a target tracking method and device of a dual-camera, a storage medium and an electronic device, and relates to the field of computer vision. The method comprises the following steps: acquiring a to-be-detected image captured by a main camera and a calibration angle control equation of an angle adjusting device of a secondary camera in a horizontal axis direction, wherein the calibration angle control equation is obtained based on multiple identification data; performing target detection on the to-be-detected image to obtain detection frame information corresponding to a target object; determining a tracking angle of the angle adjusting device in the horizontal axis direction according to the detection frame information and the calibration angle control equation; and controlling the angle adjusting device to rotate along the horizontal axis direction based on the tracking angle in the horizontal axis direction, so as to realize tracking of the target object. The application can reduce the requirement for the computing power of a terminal device, thereby reducing the cost of the device.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a dual-camera target tracking method, apparatus, storage medium, and electronic device. Background Technology

[0002] In recent years, products in fields such as audio and video conferencing and classroom recording have become increasingly intelligent, and are generally equipped with functions such as human tracking and intelligent framing. As people's demands for image quality increase, related visual modules have gradually transitioned from single-viewing to multi-viewing. Common dual-camera solutions can be divided into fixed dual-camera and "fixed-one-moving" dual-camera solutions. Fixed dual-camera solutions have both cameras fixed; however, due to the limited close-up coverage of the secondary camera, this type of dual-camera solution has limited practicality for tracking and close-ups. "Fixed-one-moving" dual-camera solutions, on the other hand, have the primary camera fixed while the secondary camera can rotate left and right and / or up and down, thus achieving a larger close-up coverage area, making it more practical.

[0003] Currently, for dual-camera systems with one camera fixed and one moving, AI detection is typically performed on both the main and secondary cameras for target localization or tracking. However, since both the main and secondary cameras require AI detection, this places high demands on the computing power of the terminal equipment, increasing equipment costs. Summary of the Invention

[0004] In view of this, this application provides a target tracking method, apparatus, storage medium and electronic device with dual cameras, the main purpose of which is to reduce the computing power requirements of terminal devices, thereby reducing equipment costs.

[0005] According to a first aspect of this application, a dual-camera target tracking method is provided, the method comprising:

[0006] The image to be detected captured by the main camera and the calibration angle control equation of the angle adjustment device of the secondary camera in the horizontal direction are obtained, wherein the calibration angle control equation is obtained by fitting multiple identification data.

[0007] Target detection is performed on the image to be detected to obtain the detection box information corresponding to the target object;

[0008] Based on the detection frame information and the calibration angle control equation, the tracking angle of the angle adjustment device in the horizontal axis direction is determined;

[0009] Based on the tracking angle in the horizontal axis direction, the angle adjustment device is controlled to rotate along the horizontal axis direction to achieve tracking of the target object.

[0010] According to a second aspect of this application, a dual-camera target tracking device is provided, the device comprising:

[0011] The acquisition unit is used to acquire the image to be detected captured by the main camera, and the calibration angle control equation of the angle adjustment device of the secondary camera in the horizontal axis direction, wherein the calibration angle control equation is obtained by fitting multiple identification data.

[0012] The detection unit is used to perform target detection on the image to be detected and obtain the detection box information corresponding to the target object;

[0013] The determining unit is used to determine the tracking angle of the angle adjustment device in the horizontal axis direction based on the detection frame information and the calibration angle control equation.

[0014] The control unit is used to control the angle adjustment device to rotate along the horizontal axis based on the tracking angle in the horizontal axis direction, so as to achieve tracking of the target object.

[0015] According to a third aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described dual-camera target tracking method.

[0016] According to a fourth aspect of this application, an electronic device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described dual-camera target tracking method.

[0017] According to a fifth aspect of this application, a camera device is provided, the camera device being equipped with the aforementioned dual-camera target tracking device.

[0018] According to a sixth aspect of this application, a video system is provided, the video system including a display and a camera device, the display being communicatively connected to the camera device, wherein the camera device is equipped with the aforementioned dual-camera target tracking device; the camera device is configured to acquire an image to be detected captured by the main camera, and a calibration angle control equation of the angle adjustment device of the secondary camera in the horizontal direction, wherein the calibration angle control equation is obtained based on fitting multiple identifier data; perform target detection on the image to be detected to obtain detection box information corresponding to the target object; determine the tracking angle of the angle adjustment device in the horizontal direction according to the detection box information and the calibration angle control equation; and control the angle adjustment device to rotate along the horizontal direction based on the tracking angle in the horizontal direction to achieve tracking of the target object.

[0019] By employing the above technical solution, the target tracking method, apparatus, storage medium, and electronic device provided in this application, compared with the prior art, improve the calibration accuracy of the dual cameras because this application is based on the calibration angle control equation of multi-point fitting. Therefore, during target tracking, only AI detection needs to be implemented on the main camera. Specifically, the rotation angle of the angle adjustment device along the horizontal axis can be determined based on the detection box information of the target object obtained by AI detection and the calibration angle control equation, thereby achieving target tracking. In contrast, since this application only requires AI detection on the main camera, it can reduce the computing power requirements of the terminal device, thereby reducing equipment costs.

[0020] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 A flowchart illustrating a dual-camera target tracking method provided in an embodiment of this application is shown.

[0023] Figure 2 A flowchart illustrating another dual-camera target tracking method provided in an embodiment of this application is shown;

[0024] Figure 3 A schematic diagram of the desired imaging provided in an embodiment of this application is shown;

[0025] Figure 4 A flowchart illustrating the target tracking process provided in an embodiment of this application is shown.

[0026] Figure 5 A schematic diagram of the calibration plate provided in an embodiment of this application is shown;

[0027] Figure 6 A schematic diagram of the calibration plate specifications provided in an embodiment of this application is shown;

[0028] Figure 7 A schematic diagram of camera installation provided in an embodiment of this application is shown;

[0029] Figure 8 A flowchart illustrating the calibration process in the horizontal direction provided in an embodiment of this application is shown;

[0030] Figure 9 This illustration shows a schematic diagram of a calibration plate region division provided in an embodiment of this application;

[0031] Figure 10 This illustration shows a schematic diagram of a calibration plate region division provided in an embodiment of this application;

[0032] Figure 11 This illustration shows the imaging diagrams of the main camera and the secondary camera before calibration, provided in an embodiment of this application.

[0033] Figure 12 A flowchart illustrating the calibration process in the longitudinal direction provided in an embodiment of this application is shown;

[0034] Figure 13 This application provides a comparison diagram showing the effect before and after vertical axis calibration according to an embodiment of the present application;

[0035] Figure 14 This illustration shows a schematic diagram of the structure of a dual-camera target tracking device according to an embodiment of this application;

[0036] Figure 15 A schematic diagram of another dual-camera target tracking device provided in an embodiment of this application is shown. Detailed Implementation

[0037] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0038] To address the technical challenges of high computing power requirements and high equipment costs associated with existing positioning methods, this embodiment provides a dual-camera target tracking method applied to the main camera, such as... Figure 1 As shown, the method includes:

[0039] Step 101: Obtain the image to be detected captured by the main camera, and the calibration angle control equation of the angle adjustment device of the secondary camera in the horizontal axis direction.

[0040] The system comprises a fixed main camera (panoramic camera) equipped with a standard wide-angle lens and an embedded processing chip with AI acceleration capabilities. The secondary camera (tracking camera) features a telephoto lens and an angle adjustment device, which can be a mechanical gimbal or a rotating mirror mechanism. The secondary camera can swing left and right and / or up and down via the gimbal or mirror mechanism. Alternatively, the main camera can be equipped with an (ultra) wide-angle lens, and the secondary camera with a telephoto lens; or both the main and secondary cameras can be equipped with telephoto lenses, the purpose being to create different field-of-view angles for the two cameras. Furthermore, the calibration angle control equation is obtained by fitting multiple identifier data. The input to the calibration angle control equation is the abscissa of the center of the target object in the image captured by the main camera, and the output is the rotation angle of the angle adjustment device along the horizontal axis. The image to be detected is the image captured by the main camera that will be used for target detection.

[0041] In this embodiment of the invention, when tracking a target, it is necessary to first acquire the image to be detected captured by the main camera, and the calibration angle control equation of the angle adjustment device in the horizontal axis direction, which is determined in advance during the calibration process. The specific formula of the calibration angle control equation is as follows:

[0042] γ=R(x)=ω0·X 2 +ω1·X+b,0≤X<1

[0043] Where X is the horizontal coordinate of the center of the target object in the image to be detected captured by the main camera, and γ is the tracking angle of the angle adjustment device, that is, the rotation angle of the angle adjustment device along the horizontal axis.

[0044] Step 102: Perform target detection on the image to be detected to obtain the detection box information corresponding to the target object.

[0045] The target object can be a human body, vehicle, animal, or other target. The detection box information includes x, y, w, and h, where x and y represent the coordinates of the center point of the detection box, and w and h represent the width and length of the detection box.

[0046] In this embodiment of the invention, a preset target detection algorithm is used to perform target detection on the image to be detected, and the detection box information of the target object in the image to be detected is obtained. The detection box information specifically includes the horizontal coordinate information x and the vertical coordinate information y of the center point of the detection box, as well as the width information w and the length information h of the detection box.

[0047] It should be noted that the preset target detection algorithm used in the embodiments of the present invention may specifically be R-CNN, Faster R-CNN, YOLO, SSD, Mask R-CNN, etc., but is not limited thereto.

[0048] Step 103: Determine the tracking angle of the angle adjustment device in the horizontal direction based on the detection frame information and the calibration angle control equation.

[0049] In this embodiment of the invention, after obtaining the detection box information of the target object, the center coordinates (x0, y0) of the target object can be determined based on the detection box information, such as... Figure 3 As shown, the center x0 of the target object's center coordinates (x0, y0) is then substituted into the governing equation γ = ω0·X 2 +ω1·X+b gives the tracking angle γ of the secondary camera in the horizontal direction.

[0050] Step 104: Based on the tracking angle in the horizontal axis direction, control the angle adjustment device to rotate along the horizontal axis direction to achieve tracking of the target object.

[0051] In this embodiment of the invention, after calculating the rotation angle (tracking angle) of the angle adjustment device along the horizontal axis, the angle adjustment device is controlled to rotate along the horizontal axis to the tracking angle to achieve target tracking, even if the target object is at the visual center of the secondary camera. The overall process is as follows: Figure 4 As shown.

[0052] Since the target tracking in this embodiment of the invention only requires AI detection on the main camera, this embodiment of the invention can reduce the computing power requirements of the terminal device, which is more conducive to reducing product costs.

[0053] Furthermore, before performing target tracking, the dual-camera system needs to be calibrated in this embodiment of the invention. This calibration process specifically includes calibration of the angle adjustment device in the vertical axis direction and calibration in the horizontal axis direction. During the horizontal axis calibration process, the control equation for the calibration angle of the angle adjustment device in the horizontal axis direction can be fitted. Here, the vertical axis direction is perpendicular to the ground, i.e., the y-axis direction; the horizontal axis direction is parallel to the ground, i.e., the x-axis direction.

[0054] In the specific calibration process, embodiments of the present invention employ a calibration board, which consists of a checkerboard pattern embedded with identification codes. For example... Figure 5 As shown, the calibration board is constructed using a black and white checkerboard pattern. Each white square contains an embedded identifier, each identifier being a unique identifier (markerID). This identifier can be one or more of the following: ChArUco code, ArUco code, QR code, or other identifiers capable of distinguishing the unique identifier (markerID). Specifically, the design process may involve... Figure 6As shown, the size of each square can be customized according to the actual calibration distance, and can also be cropped according to different accuracy requirements. It should be noted that for horizontal calibration, several markers with different IDs are evenly arranged horizontally to cover the movement range of the secondary camera within the main camera's field of view. For a specific calibration distance, provided that both the main and secondary cameras can recognize the markers, the more markers and the smaller their size, the better. More markers are more helpful in improving calibration accuracy. However, at excessive distances, if the image resolution is weak, the detection algorithm may fail to detect small identification codes, leading to calibration failure.

[0055] The embodiments of the present invention increase the number of identifiable key points by using a calibration board, thereby improving the calibration accuracy of dual cameras compared to existing calibration methods.

[0056] It should be noted that, in setting the calibration board size and calibration distance in this embodiment of the invention, the main consideration is to ensure that the size of the calibration board can cover the movement range of the secondary camera in the main frame, and that the markerID in the calibration board can be recognized by the main camera. Under this premise, the size of the squares in the calibration board should be as small as possible, so that the calibration board contains as many markers as possible, thereby improving calibration accuracy.

[0057] Furthermore, such as Figure 7 As shown, the camera is installed at the designated height, maintaining a certain distance from the calibration board. With sufficient camera resolution and stable checkerboard detection, the primary camera should detect as many marker IDs as possible within its field of view, but not all of them are required. Additionally, for calibration along the vertical axis from the secondary camera's field of view, the markers used for calibration should not be too close to the edges. After setting up the environment, the primary and secondary cameras can capture images respectively.

[0058] For details on the angle calibration method along the vertical axis, please refer to Example 2.

[0059] The angle calibration method along the horizontal axis is based on computer vision object detection and tracking functions. For example... Figure 3 As shown, assuming a human actor is detected in the image captured by the main camera, with center coordinates (x0, y0), and the same human being in the image captured by the secondary camera has center coordinates (x, y) when the angle adjustment device rotates by an angle α along the horizontal axis, and center coordinates (x', y') when the rotation angle is α', the embodiment of the present invention expects that when the rotation angle is α', the human being's position in the image captured by the secondary camera is displayed in the center, i.e., x' = 0.5, y' = 0.5.

[0060] The calibration along the vertical axis involves searching for a fixed vertical adjustment angle. Unlike the vertical axis calibration, the horizontal axis calibration involves fitting a control equation to the data. The input to this control equation is the horizontal coordinate of the target's center in the image captured by the main camera, and the output is the adjustment angle of the angle adjustment device along the horizontal axis. The mathematical expression of this problem is as follows:

[0061] 1. Establish normalized coordinate systems for the main camera image and the secondary camera image;

[0062] 2. The equation of the main image is G(x,y), 0≤x≤1, 0≤y≤1;

[0063] 3. The equation of the secondary image is g(x,y), 0≤x≤1, 0≤y≤1;

[0064] 4. Let the control equation of the angle adjustment device be γ=R(x)=ω0·x 2 +ω1·x+b

[0065] 5. Suppose there exists a mapping such that g(x,y) is mapped to a new image space along with the x-axis angle of the angle adjustment device, where γ represents the rotation angle of the angle adjustment device in the x-axis direction;

[0066] Let the function be of the form g'(x,y)=K(γ)·g(x,y)

[0067] Therefore, the calibration along the horizontal axis (x-axis) can be transformed into a constrained optimization problem, in the following form:

[0068] minf(x) = |0.5 - x|

[0069] stg'(x,y)=K(γ)·g(x,y),0≤x≤1,0≤y≤1

[0070] γ=R(x)=ω0·X 2 +ω1·X+b,0≤X<1

[0071] γ min ≤γ≤γ max

[0072] Where X is the x-coordinate of the center of the marker in the main image, and x is the x-coordinate of the center of the same marker in the secondary image. γ min ,γ max The range of rotation of the angle adjustment device in the horizontal (x-axis) direction is a set of fixed values.

[0073] To address the aforementioned constrained optimization problem, and to solve for the coefficients ω0, ω1, and b of the control equation, this invention provides a specific method for solving the coefficients, including: acquiring a main camera image captured by the main camera against a calibration board, wherein the calibration board is composed of a checkerboard pattern embedded with an identification code, the identification code being used to identify a unique identifier markerID; identifying multiple identifiers in the main camera image; based on the multiple identifiers and a secondary camera image captured during the horizontal adjustment of the angle adjustment device, determining the calibration angle control equation of the angle adjustment device in the horizontal axis direction; and calibrating the angle adjustment device according to the calibration angle control equation in the horizontal axis direction.

[0074] Furthermore, determining the calibration angle control equation of the angle adjustment device in the horizontal direction based on the plurality of identifiers and the secondary camera image captured during the horizontal adjustment of the angle adjustment device includes: selecting a plurality of sampling identifiers from the plurality of identifiers in the main camera image; gradually adjusting the angle of the secondary camera's angle adjustment device in the horizontal direction, and acquiring the secondary camera image captured by the secondary camera after each horizontal angle adjustment; and determining the calibration angle control equation of the angle adjustment device in the horizontal direction based on the plurality of sampling identifiers and the secondary camera image.

[0075] Furthermore, the step of selecting multiple sampling identifiers from multiple identifiers in the main camera image includes: deleting the leftmost and rightmost identifiers from the multiple identifiers to obtain the multiple sampling identifiers.

[0076] Further, determining the calibration angle control equation of the angle adjustment device in the horizontal direction based on the plurality of sampling markers and the sub-camera image includes: selecting a starting marker from the plurality of sampling markers; determining the target adjustment angle of the angle adjustment device in the horizontal direction based on the starting marker and the sub-camera image; repeating the process of determining the target adjustment angle in the horizontal direction for the next sampling marker corresponding to the starting marker until all sampling markers have been traversed, outputting the plurality of target adjustment angles and the center horizontal coordinates of the sampling markers corresponding to the plurality of target adjustment angles in the main camera image; and fitting the calibration angle control equation of the angle adjustment device in the horizontal direction based on the plurality of target adjustment angles and their corresponding center horizontal coordinates.

[0077] Further, determining the target adjustment angle of the angle adjustment device in the horizontal direction based on the starting identifier and the secondary camera image includes: identifying the starting identifier in the secondary camera image after each horizontal angle adjustment, and calculating the horizontal offset between the center horizontal coordinate of the identifier code corresponding to the starting identifier in the secondary camera image and the visual center of the secondary camera; outputting multiple horizontal offsets until the angle adjustment device has completed its journey in the horizontal direction; selecting the minimum horizontal offset from the multiple horizontal offsets, and determining the adjustment angle corresponding to the minimum horizontal offset as the target adjustment angle for the starting identifier.

[0078] like Figure 8 As shown, the leftmost and rightmost markers are removed from multiple markers in the main camera image to obtain multiple sampling markers, and the leftmost marker among these is determined as the starting marker. Then, the angle adjustment device of the secondary camera is gradually adjusted from left to right along the horizontal axis (x-axis), and secondary camera images are captured. The starting marker M is identified in the secondary camera images. k Next, calculate the starting identifier M. k The horizontal offset between the center x-coordinate of the corresponding identifier code in the secondary camera image and the visual center of the secondary camera is calculated. When the angle adjustment device completes its journey along the horizontal axis (x-axis), multiple horizontal offsets are output. Simultaneously, the smallest horizontal offset is selected from these, and the adjustment angle corresponding to the smallest horizontal offset is determined as the target adjustment angle. Further, this process continues with the initial identifier M... k For the next sampling identifier, repeat the above process of determining the target adjustment angle until all sampling identifiers have been traversed. Finally, output multiple target adjustment angles and their corresponding sampling identifiers in the central horizontal coordinate X of the main camera image. Then, based on the multiple target adjustment angles and their corresponding central horizontal coordinate X, fit and calibrate the coefficients ω0, ω1 and b of the angle control equation.

[0079] Furthermore, the step of fitting the calibration angle control equation of the angle adjustment device in the horizontal axis direction based on the multiple target adjustment angles and their respective corresponding center abscissas includes: fitting the calibration angle control equation of the angle adjustment device in the horizontal axis direction using the least squares method based on the multiple target adjustment angles and their respective corresponding center abscissas.

[0080] Specifically, when fitting the coefficients ω0, ω1, and b of the control equation, the least squares method can be used to fit the calibration angle control equation of the angle adjustment device in the horizontal axis direction based on multiple target adjustment angles and their corresponding center abscissas, thereby achieving the calibration of the angle adjustment device in the horizontal axis direction. It should be noted that the calibration angle control equation in this embodiment is not limited to a quadratic polynomial function; other types of nonlinear functions can also be used. This embodiment does not impose specific limitations here.

[0081] Furthermore, when fitting the governing equations, the markers can be searched and compared slowly and sequentially along one direction, such as from left to right or from right to left, in the manner described above. However, the problem with this method is that the search is too slow, the calibration is time-consuming, and the efficiency is low. Based on this, embodiments of the present invention provide another coarse-to-fine search method, specifically including: gradually adjusting the angle of the sub-camera's angle adjustment device in the horizontal axis direction with a first preset step size, and recording the initial target adjustment angle corresponding to when the plurality of marker points are closest to the visual center of the sub-camera, and the center horizontal coordinates of the plurality of marker points in the main camera image; fitting an initial calibration angle control equation of the angle adjustment device in the horizontal axis direction based on the plurality of initial target adjustment angles and the plurality of center horizontal coordinates; calculating the angle control amount corresponding to the plurality of markers based on the initial calibration angle control equation; controlling the angle adjustment device based on the angle control amount to make the plurality of markers closer to the visual center of the sub-camera; gradually adjusting the angle of the sub-camera's angle adjustment device in the horizontal axis direction with a second preset step size, and recording the target adjustment angle corresponding to when the plurality of marker points are completely centered in the sub-camera; fitting the calibration angle control equation of the angle adjustment device in the horizontal axis direction based on the plurality of target adjustment angles and the plurality of center horizontal coordinates.

[0082] Wherein, the second preset step size is smaller than the first preset step size. The size of the first preset step size and the second preset step size can be set according to actual business needs, and the embodiments of the present invention do not impose specific limitations on this.

[0083] Specifically, the calibration board is first scanned along one direction with a relatively large step size (first preset step size), and the initial target adjustment angles corresponding to multiple markers when they are closest to the visual center of the secondary camera, as well as the center abscissas of multiple marker points in the main camera image, are recorded. Then, based on the multiple initial target adjustment angles and their corresponding center abscissas, a preliminary fitting is performed to obtain the initial calibration angle control equation f1 of the angle adjustment device in the horizontal axis direction. Next, based on the initial calibration angle control equation f1 and the center abscissas of multiple markers, the angle control values ​​corresponding to multiple markers are calculated respectively, and the angle adjustment device is adjusted based on the angle control values ​​to make each marker quickly approach the visual center of the secondary camera. Further, the angle of the angle adjustment device in the horizontal axis direction is finely adjusted with a smaller step size (second preset step size) to make each marker completely centered, and the target adjustment angles corresponding to multiple markers when they are completely centered are recorded, i.e., high-precision data. Finally, data fitting is performed using high-precision data to obtain the high-precision calibration angle control equation f2.

[0084] Therefore, it can be seen that the embodiments of the present invention can shorten the calibration time and improve the calibration efficiency by adopting a coarse-to-fine fitting strategy.

[0085] Furthermore, in the current scenario, only the ability of the secondary camera to rotate horizontally is considered. Therefore, the calibration board is set to be arranged horizontally only. The final fitted calibration angle control equation is essentially a curve equation γ = f(x), where x is the center horizontal coordinate of the marker in the main camera, and γ is the target adjustment angle of the secondary camera along the horizontal axis. However, in other scenarios, there may be situations where the secondary camera can swing in both the horizontal and vertical axes. In such cases, within the field of view that the secondary camera can sweep, an entire surface of markers can be arranged as a calibration board (multiple rows and columns). Similarly, for each marker in the main camera image, the angle adjustment device of the secondary camera is controlled to rotate horizontally and vertically respectively to center the corresponding marker in the secondary camera. At the same time, the horizontal target adjustment angle and the vertical target adjustment angle corresponding to each marker are recorded, and the surface equation (β,γ) = f(x,y), β = f1(x,y), and γ = f2(x,y) are fitted.

[0086] Based on this, the method includes: gradually adjusting the angle adjustment device of the secondary camera in the horizontal and vertical directions, and recording the horizontal target adjustment angle and vertical target adjustment angle corresponding to when the plurality of marker points are completely centered in the secondary camera, as well as the center horizontal coordinate and center vertical coordinate of the plurality of marker points in the main camera image; and fitting the calibration angle control equation of the angle adjustment device in the horizontal and vertical directions based on the plurality of horizontal target adjustment angles and the plurality of vertical target adjustment angles, as well as the plurality of center horizontal coordinates and the plurality of center vertical coordinates.

[0087] Specifically, for each marker in the main camera image, the angle adjustment device of the secondary camera is controlled to rotate along the horizontal and vertical axes to center the corresponding marker within the secondary camera. Simultaneously, the horizontal and vertical target adjustment angles corresponding to each marker at the center position, as well as the center horizontal and vertical coordinates of each marker, are recorded. Finally, based on the horizontal and vertical target adjustment angles, center horizontal and vertical coordinates of all markers, surface fitting is performed to obtain the calibration angle control equation (β,γ)=f(x,y).

[0088] Therefore, it can be seen that by fitting the surface equation, the embodiments of the present invention can not only control the rotation of the secondary camera along the horizontal axis, but also control the rotation of the secondary camera along the vertical axis.

[0089] Furthermore, in this embodiment of the invention, the checkerboard pattern used for horizontal axis calibration is divided into upper and lower parts, one part for data fitting and the other part for testing and error evaluation, such as... Figure 9 As shown, using the ChArUcomarker, the first row of identifiers on the calibration board is used for fitting the control equations, and the second row of identifiers is used for evaluating and testing the error of the control equations after calibration. It should be noted that the second row of identifiers can also be used for fitting the control equations, and the first row of identifiers can be used for evaluation and testing; this embodiment of the invention does not specifically limit this. Furthermore, it can also be done as follows... Figure 10 As shown, a whole row of ArUcomarkers is used, and it is divided into two groups according to odd and even positions, such as group A and group B. One group is used for data fitting, and the other group is used for error verification.

[0090] Specifically, when evaluating the error of the calibration angle control equation, the adjustment angle of the angle adjustment device along the horizontal axis can be obtained according to the calibration angle control equation. The angle adjustment device is then adjusted based on this adjustment angle. After that, the horizontal offset between the center horizontal coordinate of the identification code corresponding to the identifier in the main camera image and the visual center of the secondary camera is calculated. If the horizontal offset is less than a preset threshold, it is determined that the error of the calibration angle control equation meets the requirements.

[0091] The embodiments of this invention, based on a fitted calibration angle control equation, ensure that the target seen on the secondary camera is sufficiently close to the visual center of the secondary camera, thereby guaranteeing that the target is centered as seen on the secondary camera. Furthermore, compared to existing technologies, the embodiments of this invention, based on data fitting using multiple identifiers identified within the coverage area of ​​the secondary camera, achieve more refined calibration.

[0092] Furthermore, before identifying the identifiers in the main camera image, it is necessary to verify the validity of the images captured by the main camera and the secondary camera. As an optional implementation, the method includes: acquiring multiple frames of main camera verification images continuously captured by the main camera against the calibration board; detecting whether the number of identifiers in each frame of the main camera verification image meets a preset requirement; if the number of identifiers in each frame of the main camera verification image meets the preset requirement, then determining that the image captured by the main camera is valid; acquiring secondary camera verification images captured by the secondary camera against the calibration board; determining whether the identifiers in each frame of the main camera verification image can be detected in the secondary camera verification images; if they can be detected, then determining that the image captured by the secondary camera is valid.

[0093] Specifically, if the number of identifiers detected in multiple frames of the main camera verification images all meet the preset requirements, such as satisfying the minimum requirements for data fitting, then the image captured by the main camera is determined to be valid. Similarly, it is checked whether all the identifiers in the main camera verification images can be detected in the secondary camera verification images. If they can be detected, then the image captured by the secondary camera is determined to be valid. The difference is that the secondary camera needs to first adjust its angle by controlling the angle adjustment device to observe whether the identifier is in the visual center of the secondary camera, but it is not required to be perfectly centered. If it is in the visual center, then it is checked whether all the identifiers in the main camera verification images can be detected in the secondary camera verification images.

[0094] Furthermore, after confirming the validity of the images captured by the main camera and the secondary camera, multiple markers in the main camera image are identified. Then, to ensure the calibration accuracy of the dual cameras, the number of markers in the main camera image needs to be detected, and it needs to be determined whether the number of markers reaches a preset number. If the number of markers reaches the preset number, based on the multiple markers and the secondary camera image captured during the adjustment of the secondary camera's angle adjustment device, the calibration angle control equations for the angle adjustment device in the vertical axis direction and the horizontal axis direction are determined sequentially. The preset number of markers can be set according to actual business needs.

[0095] Once the calibration board and dual cameras are set up, the present invention can trigger automatic calibration and complete the verification with a single click, achieving a high degree of automation, saving manpower, and improving calibration efficiency.

[0096] Compared to existing technologies, this invention, based on a multi-point fitting calibration angle control equation, improves the calibration accuracy of dual cameras. Therefore, during target tracking, AI detection only needs to be performed on the main camera. Specifically, the rotation angle of the angle adjustment device along the horizontal axis can be determined based on the target object's detection bounding box information obtained from AI detection and the calibration angle control equation, thereby achieving target tracking. Furthermore, since this invention only requires AI detection on the main camera, it reduces the computational requirements of the terminal device, thus lowering equipment costs.

[0097] Furthermore, as a refinement and extension of the specific implementation methods of the above embodiments, and to fully illustrate the implementation methods of this embodiment, this embodiment also provides another dual-camera target tracking method, such as... Figure 2 As shown, the method includes:

[0098] Step 201: Obtain the image to be detected captured by the main camera, and the calibration angle control equation of the angle adjustment device of the secondary camera in the horizontal axis direction.

[0099] The calibration angle control equation is obtained by fitting multiple identification data.

[0100] In this embodiment of the invention, the process of obtaining the image to be detected and the calibration angle control equation is exactly the same as step 101, and will not be repeated here.

[0101] Step 202: Perform target detection on the image to be detected to obtain the detection box information corresponding to the target object.

[0102] In this embodiment of the invention, a preset target detection algorithm is used to perform target detection on the image to be detected, and the detection box information of the target object in the image to be detected is obtained. The detection box information specifically includes the horizontal coordinate information x and the vertical coordinate information y of the center point of the detection box, as well as the width information w and the length information h of the detection box.

[0103] Step 203: Calculate the center coordinates of the target object based on the detection box information.

[0104] In an embodiment of the present invention, as an optional implementation, the method for calculating the center coordinates of a target object includes: determining the coordinate information of the center point of the detection box based on the detection box information; and calculating the center coordinates of the target object based on the coordinate information of the center point of the detection box and the size information of the image grid, wherein the center coordinates of the target object include a center x-coordinate and a center y-coordinate. The specific formula is as follows:

[0105] x = (x0 - grid) / grid

[0106] y = (y0 - grid) / grid

[0107] Where x and y are the coordinates of the center point of the detection box, x0 and y0 are the x-coordinate and y-coordinate of the center of the target object, respectively, and grid is the size information of the image grid.

[0108] Since the coordinates of the center point of the detection box and the size of the image grid are known, the center coordinates (x0, y0) of the target object can be calculated according to the above formula.

[0109] Step 204: Substitute the center horizontal coordinate of the center coordinates corresponding to the target object into the calibration angle control equation to obtain the tracking angle of the angle adjustment device in the horizontal axis direction.

[0110] In this embodiment of the invention, the abscissa x0 of the center coordinates (x0, y0) of the target object is substituted into the control equation γ = ω0·X 2 +ω1·X+b gives the tracking angle γ of the secondary camera in the horizontal direction.

[0111] Step 205: Based on the tracking angle in the horizontal axis direction, control the angle adjustment device to rotate along the horizontal axis direction to achieve tracking of the target object.

[0112] Furthermore, in this embodiment of the invention, before target tracking, the angle adjustment device needs to be calibrated. Specifically, the angle adjustment device is calibrated first in the vertical axis direction and then in the horizontal axis direction.

[0113] Regarding angle calibration along the vertical axis, when the angle adjustment device and the dual-camera module are assembled into a complete unit, not every device's image will be parallel to the main camera's image along the vertical axis. For example... Figure 11 As shown, the angle adjustment device needs to be calibrated separately in the longitudinal direction so that it is parallel to the main camera when the angle adjustment device is powered on and initialized.

[0114] The basic idea behind calibration along the vertical axis is to search for an optimal calibration angle γ along the vertical axis. o At this calibration angle, the tilt of the chessboard seen by the main camera should be as consistent as possible with the tilt of the chessboard seen by the secondary camera. The mathematical expression of this problem is as follows:

[0115] 1. Establish the camera image coordinate system, and let the image equation of the main camera be G(x,y);

[0116] 2. Let the equation of the imaging plane of the secondary camera, controlled by the y-axis angle of the angle adjustment device, be g(x,y);

[0117] 3. Suppose there exists a mapping such that g(x,y) is mapped to a new image space along with the y-axis angle of the angle adjustment device, where γ represents the rotation angle of the angle adjustment device in the y-axis direction;

[0118] Let the function be of the form R(x,y)=K(γ)·g(x,y)

[0119] 3. k1 represents the center coordinates (X) of the identification codes corresponding to the two reference markers in the upper half of the checkerboard pattern of the main camera. m ,Y m ), (X n ,Y n The slope of the straight line connecting the two reference markers in the upper half of the chessboard grid of the secondary camera; k2 represents the center coordinates (x, y) of the identification codes corresponding to the two reference markers in the upper half of the chessboard grid. m ,y m ), (x n ,y n The slope of the straight line connecting the two axes (y-axis and y-axis). Therefore, the calibration along the vertical axis (y-axis) can be transformed into a constrained optimization problem, specifically as follows:

[0120] minf(x,y)=|k1-k2|

[0121] stR(x,y)=K(γ)·g(x,y),γ∈|γ min ,γ max |

[0122] Where, γ min ,γ max The range of rotation of the angle adjustment device in the longitudinal (y-axis) direction is a set of fixed values.

[0123] To address the aforementioned constrained optimization problem and determine the calibration angle of the angle adjustment device along the vertical axis, this invention provides a specific method for searching the optimal solution, including: selecting a target identifier whose identifier value is close to the median from multiple identifiers in the main camera image; gradually adjusting the angle of the secondary camera's angle adjustment device along the vertical axis, and acquiring a secondary camera image after each vertical angle adjustment; determining the calibration angle of the angle adjustment device along the vertical axis based on the target identifier and the secondary camera image; and calibrating the angle adjustment device according to the calibration angle along the vertical axis.

[0124] Further, determining the calibration angle of the angle adjustment device in the vertical direction based on the target identifier and the secondary camera image includes: identifying the target identifier in the secondary camera image after each vertical angle adjustment, and determining whether the target identifier is centered horizontally in the secondary camera image based on the center coordinates of the identification code corresponding to the target identifier in the secondary camera image; if the target identifier is centered horizontally in the secondary camera image, determining the left and right identifiers corresponding to the target identifier from multiple identifiers in the main camera image; and determining the calibration angle of the angle adjustment device in the vertical direction based on the left identifier, the right identifier, and the secondary camera image.

[0125] Further, determining the calibration angle of the angle adjustment device in the vertical direction based on the left-side identifier, the right-side identifier, and the sub-camera image includes: calculating a first slope of the line connecting the center of the identifier codes of the left-side identifier and the right-side identifier in the main camera image based on the center coordinates of the identifier codes corresponding to the left-side identifier and the right-side identifier in the main camera image; identifying the left-side identifier and the right-side identifier in the sub-camera image, and calculating a second slope of the line connecting the center of the identifier codes of the left-side identifier and the right-side identifier in the sub-camera image based on the center coordinates of the identifier codes corresponding to the left-side identifier and the right-side identifier in the sub-camera image; calculating the slope error between the first slope and the second slope until the angle adjustment device has traveled the entire distance in the vertical direction, obtaining multiple slope errors corresponding to the adjustment angles in the vertical direction; selecting the minimum slope error from the multiple slope errors, and determining the adjustment angle corresponding to the minimum slope error as the calibration angle in the vertical direction.

[0126] like Figure 12 As shown, target identifier M with an identifier value close to the median is selected from multiple identifiers in the main camera image. I Then, the angle adjustment device of the secondary camera is gradually adjusted in the vertical direction, and the secondary camera image is captured. Next, the target marker M in the secondary camera image is detected. I And according to the target identifier M I The center coordinates of the corresponding identification code in the secondary camera image are used to determine the target identifier M. I Whether it is centered horizontally in the secondary camera image; if centered horizontally, then determine the target identifier M from multiple identifiers in the primary camera image. I Adjacent left-hand sign M O and the M on the right l Then, according to the label M on the left O and the M on the right l Calculate the center coordinates of the corresponding identifier codes in the main image, and then calculate the left identifier M in the main image. OThe identification code and the right-side identifier M l The first slope k1 of the center line connecting the identifiers. Further, the left identifier M is identified in the secondary image. O and the M on the right l And according to the left-hand label M O and the M on the right l Calculate the center coordinates of the corresponding identifier codes in the secondary image, and then calculate the left identifier M in the secondary image. O The identification code and the right-side identifier M l The second slope k2 of the center line connecting the identification codes is calculated. Finally, the slope error between the first slope k1 and the second slope k2 is calculated until the angle adjustment device has traveled the entire distance in the vertical axis direction. The slope errors corresponding to multiple adjustment angles are obtained, and the adjustment angle corresponding to the smallest slope error among the multiple slope errors is determined as the calibration angle in the vertical axis direction.

[0127] It should be noted that when determining the center coordinates of the identification code in the main or secondary image, the center coordinates can be calculated based on the coordinates of the four corner points of the square where the identification code is located, or the coordinates of the corner points can be directly selected during calibration. This embodiment of the invention does not impose specific limitations on this.

[0128] Similarly, the angle calibration along the vertical axis also requires verification of the image's validity and whether the number of detection markers meets the standard. The specific process is exactly the same as that described in Example 1, and will not be repeated here.

[0129] Furthermore, after determining the calibration angle of the angle adjustment device in the vertical direction, it is necessary to verify whether the calibration error meets the standard. Specifically, two markers, left and right, can be selected, and then the slope error between these two markers in the main camera image and their slope error in the secondary camera image can be determined. If the slope error is less than a preset threshold, it indicates that the calibration error meets the standard. The image comparison results before and after calibration are as follows: Figure 13 As shown. The calibration method of this invention can make the tilt of the checkerboard pattern seen in the main camera image as consistent as possible with the tilt of the checkerboard pattern seen in the secondary camera image.

[0130] The angle calibration process in the horizontal direction is exactly the same as that in Example 1, and will not be repeated here.

[0131] It should be noted that, in this embodiment of the invention, the same checkerboard pattern is used for calibration in both the vertical and horizontal directions. However, unlike the vertical axis calibration, the horizontal axis calibration divides the calibration board into upper and lower parts, one part for data fitting and the other for testing and error assessment. The calibration method provided by this embodiment is simple to operate and has good robustness.

[0132] Compared to existing technologies, this invention, based on a multi-point fitting calibration angle control equation, improves the calibration accuracy of dual cameras. Therefore, during target tracking, AI detection only needs to be performed on the main camera. Specifically, the rotation angle of the angle adjustment device along the horizontal axis can be determined based on the target object's detection bounding box information obtained from AI detection and the calibration angle control equation, thereby achieving target tracking. Furthermore, since this invention only requires AI detection on the main camera, it reduces the computational requirements of the terminal device, thus lowering equipment costs.

[0133] Furthermore, as Figure 1 and Figure 2 The specific implementation of the method shown in this embodiment provides a dual-camera target tracking device, such as... Figure 14 As shown, the device includes: an acquisition unit 31, a detection unit 32, a determination unit 33, and a control unit 34.

[0134] The acquisition unit 31 can be used to acquire the image to be detected captured by the main camera, and the calibration angle control equation of the angle adjustment device of the secondary camera in the horizontal direction, wherein the calibration angle control equation is obtained by fitting multiple identification data.

[0135] The detection unit 32 can be used to perform target detection on the image to be detected and obtain the detection box information corresponding to the target object.

[0136] The determining unit 33 can be used to determine the tracking angle of the angle adjustment device in the horizontal axis direction based on the detection frame information and the calibration angle control equation.

[0137] The control unit 34 can be used to control the angle adjustment device to rotate along the horizontal axis based on the tracking angle in the horizontal axis direction, so as to achieve tracking of the target object.

[0138] In specific application scenarios, the determining unit 33, such as Figure 15 As shown, it includes a first calculation module 331 and a second calculation module 332.

[0139] The first calculation module 331 can be used to calculate the center coordinates of the target object based on the detection box information.

[0140] The second calculation module 332 can be used to substitute the center horizontal coordinate of the center coordinate corresponding to the target object into the calibration angle control equation to obtain the tracking angle of the angle adjustment device in the horizontal axis direction.

[0141] Furthermore, the first calculation module 331 can be specifically used to determine the coordinate information of the center point of the detection box based on the detection box information; and to calculate the center coordinates of the target object based on the coordinate information of the center point of the detection box and the size information of the image grid, wherein the center coordinates of the target object include the center horizontal coordinate and the center vertical coordinate.

[0142] In specific application scenarios, the device further includes: an identification unit 35 and a calibration unit 36.

[0143] The acquisition unit 31 can also be used to acquire the main camera image captured by the main camera against the calibration board, wherein the calibration board is composed of a checkerboard pattern with an embedded identification code, and the identification code is used to identify a unique identifier.

[0144] The identification unit 35 can be used to identify multiple identifiers in the main camera image.

[0145] The determining unit 33 can also be used to determine the calibration angle control equation of the angle adjustment device in the horizontal axis direction based on the multiple identifiers and the secondary camera image captured during the horizontal adjustment of the angle adjustment device.

[0146] The calibration unit 36 ​​can be used to calibrate the angle adjustment device according to the calibration angle control equation in the horizontal axis direction.

[0147] Furthermore, the determining unit 33 also includes: a selection module 333, an acquisition module 334, and a determining module 335.

[0148] The selection module 333 can be used to select multiple sampling identifiers from multiple identifiers in the main camera image;

[0149] The acquisition module 334 can be used to gradually adjust the angle of the sub-camera angle adjustment device in the horizontal axis direction, and acquire the sub-camera image captured by the sub-camera after each horizontal angle adjustment;

[0150] The determining module 335 can be used to determine the calibration angle control equation of the angle adjustment device in the horizontal direction based on the multiple sampling identifiers and the sub-camera image.

[0151] Furthermore, the selection module 333 can be specifically used to delete the leftmost and rightmost identifiers among the plurality of identifiers to obtain the plurality of sampling identifiers.

[0152] Furthermore, the determining module 335 includes: a selection submodule, a determining submodule, and a fitting submodule.

[0153] The selection submodule can be used to select a starting identifier from the plurality of sampling identifiers.

[0154] The determining submodule can be used to determine the target adjustment angle of the angle adjustment device in the horizontal axis direction based on the starting identifier and the sub-camera image.

[0155] The determination submodule can also be used to repeat the process of determining the target adjustment angle in the horizontal direction for the next sampling marker corresponding to the starting marker, until all sampling markers are traversed, and output multiple target adjustment angles, as well as the center horizontal coordinates of the sampling markers corresponding to the multiple target adjustment angles in the main camera image.

[0156] The fitting submodule can be used to fit the calibration angle control equation of the angle adjustment device in the horizontal axis direction based on multiple target adjustment angles and their corresponding center abscissas.

[0157] Furthermore, the determining submodule can be specifically used to identify the starting marker in the secondary camera image after each lateral angle adjustment, and calculate the lateral offset between the center horizontal coordinate of the identifier code corresponding to the starting marker in the secondary camera image and the visual center of the secondary camera; until the angle adjustment device has completed the entire journey in the horizontal axis direction, output multiple lateral offsets; filter out the minimum lateral offset from the multiple lateral offsets, and determine the adjustment angle corresponding to the minimum lateral offset as the target adjustment angle for the starting marker.

[0158] Furthermore, the fitting submodule can be specifically used to fit the calibration angle control equation of the angle adjustment device in the horizontal axis direction using the least squares method based on multiple target adjustment angles and their corresponding center abscissas.

[0159] In specific application scenarios, the device further includes: a determination unit 37.

[0160] The determination unit 37 can be used to determine whether the number of identifiers has reached the preset number of identifiers.

[0161] The determining unit 33 can be specifically used to determine the calibration angle control equation of the angle adjustment device in the horizontal axis direction based on the multiple icons and the secondary camera image captured during the horizontal adjustment of the angle adjustment device if the number of the icons reaches the preset number of icons.

[0162] In specific application scenarios, the device further includes a verification unit 38.

[0163] The verification unit 38 can be used to acquire multiple frames of main camera verification images continuously captured by the main camera against the calibration board; detect whether the number of identifiers in each frame of the main camera verification image meets a preset requirement; if the number of identifiers in each frame of the main camera verification image meets the preset requirement, then the image captured by the main camera is determined to be valid; acquire secondary camera verification images captured by the secondary camera against the calibration board; determine whether the identifiers in each frame of the main camera verification image can be detected in the secondary camera verification images; if they can be detected, then the image captured by the secondary camera is determined to be valid.

[0164] It should be noted that other corresponding descriptions of the functional units involved in the dual-camera calibration device provided in this embodiment can be found in [reference needed]. Figure 1 and Figure 2 The corresponding description in [the document] will not be repeated here.

[0165] Based on the above, Figure 1 and Figure 2 Accordingly, this embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the above-described method. Figure 1 and Figure 2 The dual-camera target tracking method shown.

[0166] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0167] Based on the above, Figure 1 and Figure 2 The method shown, and Figure 14 and Figure 15 To achieve the above objectives, the present application also provides an electronic device, specifically a personal computer, tablet computer, server, or other network device, as shown in the virtual device embodiment. This device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to achieve the above-described objectives. Figure 1 and Figure 2 The dual-camera target tracking method shown.

[0168] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0169] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0170] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0171] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented using software plus necessary general-purpose hardware platforms, or it can be implemented in hardware. By applying the technical solution of this embodiment, compared with the prior art, since the embodiment of the present invention is based on the calibration angle control equation of multi-point fitting, it can improve the calibration accuracy of dual cameras. Therefore, when performing target tracking, only AI detection needs to be implemented on the main camera. Specifically, the rotation angle of the angle adjustment device along the horizontal axis can be determined according to the detection box information of the target object obtained by AI detection and the calibration angle control equation, thereby achieving target tracking. In contrast, since the embodiment of the present invention only needs to perform AI detection on the main camera, it can reduce the computing power requirements of the terminal device, thereby reducing equipment costs.

[0172] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be modified to be located in one or more apparatuses different from this embodiment. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0173] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method of target tracking with dual cameras, characterized by, Applied to the main camera, including: The image to be detected captured by the main camera and the calibration angle control equation of the angle adjustment device of the secondary camera in the horizontal direction are obtained, wherein the calibration angle control equation is obtained by fitting multiple identification data. Target detection is performed on the image to be detected to obtain the detection box information corresponding to the target object; Based on the detection frame information and the calibration angle control equation, the tracking angle of the angle adjustment device in the horizontal axis direction is determined; Based on the tracking angle in the horizontal axis direction, the angle adjustment device is controlled to rotate along the horizontal axis direction to achieve tracking of the target object; The method further includes, before acquiring the image to be detected captured by the main camera and the calibration angle control equation of the sub-camera's angle adjustment device in the horizontal axis direction: The main camera captures a main image of the calibration board, wherein the calibration board is composed of a checkerboard pattern with embedded identification codes, and the identification codes are used to identify a unique identifier. Identify multiple identifiers in the main camera image; Based on the multiple identifiers and the secondary camera images captured during the horizontal adjustment of the angle adjustment device, the calibration angle control equation of the angle adjustment device in the horizontal axis direction is determined to complete the calibration of the angle adjustment device in the horizontal axis direction.

2. The method according to claim 1, characterized in that, The step of determining the tracking angle of the angle adjustment device in the horizontal axis direction based on the detection frame information and the calibration angle control equation includes: Based on the detection box information, calculate the center coordinates corresponding to the target object; Substituting the center horizontal coordinate of the target object into the calibration angle control equation, the tracking angle of the angle adjustment device in the horizontal axis direction is obtained.

3. The method according to claim 2, characterized in that, The step of calculating the center coordinates of the target object based on the detection box information includes: Based on the detection frame information, determine the coordinate information of the center point of the detection frame; Based on the coordinate information of the center point of the detection box and the size information of the image grid, the center coordinates of the target object are calculated, wherein the center coordinates of the target object include the center x-coordinate and the center y-coordinate.

4. The method according to claim 1, characterized in that, The determination of the calibration angle control equation of the angle adjustment device in the horizontal axis direction based on the multiple identifiers and the secondary camera image captured during the horizontal adjustment of the angle adjustment device includes: Multiple sampling identifiers are selected from multiple identifiers in the main camera image; Gradually adjust the angle of the secondary camera's angle adjustment device in the horizontal axis direction, and acquire the secondary camera image captured by the secondary camera after each horizontal angle adjustment; Based on the multiple sampling identifiers and the secondary camera image, the calibration angle control equation of the angle adjustment device in the horizontal axis direction is determined.

5. The method according to claim 4, characterized in that, The step of selecting multiple sampling identifiers from multiple identifiers in the main camera image includes: Delete the leftmost and rightmost identifiers from the plurality of identifiers to obtain the plurality of sampling identifiers; The step of determining the calibration angle control equation of the angle adjustment device in the horizontal direction based on the multiple sampling identifiers and the sub-camera image includes: Select a starting identifier from the plurality of sampling identifiers; Based on the starting identifier and the secondary camera image, the target adjustment angle of the angle adjustment device in the horizontal axis direction is determined; For the next sampling marker corresponding to the starting marker, the process of determining the target adjustment angle in the horizontal direction is repeated until all sampling markers are traversed. Then, multiple target adjustment angles are output, as well as the center horizontal coordinates of the sampling markers corresponding to the multiple target adjustment angles in the main camera image. Based on the multiple target adjustment angles and their corresponding center abscissas, the calibration angle control equation of the angle adjustment device in the horizontal axis direction is fitted.

6. The method according to claim 5, characterized in that, Determining the target adjustment angle of the angle adjustment device in the horizontal axis direction based on the starting identifier and the secondary image includes: After each lateral angle adjustment, the starting marker in the secondary camera image is identified, and the lateral offset between the center horizontal coordinate of the identifier code corresponding to the starting marker in the secondary camera image and the visual center of the secondary camera is calculated. Until the angle adjustment device has traveled its entire length in the horizontal axis direction, it outputs multiple lateral offsets; The minimum lateral offset is selected from the plurality of lateral offsets, and the adjustment angle corresponding to the minimum lateral offset is determined as the target adjustment angle for the starting identifier.

7. The method according to claim 5, characterized in that, The step of fitting the calibration angle control equation of the angle adjustment device in the horizontal axis direction based on multiple target adjustment angles and their corresponding center abscissas includes: Based on the multiple target adjustment angles and their corresponding center abscissas, the least squares method is used to fit the calibration angle control equation of the angle adjustment device in the horizontal axis direction.

8. The method according to claim 1, characterized in that, The determination of the calibration angle control equation of the angle adjustment device in the horizontal axis direction based on the multiple identifiers and the secondary camera image captured during the horizontal adjustment of the angle adjustment device includes: The angle of the sub-camera's angle adjustment device in the horizontal direction is gradually adjusted by a first preset step size, and the initial target adjustment angle corresponding to when the plurality of marker points are closest to the visual center of the sub-camera is recorded, as well as the center horizontal coordinate of the plurality of marker points in the main camera image. Based on the multiple initial target adjustment angles and the multiple center abscissas, the initial calibration angle control equation of the angle adjustment device in the abscissa direction is fitted. Based on the initial calibration angle control equation, calculate the angle control quantities corresponding to the multiple identifiers respectively; Based on the angle control amount, the angle adjustment device is controlled so that the plurality of marks are close to the visual center of the secondary camera; The angle of the sub-camera's angle adjustment device in the horizontal direction is gradually adjusted by a second preset step size, and the target adjustment angles corresponding to when the multiple marker points are completely centered in the sub-camera are recorded respectively, wherein the second preset step size is smaller than the first preset step size; Based on the multiple target adjustment angles and the multiple center horizontal coordinates, the calibration angle control equation of the angle adjustment device in the horizontal axis direction is fitted.

9. The method according to claim 1, characterized in that, When the secondary camera can be tilted laterally and longitudinally respectively via the angle adjustment device, the method further includes: Gradually adjust the angle of the sub-camera's angle adjustment device in the horizontal and vertical directions, and record the horizontal and vertical target adjustment angles corresponding to when the multiple marker points are completely centered in the sub-camera, as well as the center horizontal and center vertical coordinates of the multiple marker points in the main camera image. Based on the multiple horizontal target adjustment angles and the multiple vertical target adjustment angles, as well as the multiple central horizontal coordinates and the multiple central vertical coordinates, the calibration angle control equations of the angle adjustment device in the horizontal and vertical directions are fitted.

10. The method according to claim 1, characterized in that, The method further includes: Determine whether the number of tags has reached the preset number of tags; If the number of the markers reaches the preset number of markers, then based on the multiple markers and the secondary camera image captured during the horizontal adjustment of the angle adjustment device, the calibration angle control equation of the angle adjustment device in the horizontal axis direction is determined.

11. The method according to claim 1, characterized in that, Before identifying multiple identifiers in the main camera image, the method further includes: Acquire multiple frames of main camera verification images continuously captured by the main camera against the calibration board; Each frame of the main camera verification image is checked to see if the number of markers in each frame meets the preset requirements; If the number of identifiers in each frame of the main camera verification image meets the preset requirements, then the image captured by the main camera is determined to be valid. Acquire the secondary camera verification image captured by the secondary camera in relation to the calibration board; Determine whether the identifier in each frame of the main camera verification image can be detected in the secondary camera verification image; If it can be detected, then the image captured by the secondary camera is determined to be valid.

12. A target tracking device with dual cameras, characterized in that, Applied to the main camera, including: The acquisition unit is used to acquire the image to be detected captured by the main camera, and the calibration angle control equation of the angle adjustment device of the secondary camera in the horizontal axis direction, wherein the calibration angle control equation is obtained by fitting multiple identification data. The detection unit is used to perform target detection on the image to be detected and obtain the detection box information corresponding to the target object; The determining unit is used to determine the tracking angle of the angle adjustment device in the horizontal axis direction based on the detection frame information and the calibration angle control equation. A control unit is used to control the angle adjustment device to rotate along the horizontal axis based on the tracking angle in the horizontal axis direction, so as to achieve tracking of the target object; The acquisition unit is further configured to acquire the main camera image captured by the main camera against the calibration board, wherein the calibration board is composed of a checkerboard pattern embedded with an identification code, and the identification code is used to identify a unique identifier; The identification unit is used to identify multiple identifiers in the main camera image; The determining unit is further configured to determine the calibration angle control equation of the angle adjustment device in the horizontal axis direction based on the plurality of identifiers and the secondary camera image captured during the horizontal adjustment of the angle adjustment device, so as to complete the calibration of the angle adjustment device in the horizontal axis direction.

13. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 11.

14. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 11.

15. A camera device, characterized in that, The camera equipment is equipped with a dual-camera target tracking device as described in claim 12.

16. A video system, characterized in that, The video system includes a display and a camera device, the display being communicatively connected to the camera device, wherein the camera device is equipped with a dual-camera target tracking device as described in claim 12; The camera device is used to acquire the image to be detected captured by the main camera and the calibration angle control equation of the angle adjustment device of the secondary camera in the horizontal direction, wherein the calibration angle control equation is obtained by fitting multiple identifier data; to perform target detection on the image to be detected to obtain detection box information corresponding to the target object; to determine the tracking angle of the angle adjustment device in the horizontal direction according to the detection box information and the calibration angle control equation; and to control the angle adjustment device to rotate along the horizontal direction based on the tracking angle in the horizontal direction to achieve tracking of the target object; before acquiring the image to be detected captured by the main camera and the calibration angle control equation of the angle adjustment device of the secondary camera in the horizontal direction, the device further includes: acquiring a main camera image captured by the main camera against a calibration board, wherein the calibration board is composed of a checkerboard pattern embedded with an identification code, the identification code being used to identify a unique identifier; identifying multiple identifiers in the main camera image; and determining the calibration angle control equation of the angle adjustment device in the horizontal direction based on the multiple identifiers and the secondary camera image captured during the horizontal adjustment of the angle adjustment device, to complete the calibration of the angle adjustment device in the horizontal direction.

Images