Object capturing method and apparatus based on radar-dome camera system, and electronic device

Abstract

Embodiments of the present application provide an object capturing method and apparatus based on a radar-dome camera system, and an electronic device. In the embodiments, by means of P, T and Z state parameter values when first position information of a first object is detected by a dome camera, and the mounting height h, pitch angle a1, roll angle a2 and horizontal angle a3 of the dome camera, the first position information (i.e., real-time direction / position) of the first object currently detected by the dome camera is converted from a dome camera picture coordinate system to a radar coordinate system in real time, so that radar detection can be rechecked, corrected and supplemented by dome camera detection, the problem of the dependence of a traditional radar-dome camera linkage system on radar detection precision is solved, the situation that the dome camera tracks a wrong object or no object is present in the field of view of the dome camera during object tracking due to the dome camera being linked to rotate depending solely on the radar detection in the traditional radar-dome camera linkage system is avoided, and the object tracking effect is improved.

Description

Target capture method, device and electronic equipment based on thunderball system Technical Field

[0001] The present application relates to the field of traffic monitoring, and in particular to a target capture method, device and electronic equipment based on a thunderball system. Background Art

[0002] Currently, in many scenarios, radar and dome cameras are often linked together based on radar ball calibration to capture targets. For example, at a ship checkpoint, radar (such as millimeter-wave radar) is used to detect a target (such as a ship), and then a dome camera is linked to the target through radar ball calibration to track the target. Once the target reaches a pre-set capture area, the dome camera's magnification is controlled based on the target's distance to capture a high-definition close-up image of the target.

[0003] However, current radar ball calibration generally refers to the calibration between the position of the radar and the rotation angle of the dome camera. Under this premise, when the dome camera linked to the radar tracks a target, it will rely entirely on the position accuracy of the target detected by the radar and the calibration accuracy of the radar ball calibration. Once the position accuracy and / or calibration accuracy deviate, there will be no target in the dome camera's field of view, or only part of the target, and it may even cause the dome camera to track the wrong target. Summary of the Invention

[0004] The embodiments of the present application provide a target capture method, device and electronic equipment based on the Thunderball system to improve the target tracking effect.

[0005] An embodiment of the present application provides a target capture method based on a radar system. The radar system includes a dome camera and a radar. The method includes: based on the P state parameter value, T state parameter value, and Z state parameter value when the dome camera detects first position information of the first target, as well as the dome camera's installation height h, pitch angle a1, roll angle a2, and horizontal angle a3, converting the first position information of the first target currently detected by the dome camera from the dome camera's screen coordinate system to the radar coordinate system of the radar to obtain second position information; if, based on the second position information and third position information of the second target detected by the radar, it is determined that the first target detected by the dome camera and the second target detected by the radar are the same target, adjusting the dome camera to place the first target at a specified position on the dome camera's screen; determining a rotation lead of the dome camera based on the movement speed of the first target detected by the dome camera while tracking the first target, and the current width and height of the first target relative to the dome camera's screen; controlling the rotation of the dome camera based on the rotation lead, and controlling the timing of capturing a specified portion of the first target when the specified portion enters the dome camera's screen.

[0006] The embodiment of the present application provides a target capture device based on a thunder ball system, the thunder ball system including a ball camera and a radar, the device including: a conversion module for converting the first position information of the first target currently detected by the ball camera from the ball camera screen coordinate system to the radar coordinate system of the radar based on the P state parameter value, T state parameter value, Z state parameter value when the ball camera detects the first position information of the first target, and the height h, pitch angle a1, roll angle a2 and horizontal angle a3 at which the ball camera is installed, to obtain second position information; an adjustment module for converting the first position information of the first target currently detected by the ball camera from the ball camera screen coordinate system to the radar coordinate system of the radar based on the second position information and the third position information of the second target detected by the radar The position information is used to determine that the first target detected by the ball camera and the second target detected by the radar are the same target, and the first target is placed at a specified position on the ball camera screen by adjusting the ball camera; a capture control module is used to determine the rotation advance of the ball camera according to the moving speed of the first target detected when the ball camera tracks the first target, and the current width and height of the first target relative to the ball camera screen, and control the rotation of the ball camera based on the rotation advance, and control the capture timing of capturing the specified part of the first target when the specified part of the first target enters the screen of the ball camera.

[0007] An embodiment of the present application also provides an electronic device, comprising: a processor and a machine-readable storage medium; the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the steps of the above method.

[0008] As can be seen from the above technical solutions, in this application, by using the P, T, and Z state parameter values ​​when the ball camera detects the first position information of the first target, as well as the height h, pitch angle a1, roll angle a2, and horizontal angle a3 at which the ball camera is installed, the first position information (i.e., the real-time orientation) of the first target currently detected by the ball camera is converted from the ball camera screen coordinate system to the radar coordinate system in real time, thereby realizing the verification, correction, and supplementation of the radar detection by the ball camera detection, solving the dependence of the traditional radar-ball linkage system on the radar detection accuracy, and avoiding the situation in which the traditional radar-ball linkage system simply relies on radar detection to link the ball camera rotation, resulting in the ball camera tracking the wrong target or the ball camera having no target in the field of view during the target tracking process, thereby improving the target tracking effect;

[0009] Furthermore, in this embodiment, by combining the moving speed of the first target and the current width and height of the first target relative to the dome camera screen, the rotation lead of the dome camera in the horizontal and / or vertical directions is dynamically determined, thereby avoiding the problems of empty capture and inaccurate capture position caused by the existing use of fixed lead time and fixed time interval. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0011] FIG1 is a flow chart of a method provided in an embodiment of the present application;

[0012] FIG2 is a structural diagram of a thunderball system provided in an embodiment of the present application;

[0013] FIG3 is a schematic diagram of thunder and ball fusion provided in an embodiment of the present application;

[0014] FIG4 is a flow chart of the magnification adjustment control provided in an embodiment of the present application;

[0015] 5A, 5B, and 5C are respectively snapshot effect diagrams provided in embodiments of the present application;

[0016] FIG6 is a structural diagram of a device provided in an embodiment of the present application;

[0017] FIG7 is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0019] The terms used in this application are for the purpose of describing particular embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0020] In conventional land-based traffic radar-visual fusion systems, the radar (such as a millimeter-wave radar) and the camera are fixed in one location. The radar coordinate system corresponding to the radar and the camera coordinate system corresponding to the camera can be converted to each other through fixed rotations, translations, and scaling. Once the radar and camera in the radar-visual fusion system are set up, the conversion relationship between the radar coordinate system and the camera coordinate system is fixed. The following example illustrates this conversion relationship:

[0021] Here, M1 represents the camera's external parameters (represented by a matrix), and M2 represents the camera's internal parameters (represented by a matrix). For fixed-focus cameras, the internal parameters remain fixed after the camera leaves the factory. The camera's external parameters are related to the camera's installation location. Once the camera's installation location is determined, the camera's external parameters also remain fixed.

[0022] However, in a radar-based PTZ system (also known as a radar-pan-tilt system), the dome camera (also known as a dome camera) undergoes PTZ motion, and the dome camera experiences horizontal and pitch motion relative to the radar. This means that the camera's external parameters are not fixed, and the camera's internal parameters also change during zoom. Clearly, the conversion relationship described above for a radar-based fusion system consisting of a radar and camera is not applicable. In the PTZ system, P (pan) represents the horizontal (left-right) rotation angle, T (tilt) represents the vertical (up-down) rotation angle, and Z (zoom) represents the zoom adjustment value.

[0023] Applied to the embodiments of the present application, this embodiment dynamically converts the position information of the target detected by the dome camera from the dome camera image coordinate system to the radar coordinate system by using the real-time position information of the target in the dome camera image as the dome camera PTZ phase changes. To help those skilled in the art better understand the technical solutions provided by the embodiments of the present application and to make the above-mentioned objectives, features, and advantages of the embodiments of the present application more clearly understood, the technical solutions in the embodiments of the present application are further described in detail with reference to the accompanying drawings.

[0024] See Figure 1, which is a flow chart of a method provided in an embodiment of the present application. The method is applied to an electronic device. As an embodiment, the electronic device may be a controller on a speed dome camera or a device independent of the speed dome camera, which is not specifically limited in this embodiment.

[0025] As an example, as shown in Figure 2, the radar 220 and the dome camera 210 can be mounted on the same pole. Optionally, the radar 220 and the dome camera 210 can be mounted at the same or different heights. For example, the radar 220 and the dome camera 210 can both be mounted at a height of approximately 10 to 20 meters above the horizontal plane (e.g., the ground). Furthermore, in this embodiment, the radar 220 and the dome camera 210 can each have the same monitoring range, for example, between 30 and 1000 meters.

[0026] Based on the above description, as shown in FIG1 , the process may include the following steps 101 to 103 .

[0027] Step 101: Based on the P, T, and Z state parameter values ​​when the ball camera detects the first position information of the first target, as well as the height, pitch angle, roll angle, and horizontal angle at which the ball camera is installed, the first position information of the first target currently detected by the ball camera is converted from the ball camera screen coordinate system to the radar coordinate system to obtain the second position information.

[0028] In this embodiment, the height (denoted as h), pitch angle (denoted as a1), roll angle (denoted as a2) and horizontal angle (denoted as a3) at which the ball camera is installed (such as initially installed) generally do not change after the installation is completed.

[0029] As an embodiment, the height h, pitch angle a1, roll angle a2, and horizontal angle a3 of the dome camera when it is initially installed can be obtained by measurement.

[0030] As another example, the dome camera's initial installation height h, pitch angle a1, roll angle a2, and horizontal angle a3 can also be determined using parametric regression methods. For example, several matching pairs of the dome camera's image coordinate system and the radar coordinate system are pre-collected. Each matching pair includes the position information of an object in the dome camera's image coordinate system and the position information of the object in the radar coordinate system at the same time. Then, through nonlinear least squares iteration or gradient descent optimization methods, the optimal initial installation height h, pitch angle a1, roll angle a2, and horizontal angle a3 are obtained.

[0031] It should be noted that the above-mentioned pitch angle a1 refers to the pitch angle at which the ball camera is installed (specifically, it may be the pitch angle of the base for assembling the ball camera). Similarly, the roll angle a2 and the horizontal angle a3 are similar and will not be described in detail.

[0032] In this embodiment, the radar is fixed after installation, while the dome camera tracks the target and performs horizontal and / or pitch movements. Furthermore, the dome camera's lens is telescoped along its optical axis to achieve variable magnification, so that both the intrinsic and extrinsic parameters of the dome camera change in real time with the dome camera's PTZ state. Based on this, in this embodiment, step 101 can be specifically implemented as follows: Based on the first position information of the first target detected by the dome camera (the horizontal position information u and vertical position information v of the first target in the dome camera's image), the P, T, and Z state parameter values ​​at the time the dome camera detected the first position information of the first target, and the dome camera's installation height h, pitch angle a1, roll angle a2, and horizontal angle a3, determine the current dynamic transformation matrix f(u, v, p, t, z, a1, a2, a3, h). Here, the current dynamic transformation matrix f(u,v,p,t,z,a1,a2,a3,h) is used to convert the first target's position information u and v in the dome camera's image to the radar coordinate system, thereby achieving the conversion from the dome camera's image coordinate system (also known as the pixel coordinate system) to the radar coordinate system. Conversely, the inverse of the current dynamic transformation matrix f(u,v,p,t,z,a1,a2,a3,h) yields the conversion from the radar coordinate system to the dome camera's image coordinate system. Subsequently, based on the current dynamic transformation matrix f(u,v,p,t,z,a1,a2,a3,h), the first position information is converted from the dome camera's image coordinate system to the radar coordinate system, yielding the second position information.

[0033] Optionally, in this embodiment, the current dynamic conversion matrix f(u, v, p, t, z, a1, a2, a3, h) is similar to the conversion relationship in the above-mentioned radar-vision fusion system, except that the internal and external parameters of the ball camera are not fixed.

[0034] The dynamic conversion matrix established above realizes the coordinate alignment relationship between the radar coordinate system and the dome camera screen coordinate system, which can conveniently obtain the real-time orientation of the first target detected by the dome camera in the radar coordinate system (of course, the real-time orientation of the second target detected by the radar in the dome camera screen coordinate system can also be obtained), and realizes the real-time review and correction of radar multiple detections, false detections and inaccurate detection positions by the dome camera detection, which solves the dependence of the traditional radar-ball linkage system on radar detection accuracy, and also avoids the situation in which the traditional radar-ball linkage system simply relies on radar detection to link the dome camera rotation, resulting in the dome camera tracking the wrong target or the dome camera having no target in the field of view during the target tracking process, thereby improving the accuracy and stability of target tracking.

[0035] It should be noted that, in this embodiment, the P state parameter value may be the angle of the ball camera along the horizontal direction, the T state parameter value may be the angle of the ball camera along the vertical direction, and the Z state parameter value may be the magnification of the ball camera.

[0036] In this embodiment, the dynamic conversion matrix described above enables real-time acquisition of a one-to-one correspondence between targets detected by the radar and those detected by the dome camera during the camera's rotation. For example, by comparing the second position information obtained through the dynamic conversion matrix with the third position information of the second target detected by the radar, if the second position information and the third position information are found to be identical, or if the positional deviation between the two is within a set error range, the first target detected by the dome camera and the second target detected by the radar are determined to be the same target. This achieves a one-to-one correspondence between targets detected by the radar and those detected by the dome camera during the camera's rotation. When, based on the second position information and the third position information of the second target detected by the radar, it is determined that the first target detected by the dome camera and the second target detected by the radar are the same target, step 102 is executed.

[0037] Step 102: If it is determined that the first target detected by the dome camera and the second target detected by the radar are the same target based on the second position information and the third position information of the second target detected by the radar, the dome camera is adjusted so that the first target is at the specified position of the dome camera screen.

[0038] Alternatively, in this embodiment, an example of a radar and a dome camera detecting the same target is shown in Figure 3. The right side of Figure 3 shows the projection in the radar coordinate system, where 103 is the target ID detected by the radar. The left side of Figure 3 shows the projection in the dome camera coordinate system, where 768, 769, and 773 are the target IDs detected by the dome camera. It can be seen that in the left side of Figure 3 , 103 is projected onto 769, indicating that 103 detected by the radar and 769 detected by the dome camera are the same target in the three-dimensional world.

[0039] In this embodiment, after determining that the first target detected by the dome camera and the second target detected by the radar are the same target, in order to obtain clear detail features of the target (hereinafter referred to as the first target), the dome camera is adjusted so that the first target is at a specified position (such as the center position) on the dome camera screen.

[0040] Optionally, in this embodiment, adjusting the ball camera may include the following steps:

[0041] Step a1, calculating the current width_angle and height_angle of the first target relative to the dome camera screen and the position deviation abs_shift of the first target relative to the center position of the dome camera screen.

[0042] Optionally, the current width_angle of the first target relative to the dome camera screen is determined based on the current pixel width width_pixel of the first target in the dome camera screen and the current horizontal viewing angle current_fov of the dome camera at the current magnification. For example, width_angle is expressed by the following formula: width_angle=width_pixel*current_fov.

[0043] Optionally, the current height of the first target relative to the dome camera screen, height_angle, is determined based on the current pixel height, height_pixel, of the first target in the dome camera screen and the current vertical field of view, current_fov_y, of the dome camera at the current magnification. For example, height_angle is expressed by the following formula: height_angle=height_pixel*current_fov_y.

[0044] Optionally, current_fov and current_fov_y can be obtained by looking up the table according to the current magnification value of the ball camera.

[0045] Optionally, in this embodiment, the positional deviation abs_shift of the first target relative to the center of the dome camera image is determined based on the position of the first target in the dome camera image (e.g., represented by the midpoint of the bottom edge of a target box framing the first target). For example, abs_shift is represented by the following formula: abs_shift = max(0.5-x, 0.5-y);

[0046] Wherein, (x, y) represents the position of the first target in the ball camera screen, for example, represented by the midpoint of the bottom edge of the target frame that frames the first target.

[0047] Step a2: Based on the width_angle, the height_angle and the position deviation abs_shift, the current P state parameter value, T state parameter value and / or Z state parameter value of the ball camera are adjusted so that the first target is displayed at the designated position on the ball camera screen.

[0048] As described in step a2, in order to display the first target at the specified position on the dome camera screen, the dome camera magnification may be adjusted. For example, the dome camera magnification needs to be enlarged to increase the proportion of the first target in the dome camera screen. However, in actual application scenarios, different targets have different sizes. For example, the target can be a large cargo ship with a length of more than 100 meters, or a small speedboat of about 10 meters. Under this premise, if targets of different sizes are to be taken into account, there will often be problems such as large targets being too large, resulting in incomplete targets in the dome camera screen, or small targets being too small, and the details cannot be seen clearly in the dome camera screen.

[0049] To avoid the above problems, this embodiment also requires the following control when adjusting the magnification to achieve accurate magnification control and ensure that there is no risk of target loss during the magnification process. The specific control process will be described below in conjunction with Figure 4.

[0050] Step 103, based on the moving speed of the first target detected when the dome camera tracks the first target, and the current width and height of the first target relative to the dome camera screen, determine the rotation advance of the dome camera, control the rotation of the dome camera based on the rotation advance, and control the timing of capturing the specified part of the first target when the specified part enters the screen of the dome camera.

[0051] In this embodiment, in order to capture clear details of the first target, it is necessary to capture close-up images of designated parts of the first target. Taking the first target as a ship as an example, the designated parts here are, for example, the bow, the middle of the ship, and the stern. In order to capture close-up images of designated parts of the first target, it is often necessary to control the dome camera to rotate a certain amount in advance. Traditional capture schemes generally cause the dome camera to rotate a fixed amount in advance, and then capture close-up images of different designated parts at fixed time intervals. However, due to the large difference in speed between different targets, fixed lead times and fixed time intervals can easily lead to empty captures and inaccurate capture positions.

[0052] Based on this, this embodiment calculates the moving speed of the first target by counting the moving trajectory of the first target during the process of the ball camera tracking the first target. Then, based on the moving speed of the first target and the current width and height of the first target relative to the ball camera screen, the rotation advance of the ball camera is dynamically determined. For example, based on the horizontal moving speed v of the first target detected by the ball camera when tracking the first target, p , the set ball camera linkage time t, and the current width width_angle of the first target relative to the ball camera screen, determine the advance amount of the horizontal rotation of the ball camera; and / or, according to the vertical movement speed v of the first target detected when the ball camera tracks the first target t , the set ball camera linkage time t, and the current height heght_angle of the first target relative to the ball camera screen, to determine the advance amount of rotation of the ball camera in the vertical direction.

[0053] Alternatively, the horizontal rotation lead of the ball camera can be expressed by the following formula: p *t+λ*width_angle.

[0054] The vertical rotation lead of the ball camera can be expressed by the following formula: t *t+λ*heght_angle.

[0055] Among them, λ is the target scale coefficient set.

[0056] By dynamically determining the horizontal and / or vertical rotation lead of the ball camera based on the moving speed of the first target and the current width and height of the first target on the ball camera screen, the problems of empty capture and inaccurate capture position caused by the existing use of fixed lead and fixed time interval are avoided.

[0057] It should be noted that, as an embodiment, the above-mentioned dome camera linkage time t can be set based on experience, and this embodiment does not specifically limit it.

[0058] After controlling the rotation of the dome camera based on the horizontal and / or vertical rotation lead, as the first target moves, the designated portion of the first target gradually enters the camera's image. In this embodiment, deep learning technology can be used to identify the designated portion of the first target and its attribute information. The changing trend of the first target is then tracked and combined with the attribute information of the designated portion to determine the timing for capturing the designated portion, ultimately achieving control over the timing for capturing the designated portion of the first target. Taking a ship as an example, Figures 5A, 5B, and 5C illustrate close-up images of the bow, midship, and stern, respectively, captured according to the aforementioned capture control.

[0059] At this point, the process shown in Figure 1 is completed.

[0060] It can be seen from the process shown in Figure 1 that in this embodiment, by using the P, T, Z state parameter values ​​when the dome camera detects the first position information of the first target, as well as the height h, pitch angle a1, roll angle a2 and horizontal angle a3 at which the dome camera is installed, the first position information of the first target currently detected by the dome camera (that is, the real-time orientation) is converted from the dome camera screen coordinate system to the radar coordinate system in real time, thereby realizing the review, correction and supplement of the radar detection by the dome camera; it solves the dependence of the traditional radar-ball linkage system on the radar detection accuracy and coordinates, and also avoids the situation in the traditional radar-ball linkage system where the dome camera simply relies on radar detection to link the rotation of the dome camera, resulting in the dome camera tracking the wrong target or the dome camera having no target in the field of view during the target tracking process, thereby improving the target tracking effect.

[0061] Furthermore, in this embodiment, the rotation lead of the ball camera in the horizontal and / or vertical directions is dynamically determined by combining the moving speed of the first target and the current width and height of the first target relative to the ball camera screen, thereby avoiding the problems of empty capture and inaccurate capture position caused by the existing use of fixed lead time and fixed time interval.

[0062] The following describes the process shown in Figure 4:

[0063] Refer to Figure 4, which is a flow chart of the rate control provided by an embodiment of the present application. As shown in Figure 4, the process may include the following steps:

[0064] Step 401 , compare the current magnification of the ball camera with the set target magnification; if the current magnification of the ball camera is greater than the target magnification, execute step 402 ; if the current magnification of the ball camera is less than the target magnification, execute step 403 .

[0065] Here, the target magnification may be set based on the type of the first target, which is not specifically limited in this embodiment.

[0066] Step 402: restore the current magnification of the speed dome camera to a specified magnification; the specified magnification is less than or equal to the target magnification.

[0067] For example, the specified magnification is the target magnification.

[0068] Step 403, identifying whether amplifying the current magnification of the ball camera will cause a target loss risk event. If not, increasing the current magnification of the ball camera by a set step size to obtain a new current magnification, and returning to step 401, that is, comparing the current magnification of the ball camera with the set target magnification.

[0069] In this embodiment, the target loss risk event refers to an event in which the first target is not in the screen of the dome camera during the magnification process of the dome camera. As for how to identify whether the current magnification of the dome camera will cause a target loss risk event, it needs to be determined with reference to the current position information of the first target in the dome camera screen, the driving speed of the first target, etc., which is not limited in this embodiment. Here, the current position information of the first target in the dome camera screen may include the current width width_angle of the first target relative to the dome camera screen, the current height heght_angle of the first target relative to the dome camera screen, and the position deviation abs_shift of the first target relative to the center position of the dome camera screen. The calculation method of width_angle, heght_angle, and abs_shift here is as described above.

[0070] At this point, the process shown in Figure 4 is completed.

[0071] Through the process shown in Figure 4, precise control of the dome camera magnification is achieved, avoiding the problem of the target being too large due to the inability to take into account the target scale, resulting in incomplete targets in the dome camera screen, or the problem of the target being too small and the details being unclear in the dome camera screen.

[0072] The above describes the method provided in the embodiment of the present application. The following describes the device provided in the embodiment of the present application.

[0073] Referring to FIG6 , which is a structural diagram of a device according to an embodiment of the present application, the device may include: a conversion module for converting the first position information of a first target currently detected by the dome camera from a dome camera screen coordinate system to a radar coordinate system based on the P, T, and Z state parameter values ​​of the first target detected by the dome camera, as well as the dome camera's installation height h, pitch angle a1, roll angle a2, and horizontal angle a3, to obtain second position information; an adjustment module for adjusting the dome camera so that the first target is at a specified position on the dome camera screen if the first target detected by the dome camera and the second target detected by the radar are the same target based on the second position information and the third position information of the second target detected by the radar; and a capture control module for determining a rotation lead of the dome camera based on the movement speed of the first target detected by the dome camera while tracking the first target, as well as the current width and height of the first target relative to the dome camera screen, and controlling the rotation of the dome camera based on the rotation lead to control the timing of capturing a capture of the specified portion of the first target when the specified portion of the first target enters the dome camera screen.

[0074] Optionally, the first position information is represented by position information u and v of the first target in the ball camera screen.

[0075] The first position information of the first target currently detected by the ball camera is converted from the ball camera screen coordinate system to the radar coordinate system based on the P, T, and Z state parameter values ​​when the ball camera detects the first position information of the first target, and the height h, pitch angle a1, roll angle a2, and horizontal angle a3 at which the ball camera is installed. The second position information obtained includes: the first position information of the first target detected by the ball camera, the P, T, and Z state parameter values ​​when the ball camera detects the first position information, and the height h, pitch angle a1, roll angle a2, and horizontal angle a3 at which the ball camera is installed. The roll angle a2 and the horizontal angle a3 are used to determine the current dynamic conversion matrix f(u, v, p, t, z, a1, a2, a3, h); the current dynamic conversion matrix f(u, v, p, t, z, a1, a2, a3, h) is used to convert the position information u and v of the first target in the ball camera screen to the radar coordinate system; based on the current dynamic conversion matrix f(u, v, p, t, z, a1, a2, a3, h), the first position information is converted from the ball camera screen coordinate system to the radar coordinate system to obtain the second position information.

[0076] Optionally, the adjustment of the dome camera includes: calculating the current width width_angle, height heght_angle of the first target relative to the dome camera screen and the position deviation abs_shift of the first target relative to the center position of the dome camera screen; based on the width width_angle, the height heght_angle and the position deviation abs_shift, adjusting the current P state parameter value, T state parameter value, Z state parameter value, and / or magnification of the dome camera so that the first target is displayed at the specified position on the dome camera screen.

[0077] Optionally, the adjustment of the magnification includes: comparing the current magnification of the dome camera with a set target magnification; if the current magnification of the dome camera is greater than the target magnification, restoring the current magnification of the dome camera to a specified magnification; the specified magnification is less than or equal to the target magnification; if the current magnification of the dome camera is less than the target magnification, identifying whether the current magnification of the dome camera being magnified will cause a target loss risk event, if not, increasing the current magnification of the dome camera by a set step size, and returning to the step of comparing the current magnification of the dome camera with the set target magnification; the target loss risk event refers to an event in which the first target moves out of the screen of the dome camera during the magnification magnification process of the dome camera.

[0078] Optionally, the determining the rotation advance amount of the ball camera based on the moving speed of the first target detected when the ball camera tracks the first target and the current width and height of the first target relative to the ball camera screen includes: determining the horizontal moving speed v of the first target detected when the ball camera tracks the first target based on the horizontal moving speed v of the first target detected when the ball camera tracks the first target. p , the set ball camera linkage time t, and the current width width_angle of the first target relative to the ball camera screen, determine the advance amount of the horizontal rotation of the ball camera; and / or, according to the vertical movement speed v of the first target detected when the ball camera tracks the first target t , the set ball camera linkage time t, and the current height heght_angle of the first target relative to the ball camera screen, to determine the advance amount of rotation of the ball camera in the vertical direction.

[0079] Optionally, the current width_angle of the first target relative to the screen of the dome camera is determined according to the current pixel width width_pixel of the first target in the screen of the dome camera and the current horizontal field of view current_fov of the dome camera at the current magnification;

[0080] The current height_angle of the first target relative to the screen of the dome camera is determined according to the current pixel height height_pixel of the first target in the screen of the dome camera and the current vertical field angle current_fov_y of the dome camera at the current magnification.

[0081] At this point, the device structure diagram shown in FIG6 is completed.

[0082] The present application also provides a hardware structure for the apparatus shown in FIG6 . Referring to FIG7 , FIG7 is a block diagram of an electronic device provided in an embodiment of the present application. As shown in FIG7 , the hardware structure may include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; the processor is configured to execute the machine-executable instructions to implement the method disclosed in the above example of the present application.

[0083] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0084] Exemplarily, the machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0085] The systems, devices, modules, or units described in the above embodiments may be implemented by a computer or entity, or by a product having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0086] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0087] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0088] The present application is described with reference to the flow chart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart flow or multiple flows and / or one box or multiple boxes of the block diagram.

[0089] Moreover, these computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0090] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0091] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A target capture method based on a thunder ball system, wherein the thunder ball system includes a ball camera and a radar, characterized in that: The method includes: Based on the P state parameter value, the T state parameter value, the Z state parameter value when the dome camera detects the first position information of the first target, and the height h, the pitch angle a1, the roll angle a2, and the horizontal angle a3 at which the dome camera is installed, the first position information of the first target is converted from the dome camera screen coordinate system to the radar coordinate system of the radar to obtain second position information; If it is determined based on the second position information and the third position information of the second target detected by the radar that the first target detected by the speed dome camera and the second target detected by the radar are the same target, the speed dome camera is adjusted so that the first target is at a designated position on the speed dome camera screen; determining a rotation advance amount of the dome camera according to a moving speed of the first target detected when the dome camera tracks the first target, and a current width and height of the first target relative to a screen of the dome camera; Based on the rotation advance amount, the ball camera is controlled to rotate, and When the designated part of the first target enters the screen of the ball camera, the ball camera is controlled to capture the designated part of the first target at a capture timing.

2. The method according to claim 1, characterized in that The first position information includes horizontal position information u and vertical position information v of the first target in the dome camera screen; based on the P state parameter value, T state parameter value, Z state parameter value when the dome camera detects the first position information of the first target, and the height h, pitch angle a1, roll angle a2 and horizontal angle a3 at which the dome camera is installed, the first position information of the first target is converted from the dome camera screen coordinate system to the radar coordinate system of the radar, and the second position information obtained includes: Determine a current dynamic conversion matrix f(u, v, p, t, z, a1, a2, a3, h) based on the first position information of the first target detected by the dome camera, the P state parameter value, the T state parameter value, the Z state parameter value when the dome camera detects the first position information, and the height h, pitch angle a1, roll angle a2, and horizontal angle a3 at which the dome camera is installed; the current dynamic conversion matrix f(u, v, p, t, z, a1, a2, a3, h) is used to convert the horizontal position information u and the vertical position information v of the first target in the dome camera image into the radar coordinate system; Based on the current dynamic conversion matrix f(u, v, p, t, z, a1, a2, a3, h), the first position information is converted from the dome camera screen coordinate system to the radar coordinate system to obtain the second position information.

3. The method according to claim 1 or 2, characterized in that The adjustment of the ball camera includes: Calculate the current width_angle and height_angle of the first target relative to the dome camera screen and the position deviation abs_shift of the first target relative to the center position of the dome camera screen; Based on the width width_angle, the height heght_angle and the position deviation abs_shift, one or more of the current P state parameter value, T state parameter value and Z state parameter value of the ball camera are adjusted so that the first target is displayed at the specified position of the ball camera screen.

4. The method according to claim 3, characterized in that The adjusting of the Z state parameter value includes: Comparing the current magnification of the ball camera with a set target magnification; If the current magnification of the ball camera is greater than the target magnification, the current magnification of the ball camera is restored to the specified magnification; the specified magnification is less than or equal to the target magnification; If the current magnification of the ball camera is less than the target magnification, then Identify whether the current magnification of the ball camera will cause a target loss risk event if it is amplified. If the target loss risk event will not be caused, the current magnification of the ball camera is increased by a set step size, and the process returns to the step of comparing the current magnification of the ball camera with the set target magnification; the target loss risk event refers to the event that the first target moves out of the screen of the ball camera during the magnification magnification process of the ball camera.

5. The method according to any one of claims 1 to 4, characterized in that The determining of the rotation advance amount of the ball camera based on the moving speed of the first target detected when the ball camera tracks the first target and the current width and height of the first target relative to the ball camera screen includes at least one of the following: According to the horizontal moving speed v of the first target detected by the ball camera when tracking the first target p , the set ball camera linkage time t, and the current width width_angle of the first target relative to the ball camera screen, determining the advance amount of rotation of the ball camera in the horizontal direction; and / or, According to the vertical moving speed v of the first target detected by the ball camera when tracking the first target t , the set ball camera linkage time t, and the current height heght_angle of the first target relative to the ball camera screen, to determine the advance amount of rotation of the ball camera in the vertical direction.

6. The method according to any one of claims 3 to 5, characterized in that: The current width_angle of the first target relative to the dome camera screen is determined according to the current pixel width width_pixel of the first target in the dome camera screen and the current horizontal field angle current_fov of the dome camera at the current magnification; The current height_angle of the first target relative to the dome camera screen is determined according to the current pixel height height_pixel of the first target in the dome camera screen and the current vertical field of view current_fov_y of the dome camera at the current magnification.

7. A target capture device based on the Thunderball system, characterized in that: The thunder ball system includes a ball camera and a radar, and the device includes: a conversion module, configured to convert the first position information of the first target from the dome camera's screen coordinate system to the radar coordinate system of the radar based on the P state parameter value, the T state parameter value, the Z state parameter value when the dome camera detects the first position information of the first target, and the height h, the pitch angle a1, the roll angle a2, and the horizontal angle a3 at which the dome camera is installed, to obtain second position information; an adjustment module, configured to adjust the dome camera so that the first target is at a designated position on the dome camera screen if it is determined that the first target detected by the dome camera and the second target detected by the radar are the same target based on the second position information and the third position information of the second target detected by the radar; The capture control module is used to determine the rotation advance of the ball camera based on the movement speed of the first target detected when the ball camera tracks the first target, and the current width and height of the first target relative to the ball camera screen, and control the rotation of the ball camera based on the rotation advance, and control the capture timing of the specified part of the first target when the specified part enters the screen of the ball camera.

8. The device according to claim 7, characterized in that The first position information is represented by the horizontal position information u and the vertical position information v of the first target in the ball camera screen; The first position information of the first target currently detected by the dome camera is converted from the dome camera screen coordinate system to the radar coordinate system of the radar based on the P state parameter value, the T state parameter value, the Z state parameter value, and the height h, the pitch angle a1, the roll angle a2, and the horizontal angle a3 at which the dome camera is installed when the dome camera detects the first position information of the first target, to obtain the second position information, comprising: Determine a current dynamic conversion matrix f(u, v, p, t, z, a1, a2, a3, h) based on the first position information of the first target detected by the dome camera, the P state parameter value, the T state parameter value, the Z state parameter value when the dome camera detects the first position information of the first target, and the height h, pitch angle a1, roll angle a2, and horizontal angle a3 at which the dome camera is installed; the current dynamic conversion matrix f(u, v, p, t, z, a1, a2, a3, h) is used to convert the horizontal position information u and the vertical position information v of the first target in the dome camera image to the radar coordinate system; Based on the current dynamic conversion matrix f(u, v, p, t, z, a1, a2, a3, h), the first position information is converted from the dome camera screen coordinate system to the radar coordinate system to obtain the second position information.

9. The device according to claim 7 or 8, characterized in that The adjustment module adjusts the ball camera including: Calculate the current width_angle and height_angle of the first target relative to the dome camera screen and the position deviation abs_shift of the first target relative to the center position of the dome camera screen; Based on the width width_angle, the height heght_angle and the position deviation abs_shift, the current P state parameter value, T state parameter value and / or Z state parameter value of the ball camera are adjusted so that the first target is displayed at the specified position of the ball camera screen.

10. The device according to claim 9, wherein The adjusting of the Z state parameter value includes: Comparing the current magnification of the ball camera with a set target magnification; If the current magnification of the ball camera is greater than the target magnification, the current magnification of the ball camera is restored to the specified magnification; the specified magnification is less than or equal to the target magnification; If the current magnification of the ball camera is less than the target magnification, it is identified whether the current magnification of the ball camera will cause a target loss risk event when it is enlarged. If it will not cause a target loss risk event, the current magnification of the ball camera is increased by a set step size, and the process returns to the step of comparing the current magnification of the ball camera with the set target magnification. The target loss risk event refers to an event in which the first target moves out of the screen of the ball camera during the magnification enlargement process of the ball camera.

11. The device according to any one of claims 7 to 10, characterized in that: The determining of the rotation advance amount of the ball camera according to the moving speed of the first target detected when the ball camera tracks the first target and the current width and height of the first target relative to the ball camera screen includes: According to the horizontal moving speed v of the first target detected by the ball camera when tracking the first target p , the set ball camera linkage time t, and the current width width_angle of the first target relative to the ball camera screen, determining the advance amount of rotation of the ball camera in the horizontal direction; and / or, According to the vertical moving speed v of the first target detected by the ball camera when tracking the first target t , the set ball camera linkage time t, and the current height heght_angle of the first target relative to the ball camera screen, to determine the advance amount of rotation of the ball camera in the vertical direction.

12. An electronic device, characterized in that: The electronic device includes: a processor and a machine-readable storage medium; The machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method steps of any one of claims 1-6.

Images