Gun and ball linkage control method, system and equipment and storage medium

Abstract

The invention provides a gun-dome camera linkage control method, system and device and a storage medium. The method comprises the following steps: establishing a perspective transformation matrix between a coordinate system of a gun camera and a coordinate system of a dome camera; establishing a coordinate conversion relationship between the coordinate system of the dome camera and the pose parameters of the dome holder; determining a target tracking point in a shot picture of the gun camera; on the basis of the perspective transformation matrix and the coordinate transformation relation, determining a ball machine holder pose parameter corresponding to the target tracking point; and controlling the motion of the dome camera holder according to the pose parameters of the dome camera holder. By establishing the mapping conversion relation between the gun camera picture and the dome camera picture and the corresponding relation between the dome camera picture and the holder coordinate system, the gun-dome linkage control can be quickly realized, and the efficiency and precision of the gun-dome linkage control are improved.

Description

Background Art

[0002] The linkage between a fixed gun camera and a PTZ camera is a monitoring technology that controls the two cameras in a coordinated manner. First, the global image of the monitored area is captured by the fixed gun camera with a wide field of view. When a target of interest is detected in the global image captured by the gun camera, the PTZ camera is linked to rotate to the position of the target of interest, and a local detailed image of the target is output, realizing automatic intelligent tracking and monitoring of suspicious targets and areas. The existing process of gun-PTZ linkage requires complex data processing and complex calculations combining the internal and external parameters of the cameras, and it is difficult to meet the requirements for the efficiency and accuracy of linkage control.

[0003] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present application, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0004] Aiming at the problems in the prior art, the purpose of the present application is to provide a gun-PTZ linkage control method, system, device, and storage medium. By establishing the mapping transformation relationship between the images of the gun camera and the PTZ camera and the corresponding relationship between the image of the PTZ camera and the PTZ coordinate system, the gun-PTZ linkage control can be quickly realized, and the efficiency and accuracy of the gun-PTZ linkage control can be improved.

[0005] The embodiment of the present application provides a gun-PTZ linkage control method, including the following steps:

[0006] Establish a perspective transformation matrix between the coordinate systems of the gun camera and the PTZ camera;

[0007] Establish the coordinate transformation relationship between the coordinate system of the PTZ camera and the pose parameters of the PTZ;

[0008] Determine the target tracking point in the captured image of the gun camera;

[0009] Based on the perspective transformation matrix and the coordinate transformation relationship, determine the pose parameters of the PTZ corresponding to the target tracking point;

[0010] Control the movement of the PTZ according to the pose parameters of the PTZ.

[0011] In some embodiments, before establishing the perspective transformation matrix between the coordinate systems of the gun camera and the PTZ camera, the following steps are further included:

[0012] Control the PTZ camera to rotate until there is an overlapping area between the captured image of the PTZ camera and the captured image of the gun camera.

[0013] In some embodiments, establishing the perspective transformation matrix between the coordinate systems of the gun camera and the PTZ camera includes:

[0014] Establish a perspective transformation matrix between the coordinate system of the fixed camera and the coordinate system of the PTZ camera according to the captured images of the PTZ camera and the fixed camera.

[0015] In some embodiments, establishing a perspective transformation matrix between the coordinate system of the fixed camera and the coordinate system of the PTZ camera includes the following steps:

[0016] Select multiple first key points in the captured image of the fixed camera, and determine the first coordinates of the multiple first key points in the captured image of the fixed camera;

[0017] Determine the corresponding position points of the multiple first key points in the captured image of the PTZ camera, and determine the second coordinates of the corresponding position points in the captured image of the PTZ camera;

[0018] Establish a perspective transformation matrix between the coordinate system of the fixed camera and the coordinate system of the PTZ camera according to the first coordinates of the first key points and the second coordinates of the corresponding position points.

[0019] In some embodiments, establishing a coordinate transformation relationship between the coordinate system of the PTZ camera and the pose parameters of the PTZ camera includes the following steps:

[0020] Select multiple second key points in the image of the PTZ camera, and determine the coordinates of the second key points in the coordinate system of the PTZ camera;

[0021] Record the pose parameters of the PTZ camera corresponding to each second key point respectively;

[0022] Establish a coordinate transformation relationship between the coordinate system of the PTZ camera and the pose parameters of the PTZ camera according to the coordinates of the second key points in the coordinate system of the PTZ camera and the pose parameters of the corresponding PTZ camera.

[0023] In some embodiments, establishing a coordinate transformation relationship between the coordinate system of the PTZ camera and the pose parameters of the PTZ camera includes the following steps:

[0024] Establish a relationship equation between the coordinate system of the PTZ camera and the pose parameters of the PTZ camera;

[0025] Based on the coordinates of the second key points in the coordinate system of the PTZ camera and the pose parameters of the corresponding PTZ camera, use the least squares method to solve the equation coefficients in the relationship equation.

[0026] In some embodiments, the pose parameters of the corresponding PTZ camera include the horizontal rotation angle and the vertical rotation angle of the PTZ camera.

[0027] In some embodiments, based on the perspective transformation matrix and the coordinate transformation relationship, determining the pose parameters of the PTZ camera corresponding to the target tracking point includes the following steps:

[0028] Based on the perspective transformation matrix, map the target tracking point to the coordinate system of the dome camera to obtain the coordinates of the target tracking point in the coordinate system of the dome camera;

[0029] According to the coordinate conversion relationship, determine the dome camera pan-tilt pose parameters corresponding to the coordinates of the target tracking point in the coordinate system of the dome camera.

[0030] The embodiment of the present application also provides a fixed and dome camera linkage control system for implementing the fixed and dome camera linkage control method. The system includes:

[0031] A perspective transformation module for establishing a perspective transformation matrix between the coordinate system of the fixed camera and the coordinate system of the dome camera;

[0032] A coordinate conversion module for establishing a coordinate conversion relationship between the coordinate system of the dome camera and the dome camera pan-tilt pose parameters;

[0033] A tracking conversion module for determining a target tracking point in the captured image of the fixed camera and determining the dome camera pan-tilt pose parameters corresponding to the target tracking point based on the perspective transformation matrix and the coordinate conversion relationship;

[0034] A linkage control module for controlling the movement of the dome camera pan-tilt according to the dome camera pan-tilt pose parameters.

[0035] In some embodiments, the linkage control module is further configured to control the dome camera to rotate until there is an overlapping area between the captured image of the dome camera and the captured image of the fixed camera before establishing the perspective transformation matrix between the coordinate system of the fixed camera and the coordinate system of the dome camera.

[0036] The embodiment of the present application also provides a fixed and dome camera linkage control device, including:

[0037] A processor;

[0038] A memory storing executable instructions of the processor;

[0039] Wherein, the processor is configured to execute the steps of the fixed and dome camera linkage control method by executing the executable instructions.

[0040] The embodiment of the present application also provides a computer-readable storage medium for storing a program, and when the program is executed by a processor, the steps of the fixed and dome camera linkage control method are implemented.

[0041] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure.

[0042] The fixed and dome camera linkage control method, system, device and storage medium of the present application have the following beneficial effects:

[0043] By establishing the mapping and conversion relationship between the camera images of the pan-tilt and the corresponding relationship between the camera images of the dome camera and the pan-tilt coordinate system, this application can quickly achieve the pan-tilt and dome camera linkage control, improving the efficiency and accuracy of the pan-tilt and dome camera linkage control. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Other features, objects, and advantages of this application will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings.

[0045] Figure 1 is a flowchart of the pan-tilt and dome camera linkage control method according to an embodiment of this application;

[0046] Figure 2 is a schematic diagram of the captured image of the pan-tilt camera according to an embodiment of this application;

[0047] Figure 3 is a schematic diagram of the captured image of the dome camera according to an embodiment of this application;

[0048] Figure 4 is to Figure 2 the image in Figure 3 after mapping and overlaying onto

[0049] Figure 5 is a schematic diagram of the structure of the pan-tilt and dome camera linkage control system according to an embodiment of this application;

[0050] Figure 6 is a schematic diagram of the structure of the pan-tilt and dome camera linkage control device according to an embodiment of this application;

[0051] Figure 7 is a schematic diagram of the structure of the computer-readable storage medium according to an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0053] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices. Although terms such as "first" or "second" are used in this specification to denote certain features, they are only for the purpose of indication and do not limit the quantity and importance of the specific features.

[0054] The flowcharts shown in the accompanying drawings are only exemplary illustrations and do not necessarily include all steps. For example, some steps can be further decomposed, while some steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.

[0055] Gun-ball linkage control is a commonly used technical means in the field of security technology. The fixed camera usually provides a wide-angle view for monitoring a large area. The PTZ camera can observe specific targets in detail through the pan-tilt rotation and zoom functions. The existing gun-ball linkage control algorithms are complex and it is relatively difficult to control the accuracy. Further optimization is still needed in terms of linkage control efficiency and accuracy. Based on this, the present application provides a gun-ball linkage control method based on coordinate mapping and coordinate fitting. By establishing the mapping transformation relationship between the fixed camera image and the PTZ camera image and the corresponding relationship between the PTZ camera image and the pan-tilt coordinate system, the gun-ball linkage control can be quickly realized, and the efficiency and accuracy of the gun-ball linkage control can be improved.

[0056] As Figure 1 shown, an embodiment of the present application provides a gun-ball linkage control method, including the following steps:

[0057] S100: Establish a perspective transformation matrix between the coordinate system of the fixed camera and the coordinate system of the PTZ camera;

[0058] The coordinate system of the fixed camera refers to the image coordinate system of the fixed camera. Taking a vertex or the center point of the image captured by the fixed camera as the center origin, the positive direction of the X-axis is horizontally to the right, and the positive direction of the Y-axis is horizontally downward or upward. Hereinafter, the coordinates of a point in the coordinate system of the fixed camera are represented as (x, y);

[0059] The coordinate system of the PTZ camera refers to the image coordinate system of the PTZ camera. Taking a vertex or the center point of the image captured by the PTZ camera as the center origin, the positive direction of the x-axis is horizontally to the right, and the positive direction of the y-axis is horizontally downward or upward. Hereinafter, the coordinates of a point in the coordinate system of the PTZ camera are represented as (X, Y);

[0060] By establishing a perspective transformation matrix between the coordinate system of the fixed camera and the coordinate system of the PTZ camera, any point in the captured image of the fixed camera can be mapped into the coordinate system of the PTZ camera;

[0061] S200: Establish the coordinate transformation relationship between the coordinate system of the PTZ camera and the pose parameters of the PTZ;

[0062] In this embodiment, the pose parameters corresponding to the PTZ include the horizontal rotation angle P value and the vertical rotation angle T value of the PTZ;

[0063] By establishing the coordinate transformation relationship between the coordinate system of the PTZ camera and the pose parameters of the PTZ, after determining a point to be tracked in the coordinate system of the PTZ camera, the pose parameters of the PTZ when tracking this point can be correspondingly obtained, so as to control the PTZ to perform tracking;

[0064] S300: Determine the target tracking point in the captured image of the fixed camera;

[0065] S400: Based on the perspective transformation matrix and the coordinate transformation relationship, determine the pose parameters of the PTZ corresponding to the target tracking point;

[0066] S500: Control the movement of the PTZ according to the pose parameters of the PTZ;

[0067] Among them, control the movement of the PTZ according to the horizontal rotation angle P value and the vertical rotation angle T value of the PTZ, adjust the shooting angle of the PTZ camera, so that the PTZ camera can track and shoot the target tracking point, thereby realizing the fixed-PTZ linkage control.

[0068] By adopting the fixed-PTZ linkage control method of the present application, by establishing the mapping transformation relationship between the fixed camera image and the PTZ camera image and the corresponding relationship between the PTZ camera image and the PTZ coordinate system, after determining the target tracking point according to the captured image of the fixed camera, the pose parameters of the PTZ corresponding to the target tracking point can be quickly calculated, and the fixed-PTZ linkage control can be quickly realized, improving the efficiency and accuracy of the fixed-PTZ linkage control, and continuously capturing the target of interest. The implementation method of the present application is simple, only a few sets of position coordinates need to be measured, and there is no need to perform distortion correction on the camera, and the operation is simple. The present application maps the fixed camera image into the two-dimensional coordinate system of the PTZ camera, and then establishes the corresponding relationship between the two-dimensional coordinate system of the PTZ and the PTZ coordinate system. After actual testing, the linkage control effect is accurate and the error is small.

[0069] This application can not only achieve linkage for single-gun cameras and single-dome cameras, but also apply to the linkage of multiple-gun cameras and single-dome cameras. When applied to the scenario of multiple-gun cameras and single-dome cameras, for each gun camera, a perspective transformation matrix is established between the coordinate system of the gun camera and the coordinate system of the dome camera, so that the target tracking points of each gun camera can be converted into the pan-tilt pose parameters of the dome camera. When determining a target tracking point in the captured image of the gun camera, according to the perspective transformation matrix between the gun camera and the dome camera, the target tracking point captured by the gun camera is mapped and transformed into the coordinate system of the dome camera, and then the pan-tilt pose parameters of the dome camera can be determined through the coordinate conversion relationship between the coordinate system of the dome camera and the pan-tilt pose parameters. This application can also be applied to the scenarios of single-gun cameras and multiple-dome cameras or multiple-gun cameras and multiple-dome cameras. For each pair of gun cameras and dome cameras that need to be linked and controlled, a mapping transformation matrix for linkage control is established. When there are multiple dome cameras, for each dome camera, the coordinate conversion relationship between the coordinate system of the dome camera and the pan-tilt pose parameters is established respectively.

[0070] The following combines Figures 2 to 4 to specifically introduce the implementation method of the linkage control method for the gun camera. Among them, Figure 2 is a schematic diagram of the captured image of the gun camera in an embodiment of this application. Figure 3 is a schematic diagram of the captured image of the dome camera in an embodiment of this application. Figure 4 is to Figure 2 the image in Figure 3 after mapping and overlaying. Figure 4 The slash shadow part in Figure 2 represents the part of the image in Figure 4 mapped and overlaid. It is only used for principle explanation. In actual applications, it is not necessary to map the entire captured image of the gun camera into the coordinate system of the dome camera. Only the perspective transformation matrix from Figure 2 to Figure 3 needs to be established according to the coordinates of several first key points, and it is not necessary to process each pixel point of the entire captured image of the gun camera. Therefore, the mapping transformation algorithm of this application is simple and easy to implement quickly.

[0071] In this embodiment, before the step S100: establishing a perspective transformation matrix between the coordinate system of the gun camera and the coordinate system of the dome camera, the following steps are further included:

[0072] Control the dome camera to rotate until there is an overlapping area between the captured image of the dome camera and the captured image of the gun camera.

[0073] As Figure 2 and Figure 3 shown, by controlling the dome pan-tilt, drive the dome camera to rotate until the captured image of the dome camera (Figure 3 ) The position with an overlapping area with the captured image of the pan-tilt camera Figure 2 ), so that the common first key points can be found in the captured images of the pan-tilt camera and the fixed camera, and the perspective transformation matrix between the two coordinate systems can be established.

[0074] In this embodiment, in the step S100, establishing the perspective transformation matrix between the coordinate system of the fixed camera and the coordinate system of the pan-tilt camera includes:

[0075] Selecting a plurality of first key points in the captured image of the fixed camera and determining the first coordinates of the plurality of first key points in the captured image of the fixed camera;

[0076] Taking Figure 2 as an example, four first key points p1 to p4 are selected in the captured image of the fixed camera, and the first coordinates p1(x1, y1), p2(x2, y2), p3(x3, y3), p4(x4, y4) of the four first key points in the coordinate system of the fixed camera are determined; the number and positions of the first key points here are only examples and do not serve as the requirements for the protection scope of this application. In practical applications, the number of first key points can be determined according to the actual captured image and calculation requirements, and some relatively iconic position points can be selected as the first key points;

[0077] Determining the corresponding position points of the plurality of first key points in the captured image of the pan-tilt camera and determining the second coordinates of the corresponding position points in the captured image of the pan-tilt camera;

[0078] Taking Figure 3 as an example, the corresponding position points P1 to P4 of the four first key points are determined in the captured image of the pan-tilt camera, and the second coordinates P1(X1, Y1), P2(X2, Y2), P3(X3, Y3), P4(X4, Y4) of the corresponding position points in the captured image of the pan-tilt camera are determined;

[0079] Establishing the perspective transformation matrix between the coordinate system of the fixed camera and the coordinate system of the pan-tilt camera according to the first coordinates of the first key points and the second coordinates of the corresponding position points. This perspective transformation can be realized, for example, through the cv2.warpPerspective function.

[0080] Among them, first, the equation corresponding to formula 1 is constructed as follows:

[0081]

[0082] Among them, (x, y) represents the coordinates of a point in the coordinate system of the camera on the gun body, that is, the coordinates before the mapping transformation, and (X, Y) represents the coordinates of a point in the coordinate system of the camera on the spherical machine, that is, the coordinates after the mapping transformation. Substituting the first coordinates and the second coordinates of the four first key points into Formula 1, the coefficients a 11 ~a 33 can be solved. Since it is two-dimensional image information, the z value is generally 1.

[0083] In this embodiment, the step S200: establishing the coordinate conversion relationship between the coordinate system of the camera on the spherical machine and the pose parameters of the spherical machine pan-tilt includes the following steps:

[0084] Select multiple second key points in the image of the camera on the spherical machine, and determine the coordinates of the second key points in the coordinate system of the camera on the spherical machine;

[0085] Record the pose parameters of the spherical machine pan-tilt corresponding to each second key point respectively; that is, record the current pose parameters of the spherical machine pan-tilt after adjusting the camera on the spherical machine with each second key point as the tracking point respectively;

[0086] Establish the coordinate conversion relationship between the coordinate system of the camera on the spherical machine and the pose parameters of the spherical machine pan-tilt according to the coordinates of the second key points in the coordinate system of the camera on the spherical machine and the corresponding pose parameters of the spherical machine pan-tilt.

[0087] For example, randomly select 6 second key points in the image of the camera on the spherical machine, and their coordinates are (X1, Y1)~(X6, Y6), and the corresponding pose parameters of the spherical machine pan-tilt are (p1, t1)~(p6, t6) respectively. The number of the second key points here is only an example. In practical applications, other numbers of second key points can be selected according to the algorithm requirements, and it is not limited to what is shown here.

[0088] In this embodiment, establishing the coordinate conversion relationship between the coordinate system of the camera on the spherical machine and the pose parameters of the spherical machine pan-tilt includes the following steps:

[0089] Establish the relationship equation between the coordinate system of the camera on the spherical machine and the pose parameters of the spherical machine pan-tilt;

[0090] p = b0 + b1X + b2Y + b3XY + b4X 2 + b5Y 2 (2)

[0091] t = c0 + c1X + c2Y + c3XY + c4X 2 + c5Y 2 (3)

[0092] Substitute the coordinates of the six second key points and the corresponding pose parameters of the spherical machine pan-tilt into the above Formulas (2) and (3) to obtain the following formulas:

[0093]

[0094] Let

[0095]

[0096] obtain

[0097] Based on the coordinates of the second key point in the coordinate system of the dome camera and the pose parameters corresponding to the dome camera pan-tilt, the equation coefficients in the relationship equation are solved using the least squares method;

[0098] That is, for the above formula 4, B = (M T M) -1 M T P, C = (M T M) -1 M T T is obtained by solving using the least squares method. Thus, the coordinate transformation relationship between the coordinate system of the dome camera and the pose parameters of the dome camera pan-tilt can be established.

[0099] In this embodiment, the step S400: determining the pose parameters of the dome camera pan-tilt corresponding to the target tracking point based on the perspective transformation matrix and the coordinate transformation relationship includes the following steps:

[0100] Based on the perspective transformation matrix, the target tracking point is mapped to the coordinate system of the dome camera to obtain the coordinates of the target tracking point in the coordinate system of the dome camera; specifically, the coordinates (x, y) of the target tracking point in the fixed camera coordinate system can be used to calculate the coordinates (X, Y) of the target tracking point in the coordinate system of the dome camera through the following formula 5:

[0101]

[0102] According to the coordinate transformation relationship, the pose parameters of the dome camera pan-tilt corresponding to the coordinates of the target tracking point in the coordinate system of the dome camera are determined; specifically, substituting the coordinates (X, Y) of the target tracking point in the coordinate system of the dome camera into the above formulas 2 and 3, the corresponding pose parameters p and t of the dome camera pan-tilt can be obtained.

[0103] As Figure 5 shown, an embodiment of the present application further provides a fixed-dome linkage control system for implementing the above fixed-dome linkage control method. The system includes:

[0104] The perspective transformation module M100 is used to establish a perspective transformation matrix between the coordinate system of the PTZ camera and the coordinate system of the dome camera; by establishing a perspective transformation matrix between the coordinate system of the PTZ camera and the coordinate system of the dome camera, any point in the captured image of the PTZ camera can be mapped to the coordinate system of the dome camera;

[0105] The coordinate conversion module M200 is used to establish a coordinate conversion relationship between the coordinate system of the dome camera and the pose parameters of the dome camera pan-tilt head; in this embodiment, the pose parameters corresponding to the dome camera pan-tilt head include the horizontal rotation angle P value and the vertical rotation angle T value of the dome camera pan-tilt head;

[0106] The tracking conversion module M300 is used to determine a target tracking point in the captured image of the PTZ camera, and based on the perspective transformation matrix and the coordinate conversion relationship, determine the pose parameters of the dome camera pan-tilt head corresponding to the target tracking point;

[0107] The linkage control module M400 is used to control the movement of the dome camera pan-tilt head according to the pose parameters of the dome camera pan-tilt head.

[0108] By adopting the PTZ-dome linkage control system of the present application, by establishing a mapping conversion relationship between the images of the PTZ camera and the dome camera and the corresponding relationship between the image of the dome camera and the coordinate system of the pan-tilt head, after determining the target tracking point according to the captured image of the PTZ camera, the pose parameters of the dome camera pan-tilt head corresponding to the target tracking point can be quickly calculated, and the PTZ-dome linkage control can be quickly realized, improving the efficiency and accuracy of the PTZ-dome linkage control and continuously capturing the target of interest. The implementation method of the present application is simple, only a few sets of position coordinates need to be measured, and there is no need to perform distortion correction on the camera, and the operation is simple. The present application maps the image of the PTZ camera to the two-dimensional coordinate system of the dome camera, and then establishes the corresponding relationship between the two-dimensional coordinate system of the dome camera and the coordinate system of the pan-tilt head. After actual testing, the linkage control effect is accurate and the error is small.

[0109] In this embodiment, the linkage control module M400 is further used to control the dome camera to rotate to an area where the captured image of the dome camera coincides with the captured image of the PTZ camera before establishing the perspective transformation matrix between the coordinate system of the PTZ camera and the coordinate system of the dome camera.

[0110] In this embodiment, the perspective transformation module M100 establishes a perspective transformation matrix between the coordinate system of the pan-tilt camera and the coordinate system of the dome camera, including: selecting a plurality of first key points in the captured image of the pan-tilt camera, and determining the first coordinates of the plurality of first key points in the captured image of the pan-tilt camera; determining the corresponding position points of the plurality of first key points in the captured image of the dome camera, and determining the second coordinates of the corresponding position points in the captured image of the dome camera; establishing a perspective transformation matrix between the coordinate system of the pan-tilt camera and the coordinate system of the dome camera according to the first coordinates of the first key points and the second coordinates of the corresponding position points.

[0111] In this embodiment, the coordinate conversion module M200 establishes a coordinate conversion relationship between the coordinate system of the dome camera and the pose parameters of the dome camera pan-tilt, including: selecting a plurality of second key points in the image of the dome camera, and determining the coordinates of the second key points in the coordinate system of the dome camera; respectively recording the pose parameters of the dome camera pan-tilt corresponding to each second key point; establishing a coordinate conversion relationship between the coordinate system of the dome camera and the pose parameters of the dome camera pan-tilt according to the coordinates of the second key points in the coordinate system of the dome camera and the pose parameters of the dome camera pan-tilt corresponding thereto. Specifically, establish a relationship equation between the coordinate system of the dome camera and the pose parameters of the dome camera pan-tilt; based on the coordinates of the second key points in the coordinate system of the dome camera and the pose parameters of the dome camera pan-tilt corresponding thereto, use the least squares method to solve the equation coefficients in the relationship equation.

[0112] In this embodiment, the tracking conversion module M300 determines the pose parameters of the dome camera pan-tilt corresponding to the target tracking point based on the perspective transformation matrix and the coordinate conversion relationship, including: mapping the target tracking point to the coordinate system of the dome camera based on the perspective transformation matrix, and obtaining the coordinates of the target tracking point in the coordinate system of the dome camera; and determining the pose parameters of the dome camera pan-tilt corresponding to the coordinates of the target tracking point in the coordinate system of the dome camera according to the coordinate conversion relationship.

[0113] An embodiment of the present application further provides a pan-tilt-dome linkage control device, including a processor; a memory storing executable instructions of the processor; wherein, the processor is configured to execute the steps of the pan-tilt-dome linkage control method by executing the executable instructions.

[0114] Those skilled in the art of the present application can understand that various aspects of the present application can be implemented as a system, a method, or a program product. Therefore, various aspects of the present application can be specifically implemented in the following forms, namely: a complete hardware implementation manner, a complete software implementation manner (including firmware, microcode, etc.), or an implementation manner combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "platform" here.

[0115] The following refers to Figure 6Describe the electronic device 600 according to this embodiment of the present application. Figure 6 The shown electronic device 600 is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0116] As Figure 6 shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0117] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present application described in the above-mentioned part of the gun-ball linkage control method in this specification. For example, the processing unit 610 can execute the steps as Figure 1 shown in.

[0118] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only storage unit (ROM) 6203.

[0119] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205. Such program modules 6205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples.

[0120] The bus 630 may represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any bus structure in a variety of bus structures.

[0121] The electronic device 600 can also communicate with one or more external devices 700 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device (such as a router, a modem, etc.) that enables the electronic device 600 to communicate with one or more other computing devices. Such communication can be carried out through the input / output (I / O) interface 650. Moreover, the electronic device 600 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 660. The network adapter 660 can communicate with other modules of the electronic device 600 through the bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0122] In the gun-ball linkage control device, when the program in the memory is executed by the processor, the steps of the gun-ball linkage control method are implemented. Therefore, the device can also obtain the technical effects of the above gun-ball linkage control method.

[0123] The embodiment of the present application also provides a computer-readable storage medium for storing a program, and when the program is executed by the processor, the steps of the gun-ball linkage control method are implemented. In some possible implementation manners, various aspects of the present application can also be implemented in the form of a program product, which includes program code. When the program product is executed on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments described in the above gun-ball linkage control method part of this specification.

[0124] Referring Figure 7 As shown, a program product 800 for implementing the above method according to an embodiment of the present application is described. It can adopt a portable compact disc read-only memory (CD-ROM) and include program code, and can be executed on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto. In this document, a readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0125] The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0126] The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above.

[0127] The program code for performing the operations of this application may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by using an Internet service provider to connect through the Internet).

[0128] When the program in the computer storage medium is executed by a processor, the steps of the described gun-ball linkage control method are implemented. Therefore, the computer storage medium can also achieve the technical effects of the above gun-ball linkage control method.

[0129] The above content is a further detailed description of the present application in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present application is only limited to these descriptions. For those of ordinary skill in the technical field to which the present application pertains, without departing from the concept of the present application, several simple deductions or substitutions can be made, and all should be regarded as belonging to the protection scope of the present application.

Claims

1. A gun-ball linkage control method, characterized in that: The steps include: Establish the perspective transformation matrix between the coordinate system of the box camera and the coordinate system of the ball camera; Establishing a coordinate conversion relationship between the coordinate system of the ball camera and the position parameters of the ball camera pan / tilt; Determining a target tracking point in the shooting picture of the gun camera; Based on the perspective transformation matrix and the coordinate transformation relationship, determining the posture parameters of the ball camera pan / tilt corresponding to the target tracking point; The movement of the ball camera pan / tilt is controlled according to the position parameters of the ball camera pan / tilt.

2. The gun-ball linkage control method according to claim 1, characterized in that: Before establishing the perspective transformation matrix between the coordinate system of the gun camera and the coordinate system of the ball camera, the following steps are also included: The dome camera is controlled to rotate until the shooting picture of the dome camera overlaps with the shooting picture of the gun camera.

3. The gun-ball linkage control method according to claim 1, characterized in that: Establishing a perspective transformation matrix between the coordinate system of the gun camera and the coordinate system of the ball camera includes: A perspective transformation matrix between the coordinate system of the gun camera and the coordinate system of the dome camera is established according to the shooting picture of the dome camera and the shooting picture of the gun camera.

4. The gun-ball linkage control method according to claim 1, characterized in that: The step of establishing a perspective transformation matrix between the coordinate system of the gun camera and the coordinate system of the ball camera includes the following steps: Selecting a plurality of first key points in the shooting picture of the gun camera, and determining first coordinates of the plurality of first key points in the shooting picture of the gun camera; Determine corresponding position points of the plurality of first key points in the shooting picture of the ball camera, and determine second coordinates of the corresponding position points in the shooting picture of the ball camera; A perspective transformation matrix between the coordinate system of the gun camera and the coordinate system of the ball camera is established according to the first coordinate of the first key point and the second coordinate of the corresponding position point.

5. The gun-ball linkage control method according to claim 1, characterized in that: Establishing the coordinate conversion relationship between the coordinate system of the ball camera and the position parameters of the ball camera pan / tilt includes the following steps: Selecting a plurality of second key points in the image of the ball camera, and determining the coordinates of the second key points in the coordinate system of the ball camera; Record the posture parameters of the ball camera pan / tilt corresponding to each second key point respectively; According to the coordinates of the second key point in the coordinate system of the dome camera and the posture parameters corresponding to the dome head, a coordinate conversion relationship between the coordinate system of the dome camera and the posture parameters of the dome head is established.

6. The gun-ball linkage control method according to claim 5, characterized in that: Establishing the coordinate conversion relationship between the coordinate system of the ball camera and the position parameters of the ball camera pan / tilt includes the following steps: Establishing a relationship equation between the coordinate system of the ball camera and the position parameters of the ball camera pan / tilt; Based on the coordinates of the second key point in the coordinate system of the ball camera and the posture parameters corresponding to the ball camera pan / tilt, the least squares method is used to solve the equation coefficients in the relationship equation.

7. The gun-ball linkage control method according to claim 1, characterized in that: The posture parameters corresponding to the ball camera pan head include the horizontal rotation angle and the vertical rotation angle of the ball camera pan head.

8. The gun-ball linkage control method according to claim 1, characterized in that: Based on the perspective transformation matrix and the coordinate transformation relationship, determining the posture parameters of the ball camera pan / tilt corresponding to the target tracking point comprises the following steps: Based on the perspective transformation matrix, the target tracking point is mapped to the coordinate system of the ball camera to obtain the coordinates of the target tracking point in the coordinate system of the ball camera; According to the coordinate conversion relationship, the position parameters of the ball camera pan / tilt corresponding to the coordinates of the target tracking point in the coordinate system of the ball camera are determined.

9. A gun-ball linkage control system, characterized in that: The system for implementing the gun-ball linkage control method according to any one of claims 1 to 8 comprises: The perspective transformation module is used to establish the perspective transformation matrix between the coordinate system of the box camera and the coordinate system of the ball camera; A coordinate conversion module is used to establish a coordinate conversion relationship between the coordinate system of the ball camera and the position parameters of the ball camera pan / tilt; A tracking conversion module, used to determine a target tracking point in the shooting picture of the gun camera, and determine the ball camera pan / tilt posture parameters corresponding to the target tracking point based on the perspective transformation matrix and the coordinate transformation relationship; The linkage control module is used to control the movement of the ball camera pan / tilt according to the position parameters of the ball camera pan / tilt.

10. The gun-ball linkage control system according to claim 9, characterized in that: The linkage control module is also used to control the dome camera to rotate until the shooting picture of the dome camera overlaps with the shooting picture of the gun camera before establishing the perspective transformation matrix between the coordinate system of the gun camera and the coordinate system of the dome camera.

11. A gun-ball linkage control device, characterized in that: include: processor; a memory storing executable instructions of the processor; The processor is configured to execute the steps of the gun-ball linkage control method according to any one of claims 1 to 8 by executing the executable instructions.

12. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the gun-ball linkage control method according to any one of claims 1 to 8 are implemented.

Images