Pan / tilt control method, device, equipment and medium for monitoring camera center composition

Abstract

The present invention provides a method, device, equipment and medium for controlling a pan-tilt head of a surveillance camera with a central composition, including: obtaining a video frame image collected by a camera assembly for a target object; if it is detected that the target object is located in a non-central area in the video frame image, and / or the clarity of the target object presented in the video frame image does not meet a preset clarity condition, then the rotation angle of the pan-tilt head assembly and the focal length of the camera assembly are alternately iterated to obtain a target rotation angle and a target focal length; based on the target rotation angle and the target focal length, a target video frame image collected by the camera assembly for the target object is obtained; wherein the target object is located in a central area in the target video frame image, and the clarity of the target object in the target video frame image meets a preset clarity condition. The present invention has the characteristics of being fully automatic, requiring no calibration, and not relying on high-precision pan-tilt head rotation angles.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to a pan / tilt control method, device, equipment and medium for monitoring camera center composition. Background Art

[0002] With the continuous development of the intelligent industry, video surveillance is widely used in many occasions due to its intuitiveness, timeliness and rich information content, such as smart transportation, public safety, ecological red line monitoring, natural resource supervision, etc. According to whether it can be rotated, surveillance cameras are mainly divided into fixed-position gun cameras and PTZ (Pan-Tilt-Zoom) ball cameras that can rotate freely through the pan-tilt control. By setting the horizontal rotation angle P and the vertical rotation angle T, the camera can be remotely controlled to rotate left and right in the horizontal plane and pitch in the vertical plane with the pan-tilt, so as to obtain video images at any angle in the field of view. At the same time, by changing the focal length magnification Z, the global or detailed texture of the shooting target can be obtained. In order to facilitate the observation of the target itself and the surrounding environment, it is usually necessary to make the target located near the center of the video screen and clearly visible, and accurately identify the target in the final PTZ video screen, that is, the center composition.

[0003] The realization of the central composition of the PTZ camera depends on efficient pan-tilt control, and there are two main methods: ① Manual control of the pan-tilt. Low efficiency and high labor cost. ② Calibrate the camera. That is, calibrate the internal and external parameters of the PTZ camera in advance, and then use the internal and external parameters in the central composition to calculate the parameters for controlling the movement of the pan-tilt through two processes: converting the target's existing video coordinates to the world coordinates, and converting the world coordinates to the desired video frame center coordinates. This method relies on additional high-precision geographic reference data, and because the video image and the geographic reference data have large differences in shooting angle and texture, the control points required to solve the internal and external parameters need to be manually marked. At the same time, solving accurate internal and external parameters requires a large number of control points and they are evenly distributed, which is difficult to implement in practical applications. Moreover, it depends on the angle accuracy of the pan and tilt of the pan / tilt, and the accuracy is very poor when locating distant targets, and the central composition cannot be completed. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a pan-tilt control method, device, equipment and medium for monitoring camera central composition, which has the characteristics of being fully automatic, requiring no calibration and not relying on high-precision pan-tilt rotation angle.

[0005] In a first aspect, an embodiment of the present invention provides a method for controlling a pan / tilt of a surveillance camera with a central composition, wherein the surveillance camera includes a camera component and a pan / tilt component, and the method includes:

[0006] Acquire the video frame image captured by the camera component for the target object;

[0007] If it is detected that the target object is located in a non-central area of ​​the video frame image, and / or the clarity of the target object in the video frame image does not meet the preset clarity condition, the rotation angle of the pan / tilt assembly and the focal length of the camera assembly are alternately iterated to obtain the target rotation angle and the target focal length;

[0008] Based on the target rotation angle and the target focal length, a target video frame image captured by the camera component for the target object is obtained; wherein the target object is located in a central area of ​​the target video frame image, and the clarity of the target object in the target video frame image meets a preset clarity condition.

[0009] In one embodiment, if it is detected that the target object is located in a non-central area of ​​the video frame image, and / or the clarity of the target object in the video frame image does not meet the preset clarity condition, the rotation angle of the pan / tilt assembly and the focal length of the camera assembly are alternately iterated to obtain the target rotation angle and the target focal length, including:

[0010] If the target object is detected to be located in a non-central area of ​​the video frame image based on the centroid coordinates corresponding to the target object in the video frame image, a rotation angle of the gimbal assembly is determined based on the centroid coordinates, and the gimbal assembly is controlled to rotate according to the rotation angle to adjust the position and posture of the camera assembly;

[0011] Acquire a new video frame image captured by the adjusted camera assembly for the target object, and detect the contour coordinates corresponding to the target object in the new video frame image; wherein the target object is located in the central area of ​​the new video frame image;

[0012] If it is detected based on the contour coordinates that the clarity of the target object in the new video frame image does not meet the preset clarity condition, then the focal length of the camera assembly is adjusted;

[0013] Acquire a new video frame image captured by the camera assembly for the target object during the process of adjusting the focal length, and detect the contour coordinates corresponding to the target object in the new video frame image again;

[0014] If it is determined based on the contour coordinates that the target object is located at the edge of the new video frame image, the rotation angle of the pan-tilt assembly and the focal length of the camera assembly are again solved alternately and iteratively based on the contour coordinates until the target object is located in the central area of ​​the video frame image and the clarity of the target object in the video frame image meets the preset clarity conditions, thereby obtaining the target rotation angle of the pan-tilt assembly and the target focal length of the camera assembly.

[0015] In one embodiment, determining the rotation angle of the gimbal assembly based on the center of mass coordinates includes:

[0016] Determine the current center of mass homogeneous coordinates based on the center of mass coordinates corresponding to the target object in the video frame image, and determine the expected center of mass homogeneous coordinates based on the image principal point of the video frame image;

[0017] Determine the rotation matrix of the gimbal assembly according to the intrinsic parameter matrix of the camera assembly and its inverse matrix, the current center of mass homogeneous coordinates, and the expected center of mass homogeneous coordinates; wherein the rotation matrix includes a yaw angle and a pitch angle;

[0018] The yaw angle and pitch angle in the rotation matrix are iteratively solved, and the solved yaw angle and pitch angle are used as the rotation angle of the gimbal component.

[0019] In one implementation, iteratively solving the yaw angle and the pitch angle in the rotation matrix includes:

[0020] The change value of the pitch angle of the gimbal assembly is set to 0, and the yaw angle of the gimbal assembly is iterated based on the pitch angle, so that the coordinates of the center of mass after rotation are located in the horizontal center of the video frame image; the pitch angle is iterated based on the solved yaw angle, so that the coordinates of the center of mass after rotation are located in the vertical center of the video frame image;

[0021] Alternatively, the change value of the yaw angle of the gimbal assembly is set to 0, and the pitch angle of the gimbal assembly is iterated based on the yaw angle so that the coordinates of the center of mass after rotation are located in the vertical center of the video frame image; the yaw angle is iterated based on the solved pitch angle so that the coordinates of the center of mass after rotation are located in the horizontal center of the video frame image.

[0022] In one embodiment, iterating the yaw angle of the gimbal assembly based on the pitch angle includes:

[0023] Based on the relative position relationship between the center of mass coordinates and the image principal point, determine the rotation direction corresponding to the yaw angle of the gimbal assembly;

[0024] Determine the current iteration yaw angle based on the current iteration number, iteration step and rotation direction;

[0025] Determine a new rotation matrix based on the pitch angle and the current iteration yaw angle, and determine the corresponding rotated centroid coordinates of the target object in the video frame image based on the new rotation matrix;

[0026] If the difference between the coordinates of the center of mass after rotation and the horizontal coordinates of the image principal point is greater than or equal to the preset difference threshold, the next iterative yaw angle is determined based on the next iteration number, iteration step and rotation direction until the difference between the coordinates of the center of mass after rotation and the horizontal coordinates of the image principal point is less than the preset difference threshold. The iteration is stopped and the last iterative yaw angle is used as the yaw angle of the gimbal assembly.

[0027] In one implementation, detecting the contour coordinates corresponding to the target object in the new video frame image includes:

[0028] Extracting a new video frame image and a set of point pairs with the same name between the new video frame image and the previous video frame image corresponding to the new video frame image;

[0029] Determine the homography matrix based on the set of points with the same name;

[0030] The homography matrix is ​​used to convert the contour coordinates of the target object in the previous video frame image into the contour coordinates of the target object in the new video frame image.

[0031] In one implementation, if it is detected based on the contour coordinates that the clarity of the target object in the new video frame image does not meet the preset clarity condition, adjusting the focal length of the camera assembly includes:

[0032] Based on the contour coordinates, determine the number of pixels occupied by the target object in the new video frame image;

[0033] Based on the number of pixels and the total number of pixels contained in the new video frame image, determine the area ratio of the target object in the new video frame image;

[0034] If the area ratio is less than or equal to the preset ratio threshold, it is determined that the clarity of the target object in the new video frame image does not meet the preset clarity condition, and the focal length of the camera component is enlarged.

[0035] In one implementation, determining that the target object is located at an edge of a new video frame image based on the contour coordinates includes:

[0036] Generate an outer rectangle of the target object in a new video frame image based on the contour coordinates;

[0037] Determine the shortest distance between the edge of the bounding rectangle and the edge of the new video frame image;

[0038] If the closest distance is less than or equal to the preset distance threshold, it is determined that the target object is located at the edge of the new video frame image.

[0039] In a second aspect, an embodiment of the present invention further provides a pan-tilt control device for a surveillance camera center composition, wherein the surveillance camera includes a camera component and a pan-tilt component, and the device includes:

[0040] An image acquisition module is used to acquire a video frame image captured by the camera component for a target object;

[0041] A parameter iteration module, for alternately iterating and solving the rotation angle of the pan / tilt assembly and the focal length of the camera assembly to obtain a target rotation angle and a target focal length if it is detected that the target object is located in a non-central area of ​​the video frame image and / or the clarity of the target object presented in the video frame image does not meet a preset clarity condition;

[0042] The image acquisition module is also used to acquire the target video frame image collected by the camera component for the target object based on the target rotation angle and the target focal length; wherein the target object is located in the central area of ​​the target video frame image, and the clarity of the target object in the target video frame image meets the preset clarity condition.

[0043] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement any one of the methods provided in the first aspect.

[0044] An embodiment of the present invention provides a pan-tilt control method, device, equipment and medium for a surveillance camera with a central composition. The surveillance camera includes a camera component and a pan-tilt component. First, a video frame image captured by the camera component for a target object is obtained; if it is detected that the target object is located in a non-central area in the video frame image, and / or the clarity of the target object presented in the video frame image does not meet a preset clarity condition, the rotation angle of the pan-tilt component and the focal length of the camera component are alternately iterated to obtain a target rotation angle and a target focal length; finally, based on the target rotation angle and the target focal length, a target video frame image captured by the camera component for the target object is obtained, the target object is located in the central area of ​​the target video frame image, and the clarity of the target object in the target video frame image meets the preset clarity condition. The above method determines whether the target object is located in the central area of ​​the video frame image and whether the clarity of the target object in the video frame image meets the preset clarity conditions. When the determination result is no, it will trigger the alternating iterative solution of the rotation angle of the pan-tilt assembly and the focal length of the camera assembly to obtain the target rotation angle and target focal length that can achieve central composition. The embodiment of the present invention has the characteristics of being fully automatic, requiring no calibration, and not relying on high-precision pan-tilt rotation angles.

[0045] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic flow chart of a pan / tilt control method for a surveillance camera center composition provided by an embodiment of the present invention;

[0049] Figure 2 A schematic flow chart of another method for controlling a pan / tilt of a surveillance camera center composition provided by an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of a process of controlling a pan / tilt assembly provided by an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of a target object moving in a video frame image provided by an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of a process for adjusting the focal length of a camera assembly provided by an embodiment of the present invention;

[0053] Figure 6 A schematic diagram of a process for accurately locating a target provided by an embodiment of the present invention;

[0054] Figure 7 A schematic diagram of an initial video frame image provided by an embodiment of the present invention;

[0055] Figure 8 A schematic diagram of a video frame image after the pan / tilt head is rotated for the first time provided by an embodiment of the present invention;

[0056] Fig. 9 A schematic diagram of a video frame image after a first zoom provided by an embodiment of the present invention;

[0057] Fig.10 A schematic diagram of a video frame image after a second zoom provided by an embodiment of the present invention;

[0058] Fig.11 A schematic diagram of a video frame image after a third zoom provided by an embodiment of the present invention;

[0059] Fig.12 A schematic diagram of a video frame image after a fourth zoom provided by an embodiment of the present invention;

[0060] Fig.13 A schematic diagram of a video frame image after the pan / tilt head is rotated for the second time provided by an embodiment of the present invention;

[0061] Fig.14 A schematic diagram of a video frame image after N-th zooming provided by an embodiment of the present invention;

[0062] Fig.15 A schematic diagram of the structure of a pan / tilt control device for a surveillance camera center composition provided by an embodiment of the present invention;

[0063] Fig.16 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] At present, the manual control method has the problem of low efficiency, and the camera calibration method has the problems of difficulty in obtaining high-precision control points, complex calibration algorithms, reliance on high-precision angle pan-tilt heads, high costs, etc. Based on this, the present invention implements a pan-tilt control method, device, equipment and medium for monitoring camera central composition.

[0066] To facilitate understanding of this embodiment, a pan-tilt control method for a surveillance camera center composition disclosed in an embodiment of the present invention is first described in detail. The surveillance camera includes a camera component and a pan-tilt component. Figure 1 The flowchart of a method for controlling a pan / tilt of a surveillance camera center composition is shown in FIG. 1 , and the method mainly includes the following steps S102 to S106:

[0067] Step S102, obtaining a video frame image captured by the camera component for the target object.

[0068] Step S104, if it is detected that the target object is located in a non-central area in the video frame image, and / or the clarity of the target object in the video frame image does not meet the preset clarity condition, the rotation angle of the gimbal assembly and the focal length of the camera assembly are alternately iterated to obtain the target rotation angle and the target focal length.

[0069] The central area is the center of the frame, and the clarity can be reflected by the area ratio of the target object in the video frame image. If the area ratio is greater than the preset ratio threshold, it can be considered that the clarity of the target object in the video frame image meets the preset clarity condition. In one example, first, the video frame image and the position of the target object in the video frame image are input, referred to as the target position. Second, determine whether the target object is located in the center of the frame and is clear. Third, if the target object is located in the center of the frame, the gimbal does not need to make any adjustments; if the target object is not located in the center of the frame, rotate the gimbal assembly so that the target object is located in the center of the frame. Fourth, if the target object is not clear, it is necessary to zoom in to make the target clearer and located in the picture. The second to fourth steps are executed in this way until the target is located in the center of the frame and is clearly visible. Among them, the process of looping the second to fourth steps is the process of alternately iteratively solving the rotation angle of the gimbal assembly and the focal length of the camera assembly.

[0070] Since an uncalibrated camera is used, the rotation angle estimated by the camera intrinsic parameters can only make the target object located near the center of the frame. Therefore, it is necessary to determine the precise contour position of the target object in the video frame image after the pan-tilt rotation based on the two video frame images before and after the pan-tilt rotation. In addition, since the camera intrinsic parameters are inaccurate, it is also impossible to accurately calculate the contour position of the target object in the video frame image after zooming. Therefore, it is necessary to determine the precise contour position of the target object in the video frame image after zooming based on the two video frame images before and after zooming.

[0071] Step S106, based on the target rotation angle and the target focal length, a target video frame image captured by the camera assembly for the target object is obtained, wherein the target object is located in the central area of ​​the target video frame image, and the clarity of the target object in the target video frame image meets a preset clarity condition.

[0072] The pan-tilt control method for the central composition of a surveillance camera provided in an embodiment of the present invention determines whether the target object is located in the central area of ​​a video frame image and whether the clarity of the target object presented in the video frame image meets a preset clarity condition. When the determination result is no, it triggers an alternating iterative solution of the rotation angle of the pan-tilt assembly and the focal length of the camera assembly to obtain a target rotation angle and a target focal length that can achieve the central composition. The embodiment of the present invention has the characteristics of being fully automatic, requiring no calibration, and not relying on a high-precision pan-tilt rotation angle.

[0073] For ease of understanding, an embodiment of the present invention provides a specific implementation of a pan / tilt control method for a monitoring camera center composition.

[0074] First, get the data, including:

[0075] (a) Camera internal parameters:

[0076] (a1) Image width W and height H. Unit: pixel.

[0077] (a2) Image principal point C: , = , which is consistent with the center of the image.

[0078] (a3) Focal length f. Unit: pixel.

[0079] Get the 1x focal length from the manufacturer or official website based on the camera model , the current frame zoom ratio is recorded as Zoom, then:

[0080] (1).

[0081] (b) Pan / tilt angle, including Pan and Tilt.

[0082] (c) Video frame image .

[0083] (d) Target object in video frame image The corresponding centroid coordinates in .

[0084] The target object can be a polygon, line or point. Its contour point set is represented by Contour. The centroid coordinates of the contour are represented by g( , )express.

[0085] Then, see Figure 2 The flowchart of another method for controlling the pan / tilt of a surveillance camera center composition is shown, and the following steps S1 to S4 are performed for the input video frame image and the centroid position of the target object in the video frame image:

[0086] Step S1, determine whether the target object is located in the center of the frame. If yes, go to step S2; if not, go to step S3.

[0087] Step S2, judging whether the clarity of the target object in the video frame image meets the preset clarity condition. If yes, then end; if not, proceed to step S4.

[0088] Step S3, controlling the pan / tilt assembly to rotate.

[0089] Step S4, adjust the focal length of the camera assembly, input a new video frame image, detect the contour position of the target object in the video frame image, and continue to execute step S1. The contour position of the target object in the video frame image can be detected according to step S5, and step S5 includes: based on the two video frame images before and after the pan / tilt rotation, determine the precise contour position of the target object in the video frame image after the pan / tilt rotation; and based on the two video frame images before and after zooming, determine the precise contour position of the target object in the video frame image after zooming.

[0090] In practical applications, the process of repeatedly executing step S1 to step S4 is a process of alternately iteratively solving the rotation angle of the pan / tilt assembly and the focal length of the camera assembly. In a specific implementation, see (i) to (v) below:

[0091] (i) If the target object is detected to be located in a non-central area of ​​the video frame image based on the centroid coordinates corresponding to the target object in the video frame image, the rotation angle of the pan-tilt assembly is determined based on the centroid coordinates, and the pan-tilt assembly is controlled to rotate according to the rotation angle to adjust the position and posture of the camera assembly. That is, when the judgment result of step S1 is no, step S3 is executed to make the target object located in the center of the frame of the video frame image (i.e., the new video frame image) collected after the pan-tilt assembly rotates.

[0092] (ii) obtaining a new video frame image captured by the adjusted camera assembly for the target object, and detecting the contour coordinates corresponding to the target object in the new video frame image. That is, after the pan / tilt assembly is controlled so that the target object is located in the center of the frame of the new video frame image, the contour coordinates corresponding to the target object in the new video frame image are obtained according to step S5.

[0093] (III) If it is detected based on the contour coordinates that the clarity of the target object in the new video frame image does not meet the preset clarity condition, the focal length of the camera assembly is adjusted. That is, when the judgment result of step S2 is no, step S4 is executed to make the clarity of the target object in the new video frame image meet the preset clarity condition.

[0094] (iv) obtaining a new video frame image captured by the camera component for the target object during the process of adjusting the focal length, and detecting the corresponding contour coordinates of the target object in the new video frame image again. That is, after each adjustment of the focal length of the camera component, the camera component is controlled to re-capture a new video frame image, and the corresponding contour coordinates of the target object in the new video frame image are obtained according to step S5.

[0095] (V) If the target object is determined to be located at the edge of the new video frame image based on the contour coordinates, the rotation angle of the pan-tilt assembly and the focal length of the camera assembly are alternately iterated and solved again based on the contour coordinates until the target object is located in the central area of ​​the video frame image and the clarity of the target object in the video frame image meets the preset clarity condition, thereby obtaining the target rotation angle of the pan-tilt assembly and the target focal length of the camera assembly. That is, in the process of executing step S4, a new video frame image collected by the camera assembly for the target object is obtained after each adjustment of the focal length; the contour coordinates corresponding to the target object in the new video frame image are obtained according to step S5; if the target object is determined to be located at the edge of the new video frame image based on the contour coordinates, steps S1 and S3 are triggered until the target object is located in the center of the frame of the new video frame image; after manipulating the pan-tilt assembly, the new video frame image collected by the camera assembly for the target object is continued to be obtained; the contour coordinates corresponding to the target object in the new video frame image are continued to be obtained according to step S5; and steps S2 and S4 are triggered. This cycle is repeated until the target object is located in the center of the frame of the video frame image and the target object is clear.

[0096] For ease of understanding, the embodiment of the present invention provides specific implementations of steps S1 to S5.

[0097] In one example, step S1 is equivalent to determining whether the target object is located in the central area of ​​the video frame image based on the centroid coordinates corresponding to the target object in the video frame image, and specifically includes steps S1.1 to S1.2:

[0098] Step S1.1: Calculate the distance d between the target mass center g and the center C of the frame. The formula is as follows:

[0099] (2);

[0100] Step S1.2, compare with the preset threshold:

[0101] Setting Thresholds (Unit: pixel). When d is less than or equal to When the target object is located in the center of the frame, the threshold value is determined according to the needs. The smaller the threshold value is, the higher the accuracy is, but the gimbal assembly needs to be rotated frequently.

[0102] In one example, step S2 is equivalent to determining whether the clarity of the target object in the new video frame image meets the preset clarity condition based on the contour coordinates. Whether the target object is clear is related to the size of the target object in the video frame image, so it can be determined based on the area percentage Area_percent of the target object in the video frame image. Specifically, it includes the following steps S2.1 to S2.3:

[0103] Step S2.1: Based on the contour coordinates, determine the number of pixels occupied by the target object in the new video frame image.

[0104] Step S2.2: Based on the number of pixels and the total number of pixels contained in the new video frame image, determine the area ratio of the target object in the new video frame image. The formula is as follows:

[0105] (3);

[0106] Among them, count represents the number of pixels of the target in the image.

[0107] Step S2.3: If the area ratio is less than or equal to the preset ratio threshold, it is determined that the clarity of the target object in the new video frame image does not meet the preset clarity condition, and the focal length of the camera component is enlarged. For example, the threshold is set (Value range: 0-1). When Area_percent is greater than , the target is determined to be clear.

[0108] In one example, step S3 is equivalent to determining the rotation angle of the pan-tilt assembly based on the center of mass coordinates, and controlling the pan-tilt assembly to rotate according to the rotation angle. Figure 3 The schematic diagram of a process of controlling a pan-tilt assembly shown in the figure specifically includes the following steps S3.1 to S3.4:

[0109] Step S3.1, inputting the data before the pan / tilt rotation, including:

[0110] (a) The contour coordinates of the target object before rotation, denoted as , whose centroid is .

[0111] (b) The camera intrinsic parameter matrix K is expressed as follows:

[0112] (4)

[0113] The focal length f is calculated using the current zoom factor according to formula (1).

[0114] (c) Video frame image before rotation .

[0115] Step S3.2, calculating the rotation angle of the gimbal assembly.

[0116] The basic principle is as follows: The PTZ surveillance camera only rotates, without translation, and the position relationship of the same-name points before and after the movement satisfies the following formula:

[0117] (5)

[0118] Among them, m and Respectively represent the image homogeneous coordinates of the same-name points before and after the camera is rotated, and represents the camera intrinsic matrix and its inverse matrix, and R represents the rotation matrix calculated by the gimbal rotation angle. The calculation formula of R is as follows:

[0119] (6)

[0120] in, They represent the roll angle, yaw angle, and pitch angle of the camera's rotation around the z-axis, y-axis, and x-axis respectively. The corresponding rotation matrices are as follows:

[0121] ;

[0122] ;

[0123] ;

[0124] Since the PTZ camera is fixed on the pan / tilt, it can only rotate horizontally and vertically, but cannot roll. , the degrees of freedom of R is 2.

[0125] Based on this principle and The solution specifically includes the following steps S3.2.1 to S3.2.2:

[0126] Step S3.2.1, determine the current center of mass homogeneous coordinates based on the center of mass coordinates corresponding to the target object in the video frame image, and determine the expected center of mass homogeneous coordinates based on the image principal point of the video frame image, and determine the rotation matrix of the gimbal assembly based on the intrinsic parameter matrix of the camera assembly and its inverse matrix, the current center of mass homogeneous coordinates, and the expected center of mass homogeneous coordinates.

[0127] Specifically, the current center of mass homogeneous coordinate m of the target object before the pan / tilt assembly rotates is given by Get the expected center of mass homogeneous coordinates after rotation Obtained from the image principal point C, they are all known numbers. In addition, and Also known, the rotation matrix R can be solved according to formula (5). At the same time, since the PTZ camera rotation matrix R has 2 degrees of freedom, two equations can be listed for each pair of centroids using formula (5), so There is a unique solution.

[0128] Step S3.2.3, iteratively solve the yaw angle and pitch angle in the rotation matrix, and use the solved yaw angle and pitch angle as the rotation angle of the gimbal assembly.

[0129] Example 1: Set the change value of the pitch angle of the gimbal component to 0, iterate the yaw angle of the gimbal component based on the pitch angle, so that the coordinates of the center of mass after rotation are located in the horizontal center of the video frame image; iterate the pitch angle based on the solved yaw angle, so that the coordinates of the center of mass after rotation are located in the vertical center of the video frame image. Specifically: first iterate in the horizontal direction, that is, set =0, iterative calculation , so that the coordinates of the center of mass after rotation are located in the center of the frame in the x direction; then use the calculated Iterate in the vertical direction so that the coordinates of the rotated center of mass are also located in the center of the frame in the y direction. Figure 4 The schematic diagram of a target object moving in a video frame image is shown in FIG. The black dot is the position of the centroid coordinates of the target object in the video frame image before rotation, which is obtained after iteration in the x direction and the y direction. , the pan / tilt assembly can be controlled to rotate once, so that the center of mass coordinates of the target object after rotation is located at the white dot, that is, the center of the frame.

[0130] Example 2: Set the change value of the yaw angle of the gimbal assembly to 0, iterate the pitch angle of the gimbal assembly based on the yaw angle, so that the coordinates of the center of mass after rotation are located at the center of the vertical direction of the video frame image; iterate the yaw angle based on the pitch angle obtained, so that the coordinates of the center of mass after rotation are located at the center of the horizontal direction of the video frame image. For details, please refer to Example 1, which will not be described in detail in the embodiment of the present invention.

[0131] Furthermore, the embodiment of the present invention uses Example 1 as an example to explain the iterative solution process. In addition, since the iterative calculation process in the horizontal direction is the same as that in the vertical direction, the iterative calculation process is only explained using the horizontal direction as an example, including the following ① to ⑤:

[0132] ① Initialization. Settings The initial value is 0, and the value range is [-π,π]. The iteration step can be set according to the rotation angle accuracy of the PTZ component. For common PTZ cameras, it can be set to 0.1 degrees.

[0133] The homogeneous coordinate vector corresponding to the center of mass coordinate g of the target object before rotation is as follows:

[0134] .

[0135] ②Confirm The positive and negative signs. That is, based on the relative position relationship between the center of mass coordinates and the image principal point, determine the rotation direction corresponding to the yaw angle of the gimbal component. The rotation direction is used to describe whether the gimbal component turns left or right, thereby reducing the iteration interval and increasing the robustness of the algorithm. The method is as follows:

[0136] The center of mass coordinates of the target object before rotation The x-coordinate of The x direction of the image principal point C Compare. > , that is, the target object is located on the right side of the current video frame image, and the PTZ component turns right. Take the negative sign; otherwise, Take the positive sign. The positive and negative sign operations are represented by the numerical flag, when the sign is positive, flag = 1, and when the sign is negative, flag = -1.

[0137] ③ Based on the current iteration number, iteration step and rotation direction, determine the current iteration yaw angle. The calculation formula is as follows:

[0138] ;

[0139] ④ Determine a new rotation matrix based on the pitch angle and the current iteration yaw angle, and based on the new rotation matrix, determine the corresponding rotated center of mass coordinates of the target object in the video frame image.

[0140] Specifically, the new rotation matrix R is calculated according to the above formula (6), and the iterated homogeneous coordinate vector is calculated using formula (5): Then use the third row to normalize the first two rows to get the coordinates of the center of mass after rotation. .

[0141] ⑤ If the difference between the coordinates of the center of mass after rotation and the horizontal coordinates of the image principal point is greater than or equal to the preset difference threshold, continue to determine the next iterative yaw angle based on the next iteration number, iteration step and rotation direction until the difference between the coordinates of the center of mass after rotation and the horizontal coordinates of the image principal point is less than the preset difference threshold, stop the iteration, and use the last iterative yaw angle as the yaw angle of the gimbal assembly.

[0142] Specifically, if the centroid coordinates after rotation and the horizontal coordinates of the principal point If the absolute value of the difference between is less than the preset threshold ω, the iteration is stopped. Otherwise, go to ③ and continue the iteration.

[0143] Step S3.3, rotating the pan-tilt assembly: When controlling the pan-tilt assembly to rotate, it is necessary to send an absolute position command, that is, an absolute angle, rather than the angle before and after the rotation. It is a changing angle, so a conversion is required, the formula is as follows:

[0144] ;

[0145] The subscripts f and b represent before and after the rotation, respectively. and Indicates the positive or negative sign, which is determined by whether the angle increases or decreases when the PTZ camera rotates, as follows:

[0146] When the PTZ camera rotates clockwise, the Pan angle increases. = -1; otherwise, = 1.

[0147] When the PTZ camera adjusts the pitch angle, the Tilt increases as it moves downward. = 1; otherwise, = -1.

[0148] Get a new video frame image after controlling the rotating pan-tilt assembly , that is, the rotated video frame image.

[0149] Step S3.4, calculate the contour coordinates of the rotated target object: Input the image before and after rotation and the contour coordinates of the target object before rotation , use step S5 to calculate the contour coordinates of the rotated target object .

[0150] In one example, step S4 is equivalent to adjusting the focal length of the camera assembly, and in the process of adjusting the focal length, a new video frame image is collected for the target object, and the contour coordinates of the target object in the new video frame image are detected, and the target object is determined to be located at the edge of the new video frame image based on the contour coordinates. Considering that the rotation of the remote-controlled gimbal is significantly slower than the camera zoom, it is necessary to reduce the number of rotations of the gimbal, rather than immediately executing step S1 every time the focal length is adjusted, thereby triggering step S3. Design a zoom iteration algorithm, zoom in continuously for multiple times, and stop the iteration until the target is close to the edge of the frame or clearly visible, and then trigger step S1 and step S3.

[0151] After each iteration, first determine whether the contour coordinates of the target object are close to the edge of the video frame image. If so, stop the iteration. If not, use the method in step S2 to further determine whether the target object is clear. If yes, stop the iteration; if not, enter the next iteration. The following is a detailed description of each iteration process. See Figure 5 The flowchart of adjusting the focal length of a camera assembly shown in the figure specifically includes the following steps S4.1 to S4.5:

[0152] Step S4.1, inputting data for the i-th iteration, includes:

[0153] The contour coordinates of the target object after the i-1th zoom are recorded as the input of the i-th iteration as .

[0154] And, the video frame image after the i-1th zoom is recorded as the input of the i-th iteration as .

[0155] Step S4.2, zooming in, includes:

[0156] Zoom cameras have a certain maximum zoom ratio, such as 32x zoom and 40x zoom. That is, the maximum zoom ratio is known and is recorded as max_zoom, and the minimum zoom ratio is 1. Its value is divided into integer zoom and floating-point zoom according to different camera manufacturers and models.

[0157] Integer zoom, the zoom ratio is discrete, such as 1x, 2x, etc. All zoom ratio values ​​greater than the current zoom ratio cur_zoom are stored in the array candidate_zoom, and the candidate focal length ratio of each iteration is candidate_zoom[i].

[0158] Floating point zoom, the zoom ratio is continuous. The preset zoom step is step_zoom, and the candidate focal length ratio iter_zoom_i for the i-th iteration can be calculated using the following formula:

[0159]

[0160] Get a new video frame image after zooming , that is, the zoomed video frame image.

[0161] Step S4.3, calculate the contour coordinates of the zoomed target object:

[0162] Input the image before and after zoom and the contour coordinates of the target object before zooming , use step S5 to calculate the contour coordinates of the zoomed target object .

[0163] Step S4.4, determining whether the target object is located at the edge of the new video frame image (i.e. the zoomed video frame image), includes the following steps S4.4.1 to S4.4.3:

[0164] Step S4.4.1, generating the bounding rectangle of the target object in the new video frame image based on the contour coordinates, that is, the contour coordinates of the zoomed target object The bounding rectangle of .

[0165] Step S4.4.2, determine the shortest distance between the edge of the bounding rectangle and the edge of the new video frame image, that is, calculate A rectangle with four sides and the video image frame The closest distance between the four edges .in The coordinates of the upper left vertex are (0,0) and the coordinates of the lower right vertex are (W,H).

[0166] Step S4.4.3, if the closest distance is less than or equal to the preset distance threshold, it is determined that the target object is located at the edge of the new video frame image, that is, With preset threshold Compare: When <= , it is judged to be close to the edge of the frame.

[0167] Step S4.5, target clarity judgment: input the contour coordinates of the zoomed target object , use step S2 to make a judgment.

[0168] In one example, step S5 is equivalent to detecting the corresponding contour coordinates of the target object in the new video frame image. Figure 6 A schematic diagram of a process for accurately locating a target is shown, and specifically see the following steps S5.1 to S5.3:

[0169] Step S5.1, extract the set of points with the same name between the new video frame image and the previous video frame image corresponding to the new video frame image. That is, input two images before and after the pan / tilt rotation or zoom and , output the same-name point pair dataset and The specific process is as follows:

[0170] Extract images separately and The feature point sets A and B are as follows. The feature extraction operators using scale-invariant and rotation-invariant consistency include but are not limited to SIFT, Surf, Orb, Akaze, SuperPoint, etc.

[0171] Match point sets A and B to obtain initial matching point sets C and D. Matching search methods include but are not limited to Kd tree, Hash histogram, SuperGlue, LightGlue, etc.

[0172] Eliminate the wrong points and get the final set of points with the same name and . The method of estimating the homography matrix H based on random sampling consistency is used to remove erroneous points.

[0173] Step S5.2, determining the homography matrix based on the set of same-name point pairs. Specifically, it includes: using the same-name point pair dataset and , the homography matrix H is estimated using a method based on random sampling consistency.

[0174] Step S5.3, calculating the contour coordinates of the target object, specifically includes: using the homography matrix to convert the contour coordinates corresponding to the target object in the previous video frame image into the contour coordinates corresponding to the target object in the new video frame image.

[0175] In practical applications, the contour coordinates of the target object are two images before and after the pan / tilt rotates or zooms. and In the equation, the corresponding points are transformed into each other through the homography matrix H, as follows:

[0176] (7)

[0177] Where m and are the homogeneous coordinates before and after the transformation. Each point in the contour before the pan / tilt rotation or zooming constructs the homogeneous coordinate m, and the homogeneous coordinate after the pan / tilt rotation or zooming can be calculated using formula (7): , thereby obtaining the contour coordinates of the target object.

[0178] In summary, the embodiments of the present invention are applicable to any PTZ surveillance camera. They only require the initial focal length provided by the camera manufacturer, do not rely on geographic reference data and do not require human intervention, so as to achieve the purpose of automatic intelligent monitoring. They have low data requirements and simple calculations, and can perform central composition shooting intelligently, quickly and at low cost, providing technical support for video surveillance and monitoring in the fields of smart transportation, security, ecological red lines and natural resources.

[0179] Furthermore, the embodiment of the present invention also provides an application example of a PTZ control method for monitoring camera center composition. A PTZ camera on a tower is selected as an example to obtain an initial video frame image, such as Figure 7 The schematic diagram of an initial video frame image is shown in FIG. The diagonal filled area is used to represent the background in the video frame image. The lower left corner of the image is the current PTZ parameters, Pan = 264.4, Tilt = 17.2, Zoom = 1. The target is a house in the distance, as shown by the black outline. The house is located at the edge of the image and is not clear. The center composition method is used to automatically shoot, and finally the house is located in the center of the image and is clear.

[0180] (1) Execute step S1 for the first time; if the judgment result is negative, execute step S3.

[0181] (2) Execute step S3 for the first time:

[0182] First calculate the pan / tilt angle Pan = 270.4, Tilt = 5.2. Then rotate the pan / tilt. Finally execute S5 to get the exact position of the house. Figure 8Schematic diagram of a video frame image after the pan / tilt is rotated for the first time. Figure 8 (a) and (b) in the figure respectively illustrate a house where the pan / tilt head performs S5 front and rear positioning. Figure 8 (a) locates the house outline coordinates to the trees, while Figure 8 (b) The house outline coordinates are located more accurately.

[0183] (3) Execute step S2 for the first time; if the result is negative, execute S4.

[0184] (4) Execute step S4 for the first time:

[0185] The current zoom ratio is 1, and the iterative zoom step is set to 1.

[0186] Iteration 1: Zoom ratio 2. After zooming, the result of step S5 is as follows: Fig. 9 As shown, that is Fig. 9 Execute step S4.4, the result is no, further execute step S2, the result is no, execute the second iteration.

[0187] Iteration 2: Zoom ratio 3. After zooming, the result of step S5 is as follows: Fig.10 , that is Fig.10 Execute step S4.4, the result is no, further execute step S2, the result is no, execute the third iteration.

[0188] Iteration 3: Zoom ratio 4. After zooming, the result of step S5 is as follows: Fig.11 , that is Fig.11 Execute step S4.4, the result is no, further execute step S2, the result is no, execute the fourth iteration.

[0189] Iteration 4: Zoom ratio 5. After zooming, the result of step S5 is as follows: Fig.12 , that is Fig.12 Execute step S4.4, and if the result is yes, the iteration ends.

[0190] (5) Execute step S1 for the second time; if the judgment result is negative, execute step S3.

[0191] (6) Execute step S3 for the second time:

[0192] First calculate the pan / tilt angle Pan = 270.3, Tilt = 2.9. Then rotate the pan / tilt. Finally execute S5 to get the exact position of the house. Fig.13A schematic diagram of a video frame image after the pan / tilt head is rotated for the second time is shown.

[0193] (7) Execute step S2 for the second time; if the result is negative, execute step S4.

[0194] (8) Execute step S4 for the second time:

[0195] The current zoom ratio is 5. After 3 iterations, that is, when the zoom ratio is 8, the result of step S2 is yes, and the iteration is exited. The result is as follows Fig.14 As shown, that is Fig.14 is a schematic diagram of a video frame image after the Nth zoom.

[0196] (9) Execute step S1 for the third time. If the result is yes, execute step S2.

[0197] (10) Execute step S2 for the third time; if the result is yes, exit the entire program.

[0198] Finally Fig.14 In the result shown, the house is located near the center of the frame and is clearly visible.

[0199] During the entire algorithm process, the gimbal was automatically controlled to rotate 2 times, and finally moved to Pan = 270.3, Tilt = 2.9, and the camera was automatically controlled to zoom 7 times to Zoom = 8.

[0200] Based on the above-mentioned embodiment, the embodiment of the present invention provides a pan-tilt control device for a monitoring camera center composition, wherein the monitoring camera includes a camera component and a pan-tilt component. Fig.15 The schematic diagram of the structure of a pan-tilt control device for a surveillance camera center composition is shown, and the device mainly includes the following parts:

[0201] The first image acquisition module 1502 is used to acquire a video frame image captured by the camera assembly for the target object;

[0202] The parameter iteration module 1504 is used to alternately iterate and solve the rotation angle of the pan / tilt assembly and the focal length of the camera assembly to obtain a target rotation angle and a target focal length if it is detected that the target object is located in a non-central area of ​​the video frame image and / or the clarity of the target object presented in the video frame image does not meet a preset clarity condition;

[0203] The second image acquisition module 1506 is also used to acquire a target video frame image captured by the camera component for the target object based on the target rotation angle and the target focal length; wherein the target object is located in the central area of ​​the target video frame image, and the clarity of the target object in the target video frame image meets a preset clarity condition.

[0204] The pan-tilt control device for the central composition of a surveillance camera provided in an embodiment of the present invention determines whether a target object is located in the central area of ​​a video frame image and whether the clarity of the target object presented in the video frame image meets a preset clarity condition. When the determination result is no, it triggers alternating iterative solutions of the rotation angle of the pan-tilt assembly and the focal length of the camera assembly to obtain a target rotation angle and a target focal length that can achieve the central composition. The embodiment of the present invention has the characteristics of being fully automatic, requiring no calibration, and not relying on a high-precision pan-tilt rotation angle.

[0205] In one implementation, the parameter iteration module 1504 is specifically configured to:

[0206] If the target object is detected to be located in a non-central area of ​​the video frame image based on the centroid coordinates corresponding to the target object in the video frame image, a rotation angle of the gimbal assembly is determined based on the centroid coordinates, and the gimbal assembly is controlled to rotate according to the rotation angle to adjust the position and posture of the camera assembly;

[0207] Acquire a new video frame image captured by the adjusted camera assembly for the target object, and detect the contour coordinates corresponding to the target object in the new video frame image; wherein the target object is located in the central area of ​​the new video frame image;

[0208] If it is detected based on the contour coordinates that the clarity of the target object in the new video frame image does not meet the preset clarity condition, then the focal length of the camera assembly is adjusted;

[0209] Acquire a new video frame image captured by the camera component for the target object during the process of adjusting the focal length, and detect the contour coordinates corresponding to the target object in the new video frame image again;

[0210] If it is determined based on the contour coordinates that the target object is located at the edge of the new video frame image, the rotation angle of the pan-tilt assembly and the focal length of the camera assembly are again solved alternately and iteratively based on the contour coordinates until the target object is located in the central area of ​​the video frame image and the clarity of the target object in the video frame image meets the preset clarity conditions, thereby obtaining the target rotation angle of the pan-tilt assembly and the target focal length of the camera assembly.

[0211] In one implementation, the parameter iteration module 1504 is specifically configured to:

[0212] Determine the current center of mass homogeneous coordinates based on the center of mass coordinates corresponding to the target object in the video frame image, and determine the expected center of mass homogeneous coordinates based on the image principal point of the video frame image;

[0213] Determine the rotation matrix of the gimbal assembly according to the intrinsic parameter matrix of the camera assembly and its inverse matrix, the current center of mass homogeneous coordinates, and the expected center of mass homogeneous coordinates; wherein the rotation matrix includes a yaw angle and a pitch angle;

[0214] The yaw angle and pitch angle in the rotation matrix are iteratively solved, and the solved yaw angle and pitch angle are used as the rotation angle of the gimbal component.

[0215] In one implementation, the parameter iteration module 1504 is specifically configured to:

[0216] The change value of the pitch angle of the gimbal assembly is set to 0, and the yaw angle of the gimbal assembly is iterated based on the pitch angle, so that the coordinates of the center of mass after rotation are located in the horizontal center of the video frame image; the pitch angle is iterated based on the solved yaw angle, so that the coordinates of the center of mass after rotation are located in the vertical center of the video frame image;

[0217] Alternatively, the change value of the yaw angle of the gimbal assembly is set to 0, and the pitch angle of the gimbal assembly is iterated based on the yaw angle so that the coordinates of the center of mass after rotation are located in the vertical center of the video frame image; the yaw angle is iterated based on the solved pitch angle so that the coordinates of the center of mass after rotation are located in the horizontal center of the video frame image.

[0218] In one implementation, the parameter iteration module 1504 is specifically configured to:

[0219] Based on the relative position relationship between the center of mass coordinates and the image principal point, determine the rotation direction corresponding to the yaw angle of the gimbal assembly;

[0220] Determine the current iteration yaw angle based on the current iteration number, iteration step and rotation direction;

[0221] Determine a new rotation matrix based on the pitch angle and the current iteration yaw angle, and determine the corresponding rotated centroid coordinates of the target object in the video frame image based on the new rotation matrix;

[0222] If the difference between the coordinates of the center of mass after rotation and the horizontal coordinates of the image principal point is greater than or equal to the preset difference threshold, the next iterative yaw angle is determined based on the next iteration number, iteration step and rotation direction until the difference between the coordinates of the center of mass after rotation and the horizontal coordinates of the image principal point is less than the preset difference threshold. The iteration is stopped and the last iterative yaw angle is used as the yaw angle of the gimbal assembly.

[0223] In one implementation, the parameter iteration module 1504 is specifically configured to:

[0224] Extracting a new video frame image and a set of point pairs with the same name between the new video frame image and the previous video frame image corresponding to the new video frame image;

[0225] Determine the homography matrix based on the set of points with the same name;

[0226] The homography matrix is ​​used to convert the contour coordinates of the target object in the previous video frame image into the contour coordinates of the target object in the new video frame image.

[0227] In one implementation, the parameter iteration module 1504 is specifically configured to:

[0228] Based on the contour coordinates, determine the number of pixels occupied by the target object in the new video frame image;

[0229] Based on the number of pixels and the total number of pixels contained in the new video frame image, determine the area ratio of the target object in the new video frame image;

[0230] If the area ratio is less than or equal to the preset ratio threshold, it is determined that the clarity of the target object in the new video frame image does not meet the preset clarity condition, and the focal length of the camera component is enlarged.

[0231] In one implementation, the parameter iteration module 1504 is specifically configured to:

[0232] Generate an outer rectangle of the target object in a new video frame image based on the contour coordinates;

[0233] Determine the shortest distance between the edge of the bounding rectangle and the edge of the new video frame image;

[0234] If the closest distance is less than or equal to the preset distance threshold, it is determined that the target object is located at the edge of the new video frame image.

[0235] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0236] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned implementation methods.

[0237] Fig.16 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 160, a memory 161, a bus 162 and a communication interface 163, the processor 160, the communication interface 163 and the memory 161 are connected via the bus 162; the processor 160 is used to execute an executable module stored in the memory 161, such as a computer program.

[0238] The memory 161 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 163 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0239] The bus 162 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.16 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0240] Among them, the memory 161 is used to store programs, and the processor 160 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any of the embodiments of the present invention can be applied to the processor 160 or implemented by the processor 160.

[0241] The processor 160 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 160. The above processor 160 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 161, and the processor 160 reads the information in the memory 161 and completes the steps of the above method in combination with its hardware.

[0242] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0243] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0244] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A pan / tilt control method for a surveillance camera center composition, characterized in that: The monitoring camera includes a camera component and a pan / tilt component, and the method includes: Acquire a video frame image captured by the camera assembly for a target object; If it is detected that the target object is located in a non-central area of ​​the video frame image, and / or the clarity of the target object in the video frame image does not meet a preset clarity condition, the rotation angle of the pan / tilt assembly and the focal length of the camera assembly are alternately iterated to obtain a target rotation angle and a target focal length; Based on the target rotation angle and the target focal length, a target video frame image captured by the camera assembly for the target object is acquired; wherein the target object is located in a central area of ​​the target video frame image, and the clarity of the target object in the target video frame image meets the preset clarity condition; If it is detected that the target object is located in a non-central area in the video frame image, and / or the clarity of the target object in the video frame image does not meet a preset clarity condition, the rotation angle of the pan-tilt assembly and the focal length of the camera assembly are alternately iterated to obtain a target rotation angle and a target focal length, including: if it is detected that the target object is located in a non-central area in the video frame image based on the center of mass coordinates corresponding to the target object in the video frame image, the rotation angle of the pan-tilt assembly is determined based on the center of mass coordinates, and the pan-tilt assembly is controlled to rotate according to the rotation angle to adjust the posture of the camera assembly; a new video frame image captured by the adjusted camera assembly for the target object is obtained, and the contour coordinates corresponding to the target object in the new video frame image are detected; wherein the target object is located in the new video frame the central area of ​​the image; if, based on the contour coordinates, it is detected that the clarity of the target object in the new video frame image does not meet the preset clarity condition, the focal length of the camera assembly is adjusted; in the process of adjusting the focal length, a new video frame image collected by the camera assembly for the target object is obtained, and the contour coordinates corresponding to the target object in the new video frame image are detected again; if it is determined based on the contour coordinates that the target object is located at the edge of the new video frame image, the rotation angle of the pan-tilt assembly and the focal length of the camera assembly are alternately iterated and solved again based on the contour coordinates until the target object is located in the central area of ​​the video frame image and the clarity of the target object in the video frame image meets the preset clarity condition, thereby obtaining the target rotation angle of the pan-tilt assembly and the target focal length of the camera assembly; Determining the rotation angle of the gimbal assembly based on the centroid coordinates includes: determining the current centroid homogeneous coordinates based on the centroid coordinates corresponding to the target object in the video frame image, and determining the expected centroid homogeneous coordinates based on the image principal point of the video frame image; determining the rotation matrix of the gimbal assembly according to the intrinsic parameter matrix of the camera assembly and its inverse matrix, the current centroid homogeneous coordinates, and the expected centroid homogeneous coordinates; wherein the rotation matrix includes a yaw angle and a pitch angle; iteratively solving the yaw angle and the pitch angle in the rotation matrix, and using the solved yaw angle and the pitch angle as the rotation angle of the gimbal assembly; Iteratively solving the yaw angle and the pitch angle in the rotation matrix, including: setting the change value of the pitch angle of the gimbal assembly to 0, iterating the yaw angle of the gimbal assembly based on the pitch angle, so that the coordinates of the center of mass after rotation are located at the center of the horizontal direction of the video frame image; iterating the pitch angle based on the solved yaw angle, so that the coordinates of the center of mass after rotation are located at the center of the vertical direction of the video frame image; or, setting the change value of the yaw angle of the gimbal assembly to 0, iterating the pitch angle of the gimbal assembly based on the yaw angle, so that the coordinates of the center of mass after rotation are located at the center of the vertical direction of the video frame image; iterating the yaw angle based on the solved pitch angle, so that the coordinates of the center of mass after rotation are located at the center of the horizontal direction of the video frame image; The yaw angle of the gimbal assembly is iterated based on the pitch angle, including: determining the rotation direction corresponding to the yaw angle of the gimbal assembly based on the relative position relationship between the centroid coordinates and the image principal point; determining the current iteration yaw angle based on the current iteration number, iteration step and the rotation direction; determining the new rotation matrix based on the pitch angle and the current iteration yaw angle, and determining the rotated centroid coordinates corresponding to the target object in the video frame image based on the new rotation matrix; if the difference between the rotated centroid coordinates and the horizontal coordinates of the image principal point is greater than or equal to a preset difference threshold, continuing to determine the next iteration yaw angle based on the next iteration number, the iteration step and the rotation direction, until the difference between the rotated centroid coordinates and the horizontal coordinates of the image principal point is less than the preset difference threshold, stopping the iteration, and taking the last iteration yaw angle as the yaw angle of the gimbal assembly.

2. The PTZ control method of the surveillance camera center composition according to claim 1, characterized in that: Detecting the contour coordinates corresponding to the target object in the new video frame image includes: Extracting a set of point pairs with the same name between a new video frame image and a previous video frame image corresponding to the new video frame image; Determine a homography matrix based on the set of same-name point pairs; The contour coordinates corresponding to the target object in the previous video frame image are converted into the contour coordinates corresponding to the target object in the new video frame image using the homography matrix.

3. The pan / tilt control method of the surveillance camera center composition according to claim 1, characterized in that: If it is detected based on the contour coordinates that the clarity of the target object in the new video frame image does not meet the preset clarity condition, adjusting the focal length of the camera assembly includes: Based on the contour coordinates, determining the number of pixels occupied by the target object in the new video frame image; Determine the area ratio of the target object in the new video frame image based on the number of pixels and the total number of pixels included in the new video frame image; If the area ratio is less than or equal to a preset ratio threshold, it is determined that the clarity of the target object in the new video frame image does not meet the preset clarity condition, and the focal length of the camera assembly is enlarged.

4. The method for controlling the pan / tilt of a surveillance camera center composition according to claim 1, characterized in that: Determining that the target object is located at an edge of the new video frame image based on the contour coordinates includes: Generate a circumscribed rectangle of the target object in the new video frame image based on the contour coordinates; Determine the shortest distance between the edge of the circumscribed rectangle and the edge of the new video frame image; If the closest distance is less than or equal to a preset distance threshold, it is determined that the target object is located at the edge of the new video frame image.

5. A pan / tilt control device for monitoring camera center composition, characterized in that: The monitoring camera includes a camera assembly and a pan / tilt assembly, and the device includes: An image acquisition module, used to acquire a video frame image captured by the camera assembly for a target object; A parameter iteration module, configured to alternately iterate and solve the rotation angle of the pan / tilt assembly and the focal length of the camera assembly to obtain a target rotation angle and a target focal length if it is detected that the target object is located in a non-central area of ​​the video frame image and / or the clarity of the target object presented in the video frame image does not meet a preset clarity condition; The image acquisition module is further used to acquire a target video frame image captured by the camera assembly for the target object based on the target rotation angle and the target focal length; wherein the target object is located in a central area of ​​the target video frame image, and the clarity of the target object in the target video frame image meets the preset clarity condition; The parameter iteration module is specifically used for: if it is detected that the target object is located in a non-central area in the video frame image based on the centroid coordinates corresponding to the target object in the video frame image, then the rotation angle of the pan-tilt assembly is determined based on the centroid coordinates, and the pan-tilt assembly is controlled to rotate according to the rotation angle to adjust the posture of the camera assembly; a new video frame image captured by the adjusted camera assembly for the target object is obtained, and the contour coordinates corresponding to the target object in the new video frame image are detected; wherein the target object is located in the central area of ​​the new video frame image; if it is detected based on the contour coordinates that the clarity of the target object in the new video frame image is not If the preset clarity condition is met, the focal length of the camera assembly is adjusted; a new video frame image captured by the camera assembly for the target object during the process of adjusting the focal length is obtained, and the contour coordinates corresponding to the target object in the new video frame image are detected again; if it is determined based on the contour coordinates that the target object is located at the edge of the new video frame image, the rotation angle of the pan-tilt assembly and the focal length of the camera assembly are alternately iterated and solved again based on the contour coordinates until the target object is located in the central area of ​​the video frame image, and the clarity of the target object in the video frame image meets the preset clarity condition, thereby obtaining the target rotation angle of the pan-tilt assembly and the target focal length of the camera assembly; The parameter iteration module is specifically used to: determine the current center of mass homogeneous coordinates based on the center of mass coordinates corresponding to the target object in the video frame image, and determine the expected center of mass homogeneous coordinates based on the image principal point of the video frame image; determine the rotation matrix of the gimbal assembly according to the intrinsic parameter matrix of the camera assembly and its inverse matrix, the current center of mass homogeneous coordinates, and the expected center of mass homogeneous coordinates; wherein the rotation matrix includes a yaw angle and a pitch angle; iteratively solve the yaw angle and the pitch angle in the rotation matrix, and use the solved yaw angle and the pitch angle as the rotation angle of the gimbal assembly; The parameter iteration module is specifically used for: setting the change value of the pitch angle of the gimbal assembly to 0, iterating the yaw angle of the gimbal assembly based on the pitch angle, so that the coordinates of the center of mass after rotation are located at the center of the horizontal direction of the video frame image; iterating the pitch angle based on the solved yaw angle, so that the coordinates of the center of mass after rotation are located at the center of the vertical direction of the video frame image; or setting the change value of the yaw angle of the gimbal assembly to 0, iterating the pitch angle of the gimbal assembly based on the yaw angle, so that the coordinates of the center of mass after rotation are located at the center of the vertical direction of the video frame image; iterating the yaw angle based on the solved pitch angle, so that the coordinates of the center of mass after rotation are located at the center of the horizontal direction of the video frame image; The parameter iteration module is specifically used to: determine the rotation direction corresponding to the yaw angle of the gimbal assembly based on the relative position relationship between the center of mass coordinates and the image principal point; determine the current iteration yaw angle based on the current iteration number, iteration step and the rotation direction; determine the new rotation matrix based on the pitch angle and the current iteration yaw angle, and determine the rotated center of mass coordinates corresponding to the target object in the video frame image based on the new rotation matrix; if the difference between the rotated center of mass coordinates and the horizontal coordinates of the image principal point is greater than or equal to a preset difference threshold, continue to determine the next iteration yaw angle based on the next iteration number, the iteration step and the rotation direction, until the difference between the rotated center of mass coordinates and the horizontal coordinates of the image principal point is less than the preset difference threshold, stop iteration, and use the last iteration yaw angle as the yaw angle of the gimbal assembly.

6. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 4.

Images