Remote personnel identification and tracking method based on zoom pan-tilt camera

Abstract

The invention relates to a remote personnel identifying and tracking method based on a zoom pan-tilt camera, and belongs to the technical field of image processing. According to the method, an image is acquired through a camera, if no suspicious person exists in a current frame or the suspicious person exists but the confidence coefficient score is smaller than a first threshold value through yov11 + DeepSOR processing, it is judged that no target exists in the current frame, a PTZ control command is sent, and reset operation is executed; and if the suspicious person exists, the pan-tilt and the camera zoom are accurately controlled, so that effective identification and tracking of the remote target person are realized. The method provided by the invention not only improves the image capture quality, but also ensures that the picture of each target person in the center of the image is stored, thereby optimizing the efficiency and accuracy of a tracking task.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a long-distance personnel recognition and tracking method based on a variable-focus pan-tilt camera. Background Art

[0002] In existing surveillance systems, cameras are widely used for people detection and tracking between buildings, but the accuracy of long-distance recognition images has always been a challenge. Usually, in the images captured by the camera, the target person may be small and may not be in the center of the image, which makes recognition and tracking more difficult. Therefore, an effective method is needed to improve the accuracy and efficiency of long-distance target recognition.

[0003] Traditional camera systems can only provide images with fixed viewing angles and focal lengths when tracking target personnel, which limits the quality of capturing distant targets. In addition, when target personnel move between different buildings, their images may flash rapidly, making it difficult to capture and store effective tracking information. To overcome these problems, a method is needed to dynamically adjust the camera's viewing angle and focal length to ensure that the target person's image is captured clearly. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] The technical problem to be solved by the present invention is how to provide a long-distance personnel recognition and tracking method based on a variable-focus pan-tilt camera to solve the problem of long-distance target recognition and tracking.

[0006] (II) Technical solution

[0007] In order to solve the above technical problems, the present invention proposes a long-distance personnel recognition and tracking method based on a variable-focus pan-tilt camera, the method comprising the following steps:

[0008] S1. Obtain an image through the camera and process it through yolov11+DeepSOR. If there is no suspicious person in the current frame or there is a suspicious person but the confidence score is less than the first threshold, it is determined that there is no target in the current picture, and a PTZ control command is sent to perform a reset operation;

[0009] S2. If there is a suspicious person and his confidence score is greater than the first threshold, it is determined that there is a suspicious target in the current picture, and then it is determined whether the confidence score is greater than the second threshold. If it is greater than the second threshold, it is determined that there is a target person in the current picture, and the current picture is saved. If the confidence score is less than or equal to the second threshold, the target information in the current frame image is obtained, all detection results in the current frame are traversed, and the average coordinate value of all target persons in the current frame is obtained. The difference between the coordinate value and the center point of the image is calculated, and then it is decided whether to control the PTZ to rotate the camera until the target person is located in the center of the image; then it is determined whether the proportion of the target person in the picture is too large or too small to determine the camera zoom operation.

[0010] (III) Beneficial effects

[0011] The present invention proposes a long-distance personnel identification and tracking method based on a zoomable pan-tilt camera. By combining the target detection algorithm, the tracking algorithm, and the precise control of the pan-tilt and camera zoom, the effective identification and tracking of the long-distance target personnel is realized. This method not only improves the image capture quality, but also ensures that the image of each target person located at the center of the image is stored, thereby optimizing the efficiency and accuracy of the tracking task. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is the overall flow chart of the present invention;

[0013] Figure 2 This is the magnification flow chart;

[0014] Figure 3 This is the gimbal control flow chart. DETAILED DESCRIPTION

[0015] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0016] The present invention proposes a long-distance personnel recognition and tracking method based on a variable-focus pan-tilt camera, such as Figure 1 As shown, the method of the present invention comprises the following steps:

[0017] S1. Obtain an image through a camera and process it through yolov11+DeepSOR. If there is no suspicious person in the current frame or there is a suspicious person but the confidence score is less than the first threshold, it is determined that there is no target in the current picture, and a PTZ control command is sent to perform a reset operation. In one embodiment, the first threshold is 0.3.

[0018] When the video is detected based on the yolov11 target detection algorithm, the video is split into frames for processing. During this processing, the detected target cannot be associated between frames, and if the detection frame shakes due to occlusion or false detection, the target may be lost. Therefore, based on the yolov11 target detection algorithm, the DeepSOR model tracking algorithm is added. DeepSOR can assign a unique ID to each detected target, so that each target can be tracked continuously in the video, and when the target is temporarily occluded or not detected, DeepSORT can continue to track the target based on the motion trajectory and appearance information, and predict the target motion trajectory through Kalman filtering. Even if the detection fails, it can maintain a relatively stable tracking result. In this way, the phenomenon of camera tracking jamming caused by detection failure when tracking long-distance targets is solved. After being processed by the yolov11 and DeepSOR algorithms, the image can provide more accurate information about the target personnel for PTZ (Pan-Tilt-Zoom) control.

[0019] S2. If there is a suspicious person and his confidence score is greater than the first threshold, it is determined that there is a suspicious target in the current picture, and then whether its confidence score is greater than the second threshold. If it is greater than the second threshold, it is determined that there is a target person in the current picture, and the current picture is saved. If the confidence score is less than or equal to the second threshold, the target information in the current frame image is obtained, all detection results in the current frame are traversed, and the average coordinate value of all target persons in the current frame is obtained, and the difference between the coordinate value and the center point of the image is calculated, and then it is determined whether to control the PTZ to rotate the camera until the target person is located in the center of the image; then it is determined whether the proportion of the target person in the picture is too large or too small to determine the camera zoom operation. In a certain embodiment, the second threshold is 0.8.

[0020] like Figure 2 As shown in FIG. 1 , the specific process of the zoom operation includes:

[0021] S211, according to the information of all target boxes in the current frame, the width and height box_width and box_height of the minimum circumscribed rectangle that can surround all detection boxes are obtained;

[0022] The width of the minimum circumscribed rectangle is calculated as: box_width = max_x - min_x; max_x is the maximum x coordinate of all detection boxes, and min_x is the minimum x coordinate of all detection boxes;

[0023] The height of the minimum circumscribed rectangle is calculated as: box_height = max_y - min_y; max_y is the maximum y coordinate of all detection boxes, and min_y is the minimum y coordinate of all detection boxes;

[0024] S212, defining the required screen margin coefficient margin_ratio;

[0025] In one embodiment, margin_ratio=0.1.

[0026] S213, obtaining the required width and height required_width and required_height according to the screen margin coefficient;

[0027] Among them, required_width=self.frame_width*(1-margin_ratio*2);

[0028] Among them, required_height=self.frame_height*(1-margin_ratio*2);

[0029] self.frame_width and self.frame_height are the width and height of the current frame.

[0030] S214, determine the relationship between the width and height of the minimum circumscribed rectangle and the required width and height, and obtain the zoom instruction; if box_width>required_width or box_height>required_height, it means that the target box occupies a large proportion of the picture, and the camera needs to be controlled to zoom out. Otherwise, it means that the target box occupies a small proportion of the picture. In order to confirm whether the target box is indeed the target person and improve the confidence score, it is necessary to zoom in.

[0031] like Figure 3 As shown in the figure, the specific process of the gimbal controlling the camera rotation operation is as follows:

[0032] S221. First, by obtaining the width frame.shape[1] and height frame.shape[0] of the image, the horizontal center img_center_x and the vertical center img_center_y of the image are calculated;

[0033] S222, defining a third threshold value threshold, used to determine whether the position difference between the center of the target frame and the center of the image reaches a level that requires controlling the movement of the camera; wherein the differences diff_x and diff_y between the horizontal center box_center_x and the vertical center box_center_y of the target frame and the horizontal center and vertical center of the image are calculated;

[0034] S223, initialize a command variable for storing the final control command; if the absolute value of diff_x or diff_y is greater than a third threshold, set a different control command according to the specific direction difference, otherwise do not move;

[0035] Control commands include:

[0036] If the target box is in the upper left corner of the image, the command is UP_LEFT.

[0037] If the target box is in the upper right corner of the image, the command is UP_RIGHT.

[0038] If the target box is in the lower left corner of the image, the command is DOWN_LEFT.

[0039] If the target box is in the lower right corner of the image, the command is DOWN_RIGHT.

[0040] If the target box is only offset to the left in the horizontal direction, the command is PAN_LEFT.

[0041] If the target box is offset to the right only in the horizontal direction, the command is PAN_RIGHT.

[0042] If the target box is only tilted up vertically, the command is TILT_UP.

[0043] If the target box is only tilted down in the vertical direction, the command is TILT_DOWN.

[0044] In one embodiment, the third threshold value threshold=20.

[0045] S224: If the difference is greater than the threshold and the command is determined, the gimbal is controlled to move in a specified manner.

[0046] When calling the camera control command, first send a signal to start the control (parameter is 0), then wait according to the appropriate dynamic_sleep time calculated by the calculate_dynamic_sleep function, and finally send a signal to stop the control (parameter is 1). The purpose of the calculate_dynamic_sleep function is to calculate the appropriate waiting time according to the difference size and direction, so that the movement of the gimbal is smoother.

[0047] The present invention proposes a long-distance target recognition and tracking method based on a zoomable pan-tilt camera. By combining the target detection algorithm, the tracking algorithm, and the precise control of the pan-tilt and camera zoom, the effective recognition and tracking of the long-distance target personnel is realized. This method not only improves the image capture quality, but also ensures that the image of each target person located at the center of the image is stored, thereby optimizing the efficiency and accuracy of the tracking task.

[0048] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A long-distance personnel identification and tracking method based on a zoomable pan-tilt camera, characterized in that: The method comprises the following steps: S1. Obtain an image through the camera and process it through yolov11+DeepSOR. If there is no suspicious person in the current frame or there is a suspicious person but the confidence score is less than the first threshold, it is determined that there is no target in the current picture, and a PTZ control command is sent to perform a reset operation; S2. If there is a suspicious person and his confidence score is greater than the first threshold, it is determined that there is a suspicious target in the current picture, and then it is determined whether the confidence score is greater than the second threshold. If it is greater than the second threshold, it is determined that there is a target person in the current picture, and the current picture is saved. If the confidence score is less than or equal to the second threshold, the target information in the current frame image is obtained, all detection results in the current frame are traversed, and the average coordinate value of all target persons in the current frame is obtained. The difference between the coordinate value and the center point of the image is calculated, and then it is decided whether to control the PTZ to rotate the camera until the target person is located in the center of the image; then it is determined whether the proportion of the target person in the picture is too large or too small to determine the camera zoom operation.

2. The long-distance personnel identification and tracking method based on a variable-focus pan-tilt camera according to claim 1, characterized in that: In S1, the first threshold is 0.

3.

3. The long-distance personnel identification and tracking method based on a variable-focus pan-tilt camera according to claim 1, characterized in that: In S1, based on the yolov11 target detection algorithm, the DeepSOR model tracking algorithm is added. DeepSOR assigns a unique ID to each detected target, so that each target can be continuously tracked in the video. When the target is temporarily occluded or not detected, DeepSORT continues to track the target according to the motion trajectory and appearance information, and predicts the target motion trajectory through Kalman filtering. Even if the detection fails, it can maintain a relatively stable tracking result. After being processed by the yolov11 and DeepSOR algorithms, the image can provide the target person information for PTZ control more accurately.

4. The long-distance personnel identification and tracking method based on a variable-focus pan-tilt camera according to claim 1, characterized in that: In S2, the second threshold is 0.

8.

5. The long-distance personnel identification and tracking method based on a variable-focus pan-tilt camera according to any one of claims 1 to 4, characterized in that: In S2, the specific process of the zoom operation includes: S211, according to the information of all target boxes in the current frame, the width and height box_width and box_height of the minimum circumscribed rectangle that can surround all detection boxes are obtained; S212, defining the required screen margin coefficient margin_ratio; S213, obtaining the required width and height required_width and required_height according to the screen margin coefficient; S214, determine the relationship between the width and height of the minimum circumscribed rectangle and the required width and height, and obtain the zoom instruction; if box_width>required_width or box_height>required_height, it means that the target box occupies a large proportion of the picture, and the camera needs to be controlled to zoom out. Otherwise, it means that the target box occupies a small proportion of the picture. In order to confirm whether the target box is indeed the target person and improve the confidence score, it is necessary to zoom in.

6. The long-distance personnel identification and tracking method based on a variable-focus pan-tilt camera according to claim 5, characterized in that: In the S211, The width of the minimum circumscribed rectangle is calculated as: box_width = max_x - min_x; max_x is the maximum x coordinate of all detection boxes, and min_x is the minimum x coordinate of all detection boxes; The height of the minimum circumscribed rectangle is calculated as: box_height = max_y - min_y; max_y is the maximum y coordinate of all detection boxes, and min_y is the minimum y coordinate of all detection boxes.

7. The long-distance personnel identification and tracking method based on a variable-focus pan-tilt camera according to claim 5, characterized in that: In S212, margin_ratio=0.

1.

8. The long-distance personnel identification and tracking method based on a variable-focus pan-tilt camera according to claim 5, characterized in that: In the S213, required_width=self.frame_width*(1-margin_ratio*2); required_height=self.frame_height*(1-margin_ratio*2); Among them, self.frame_width and self.frame_height are the width and height of the current frame.

9. The long-distance personnel identification and tracking method based on a variable-focus pan-tilt camera according to claim 5, characterized in that: The camera rotation in S2 includes the following steps: S221. First, by obtaining the width frame.shape[1] and height frame.shape[0] of the image, the horizontal center img_center_x and the vertical center img_center_y of the image are calculated; S222, defining a third threshold value threshold, used to determine whether the position difference between the center of the target frame and the center of the image reaches a level that requires controlling the movement of the camera; wherein the differences diff_x and diff_y between the horizontal center box_center_x and the vertical center box_center_y of the target frame and the horizontal center and vertical center of the image are calculated; S223, initialize a command variable for storing the final control command; if the absolute value of diff_x or diff_y is greater than a third threshold, set a different control command according to the specific direction difference, otherwise do not move; S224. If the difference is greater than the threshold and the command is determined, the gimbal is controlled to move in the specified manner; when the camera control command is called, a signal to start the control is first sent, then the appropriate time dynamic_sleep calculated by the calculate_dynamic_sleep function is waited, and finally a signal to stop the control is sent; the purpose of the calculate_dynamic_sleep function is to calculate the appropriate waiting time according to the size and direction of the difference, so that the movement of the gimbal is smoother.

10. The long-distance personnel identification and tracking method based on a variable-focus pan-tilt camera according to claim 9, characterized in that: In the S223, If the target box is in the upper left corner of the image, the command is UP_LEFT; If the target box is in the upper right corner of the image, the command is UP_RIGHT; If the target box is in the lower left corner of the image, the command is DOWN_LEFT; If the target box is in the lower right corner of the image, the command is DOWN_RIGHT; If the target box is only horizontally left, the command is PAN_LEFT; If the target box is only horizontally right, the command is PAN_RIGHT; If the target box is only upward in the vertical direction, the command is TILT_UP; If the target box is only tilted down in the vertical direction, the command is TILT_DOWN.

Images