Double-target automatic tracking shooting method and automatic tracking shooting device

Abstract

The invention discloses a double-target automatic tracking shooting method and an automatic tracking shooting device, and the method comprises the steps: determining two to-be-tracked targets in a real-time shooting image; obtaining respective feature vector queues of two to-be-tracked targets through a human body feature algorithm, and obtaining respective reference target frame serial numbers of the two to-be-tracked targets through a target detection algorithm; based on the feature vector queue and a reference target frame serial number, matching a target frame in each frame in a subsequent real-time shot picture; if matching succeeds or partial matching succeeds, determining the position of a tracking point according to the position of the successfully matched target frame; if matching fails, the position of the fixed tracking point is the center of the current picture; and when the matching of the continuous preset frames fails, adjusting the focal length to expand the field of view of the real-time shot picture. According to the dual-target cooperative tracking mechanism, dual-target equivalent tracking is realized through parallel feature vector queues and dynamic weight distribution.

Description

Technical Field

[0001] The embodiments of the present application relate to the fields of computer vision technology and motion tracking photography, and in particular to a dual-target automatic tracking photography method and an automatic tracking photography device. Background Art

[0002] Dual-target automatic tracking technology is a core requirement in intelligent surveillance, sports photography, and drone tracking. While multi-target tracking algorithms are becoming increasingly popular in security, film, and television production, existing technologies still suffer from two core flaws:

[0003] 1. Insufficient stability of dual-target collaborative tracking

[0004] Current mainstream systems use a single-target priority tracking strategy (such as face lock or moving subject priority). When two targets of equal importance appear in the picture (such as opposing players in sports events or cooperating suspects in security), the system needs to frequently switch the tracking subject. This switching results in:

[0005] Lens shake and focus drift: The camera repeatedly jumps between two targets, and the shooting image is unstable;

[0006] High target loss rate: According to experimental data, the loss rate of traditional feature matching algorithms exceeds 40% in scenarios where targets are cross-occluded.

[0007] Waste of hardware resources: Continuous fine-tuning of the camera gimbal increases power consumption, reducing the battery life of battery-powered devices (such as drones) by more than 30%.

[0008] 2. Rigid recovery mechanism after target loss

[0009] Existing solutions typically use two inefficient strategies when the target is lost, failing to balance the requirements of real-time performance and tracking continuity:

[0010] Global search: Expanding the detection range to the entire screen increases computational complexity exponentially;

[0011] Fixed Position Freeze: Locks the tracking point to the last known position, causing the camera to move away from the valid area. Summary of the Invention

[0012] In order to solve the above technical problems, this application adopts the following technical solutions to solve the above problems that may arise when directly using existing technologies in scenes where tracking and shooting are required in some two-player competitions (such as fencing, boxing and other fighting competitions) or in scenes where tracking of two designated athletes is required in multi-player competitions.

[0013] In a first aspect of the present application, a dual-target automatic tracking and shooting method is provided, comprising:

[0014] S1: Determine two targets to be tracked in the real-time shooting picture;

[0015] S2: Obtaining feature vector queues of the two targets to be tracked respectively through a human feature algorithm, and obtaining reference target frame serial numbers of the two targets to be tracked respectively through a target detection algorithm;

[0016] S3: matching the target frame in each frame of the subsequent real-time shooting picture based on the feature vector queue and the reference target frame sequence number;

[0017] If the match is successful or partially successful, the tracking point position is determined based on the position of the successfully matched target frame;

[0018] If the matching fails, the fixed tracking point position is the center of the current image;

[0019] S4: When the continuous preset frame matching fails, the focus is adjusted to expand the field of view of the real-time shooting picture.

[0020] In another embodiment, S4 of the dual-target automatic tracking shooting method can be replaced by S41: when the continuous preset frame matching fails, the shooting angle of view is adjusted in the opposite direction of the yaw direction of the shooting angle of view.

[0021] The S3 step includes:

[0022] S31: Detect the current frame using a target detection algorithm to obtain target frame sequence numbers of all human targets in the current frame;

[0023] S32: For each target to be tracked, match the target frame serial numbers of all human targets with the reference target frame serial number of the target to be tracked.

[0024] Preferably, step S32 includes:

[0025] S321: Acquire feature vectors of all human targets in the current frame using a human feature algorithm;

[0026] For each target to be tracked, if the matching in step S32 fails, execute S322: match the feature vectors of all human targets with the feature vector queue of the target to be tracked one by one. If there is a match whose vector Euclidean distance is less than the threshold, it is considered that the matching in step S3 is successful, and the human target frame whose feature vector is successfully matched is used as the successfully matched target frame.

[0027] When there is a matching item whose vector Euclidean distance is less than the threshold, the following steps are performed:

[0028] S3221: Increasing the weight of the corresponding feature vector that has been successfully matched in the feature vector queue;

[0029] S3222: Reduce the weights of other feature vectors in the feature vector queue;

[0030] S3223: Add the successfully matched feature vector to the feature vector queue and assign an initial weight;

[0031] S3224: Update the reference target frame number to the target frame number of the human target whose feature vector is successfully matched.

[0032] Preferably, step S32 further includes:

[0033] For each target to be tracked, if the match in step S32 is successful, then execute S323: determine whether the human target frame whose target frame sequence number matches successfully overlaps with other target frames:

[0034] If there is no overlap, the matching in step S3 is considered successful, and the human target frame with the successfully matched target frame sequence number is regarded as the successfully matched target frame;

[0035] If there is overlap, execute step S322.

[0036] If the match is successful or partially successful, the tracking point position is determined based on the position of the successfully matched target frame, including:

[0037] If the match is successful, the midpoint of the line connecting the center points of the target frames of the two objects to be tracked is used as the tracking point position;

[0038] If the partial match is successful, the center point of the successfully matched target frame is used as the tracking point position.

[0039] Preferably, if the partial match is successful, the center point of the successfully matched target frame is used as the tracking point position, including:

[0040] Determine whether there is a successful matching record for the target to be tracked that failed to match in the previous preset frame:

[0041] If it exists, the midpoint of the line connecting the center point of the successfully matched target frame and the center point of the last successfully matched target frame of the target to be tracked that failed to match is taken as the tracking point position;

[0042] If it does not exist, the center point of the successfully matched target frame will be used as the tracking point position.

[0043] In a second aspect of the present application, an automatic tracking shooting device is provided, comprising a shooting module, an analysis module and a rotation module. The shooting module is physically connected to the rotation module, the rotation module is suitable for horizontal rotation, the shooting module is suitable for continuously acquiring real-time shooting images, and sending the real-time shooting images to the analysis module; the analysis module is suitable for executing the method of the first aspect of the present application to calculate the tracking point position, generate a rotation instruction based on the tracking point position, and send the rotation instruction to the rotation module; the rotation module is suitable for changing the angle of the shooting module according to the rotation instruction.

[0044] The third aspect of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the method of the first aspect of the present application when executing the computer program.

[0045] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method of the first aspect of the present application when executed by a processor.

[0046] In a fifth aspect of the present application, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, the method of the first aspect of the present application is implemented.

[0047] This application's dual-target collaborative tracking mechanism achieves equivalent tracking of two targets through parallel feature vector queues and dynamic weight allocation. It uses target box center mean positioning to eliminate camera shake. A hierarchical recovery strategy first locks the center point when a match fails to avoid invalid searches. After continuous loss, adaptive focus extension is triggered to systematically recover the target. Online feature vector queue updates mitigate lighting and occlusion interference. Overlapping target decoupling logic resolves cross-occlusion ambiguity. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of a dual-target automatic tracking and shooting method based on an embodiment of the present application;

[0049] Figure 2 is a block diagram of an automatic tracking shooting device according to an embodiment of the present application;

[0050] Figure 3 It is a structural diagram of a terminal device or server suitable for implementing the embodiments of the present application. DETAILED DESCRIPTION

[0051] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0052] In the field of computer vision analysis, it is generally assumed that the origin of the coordinate system is located in the upper left corner of the screen. In the various embodiments of the present application, unless otherwise stated, the upper left corner is used as the origin of the coordinate system of the screen. Those skilled in the art should know that such a coordinate system setting is not absolutely fixed. When the origin of the coordinate system is set at any position inside or outside the screen, the corresponding technical solutions that can be obtained by simple adjustments to this solution without creative labor are all within the scope of protection of this application.

[0053] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0054] Figure 1 A flow chart of a dual-target automatic tracking and shooting method based on an embodiment of the present application is shown. Figure 1 As shown:

[0055] S1: Identify two targets to be tracked in the real-time shooting image.

[0056] The real-time footage is the actual footage of the game captured by the user. In one possible implementation, the user can directly select two targets to be tracked by touching the camera. In another possible implementation, the user views the real-time footage on the camera through a remote control and selects two targets to be tracked on the remote control. The remote control synchronizes the user's selection with the camera.

[0057] The target to be tracked determined by the user should be a full-body target so that S2 can extract both half-body features and full-body features.

[0058] S2: Obtain the feature vector queues of the two targets to be tracked respectively through the human feature algorithm, and obtain the reference target frame sequence numbers of the two targets to be tracked respectively through the target detection algorithm.

[0059] The human feature algorithm is used to extract unique features such as the athlete's face and body shape for subsequent similarity matching. This application does not impose any restrictions on the specific selection of the human feature algorithm, and any technical solution using any open source or proprietary feature algorithm is within the scope of protection of this application. For example, the human feature algorithm can use facial feature recognition algorithms such as FaceNet and InsightFace, or human body shape feature recognition algorithms such as OpenPose and Multi-Human-Shape CNN. It can also use a mixture of the two to extract multiple features for more accurate target matching. Among them, for the target to be tracked, at least two different feature vectors, half-body and full-body, need to be extracted for comparison. These feature vectors are stored in a feature vector queue.

[0060] The target detection algorithm is used to extract all human body information from the video frame and output it in the form of a detection frame. The specific choice of the target detection algorithm is not limited in this application: the YOLOv8 model can be used, or recognition algorithms such as CNN (convolutional neural network) and ViT (Vision Transformer) can be used. In order to make the recognition results more accurate, a large number of annotated pictures can be used to train the algorithm model. The training of algorithms and models is a common technical means that those skilled in the art should know. It does not belong to the content that needs to be protected in this application and will not be described here. The target detection algorithm will use four parameters for each detected target: the horizontal and vertical coordinates (x, y) of the center point of the target frame and the width (abbreviated as W) and height (abbreviated as H) of the target frame. A unique serial number will also be assigned to each target frame. In the recognition of consecutive frames, the target detection algorithm will use the same serial number for the same target that is tracked continuously.

[0061] In the initial state, the target frame number of the target to be tracked is first recognized during initialization as the reference target frame number.

[0062] S3: Based on the feature vector queue and the reference target frame sequence number, the target frame in each frame of the subsequent real-time shooting picture is matched. If the match is successful or partially successful, the tracking point position is determined according to the position of the successfully matched target frame; if the match fails, the tracking point position is fixed to the center of the current picture.

[0063] After initialization, real-time tracking of dual targets can be performed in subsequent live captures. Specifically, for each target to be tracked, the target feature vector and target frame number extracted from each frame are matched with the feature vector queue and the reference target frame number. If a match is successful, the corresponding target in the current frame is considered to be the same person as the target to be tracked.

[0064] Since this application involves automatic tracking of two targets, the matching results of step S3 may have three situations:

[0065] A successful match means both targets are matched. At this point, you can choose either one as the tracking point, or the midpoint between the two. Generally, the position of the target box is represented by the (x, y) coordinates of its center point.

[0066] Partial matching is successful, that is, only one of the two targets is matched successfully and the other fails. In this case, the center point of the successfully matched target box can be used as the tracking point position.

[0067] If the match fails, meaning both targets fail to match, the tracking point is fixed at the center of the current frame, meaning the camera angle remains unchanged. If both targets are lost, there's a high probability of severe occlusion, so it's not advisable to immediately adjust the camera to continue searching.

[0068] S4: When the continuous preset frame matching fails, the focus is adjusted to expand the field of view of the real-time shooting picture.

[0069] If S3 fails to match in consecutive preset frames, it can be considered that the target to be tracked has left the real-time shooting picture, and it is necessary to adjust the real-time shooting picture to continue searching for the target to be tracked.

[0070] In one possible implementation, the focal length of the camera is reduced to expand the field of view so as to continue searching for the target to be tracked.

[0071] In another possible implementation, the camera angle is adjusted in the opposite direction of the camera's yaw angle to continue searching for the target to be tracked. Specifically, if the current camera angle's yaw angle is positive, it indicates that the current live view is on the right side of the playing field. Since the target to be tracked has left the live view, it is more likely to have reached the left side of the playing field. Therefore, the camera angle is adjusted in the negative direction. The reverse direction also applies, which will not be further explained here.

[0072] In one possible implementation, when there are only two players on the field and the possibility of overlap or confusion between targets is low, such as when shooting a fencing match, S3 can only use the target frame number for matching. This includes:

[0073] S31: Detect the current frame using a target detection algorithm to obtain target frame sequence numbers of all human targets in the current frame;

[0074] S32: For each target to be tracked, match the target frame serial numbers of all human targets with the reference target frame serial number of the target to be tracked.

[0075] Specifically, for each target to be tracked, its reference target frame number is matched with the target frame numbers of all human targets in the current frame in sequence. When the target frame number of a human target in the current frame is the same as the reference target frame number, the target to be tracked is considered to be matched successfully, that is, it is judged that the corresponding target in the current frame and the target to be tracked are the same person.

[0076] In another possible implementation, when there are many players on the field and they move frequently, the target frame serial numbers may be frequently confused or changed. In this case, when the matching fails in step S32, human body features can be used to supplement the target matching.

[0077] S321: Acquire feature vectors of all human targets in the current frame using a human feature algorithm;

[0078] The method of extracting human feature vectors by the human feature algorithm is the same as the method of extracting human feature vectors in the initialization work, which will not be repeated here; if the target shows the whole body in the picture, the whole body feature vector is extracted; if the target only has half of the body in the picture, the half-body feature vector is extracted.

[0079] For each target to be tracked, if the matching in step S32 fails, execute S322: match the feature vectors of all human targets with the feature vector queue of the target to be tracked one by one. If there is a match whose vector Euclidean distance is less than the threshold, it is considered that the matching in step S3 is successful, and the human target frame whose feature vector is successfully matched is used as the target frame with successful matching, and the reference target frame serial number is updated to the target frame serial number of the human target whose feature vector is successfully matched.

[0080] If the matching in step S32 fails, it means that there is no target frame number that is the same as the reference target number in the current picture. At this time, the target detection algorithm may have refreshed the target frame number, or the number may have been reset because the target was lost. At this time, the human target is supplemented by human features. Specifically, for each target to be tracked, the Euclidean distance between its feature vector queue and the feature vectors of all human targets in the current frame is calculated in sequence. When the Euclidean distance between the feature vector of the human target in the current frame and a vector element in the feature vector queue is less than the threshold, the target to be tracked is deemed to be matched successfully, that is, it is judged that the corresponding target in the current frame and the target to be tracked are the same person. Generally, when the Euclidean distance between two feature vectors is less than 0.05, it can be considered that the human bodies represented by the two feature vectors are the same person.

[0081] After the target to be tracked is successfully matched by means of a feature vector, it indicates that its target frame number has changed. Therefore, its new target frame number is used as the reference target frame number so that matching can continue through step S32 in subsequent frames.

[0082] Preferably, after the target to be tracked is successfully matched by means of feature vectors, the feature vector queue of the corresponding target to be tracked can be updated and the weight adjusted so that in subsequent matching, features that are more likely to be successfully matched can be matched first, thereby reducing a certain amount of algorithm running time.

[0083] S3221: Increasing the weight of the corresponding feature vector that has been successfully matched in the feature vector queue;

[0084] S3222: Reduce the weights of other feature vectors in the feature vector queue;

[0085] S3223: Add the successfully matched feature vector to the feature vector queue and assign an initial weight.

[0086] The following is an example to illustrate the above steps S3221-S3223. In the following example, the specific values ​​of the weights are only examples, and those skilled in the art can adjust the values ​​according to actual conditions.

[0087] The feature vector queue contains three vectors: [vector 1, vector 2, vector 3], all with an initial weight of 8. In one match, feature vector 3 successfully matches vector 4 of the corresponding target in the current frame. This indicates that the target to be tracked is more likely to be matched with vector 3 in subsequent video frames. Therefore, the weight of vector 3 is increased by +1. At the same time, the weights of vectors 1 and 2 can be reduced by -1 so that if vectors 1 and 2 continue to fail to match, they can be removed from the feature vector queue. Finally, vector 4 is added to the feature vector queue and assigned an initial weight. Finally, the elements in the feature vector queue are sorted from high to low by weight, resulting in a new feature vector queue: [vector 3, vector 4, vector 1, vector 2], with weights of 9, 8, 7, and 7, respectively.

[0088] Since in the target detection algorithm, when the human target frames overlap and then separate, the algorithm has a certain probability of confusing the two target frames. Therefore, in a preferred implementation, when S32 matches successfully, a secondary verification method can be used to further avoid false detection.

[0089] For each target to be tracked, if step S32 is successfully matched, then step S323 is executed: determining whether the human target frame whose target frame sequence number matches successfully overlaps with other target frames. Generally, when the area of ​​target frame overlap exceeds 20% of the total area, the target detection algorithm is likely to confuse the human target. Therefore, in this application, a target frame overlap event is determined when the target frame of the current frame overlaps by more than 20% of the area of ​​the current target frame.

[0090] The specific calculation method of target frame overlap is as follows:

[0091] First, we preliminarily screen whether there are other target frames close to the target frame to be judged. It can be assumed that if the distance between the midpoints of two target frames is less than the height of the target frame to be judged, there is a possibility of overlap.

[0092] For each target frame that may overlap, the center point coordinates are converted to corner point coordinates, and the judgment can be made by comparing the coordinate relationship.

[0093] 1. Bounding box corner point calculation (YOLO center point coordinates to corner point coordinates):

[0094] Coordinates of the upper left corner of box 1: box1_x1=x1−w1 / 2, box1_y1=y1−h1 / 2

[0095] Coordinates of the lower right corner of box 1: box1_x2=x1+w1 / 2, box1_y2=y1+h1 / 2

[0096] Coordinates of the upper left corner of box 2: box2_x1=x2−w2 / 2, box2_y1=y2−h2 / 2

[0097] Coordinates of the lower right corner of box 2: box2_x2=x2+w2 / 2, box2_y2=y2+h2 / 2

[0098] Coincidence conditions (must be met at the same time):

[0099] Horizontal overlap: box1_x2>box2_x1 and box1_x1 <box2_x2

[0100] Vertical overlap: box1_y2>box2_y1 and box1_y1 <box2_y2

[0101] If any of the conditions is not met, the two frames do not overlap.

[0102] 2. Calculate the overlapping area

[0103] If the two boxes overlap, calculate the intersection area as follows:

[0104] Calculate the intersection area coordinates:

[0105] Coordinate of the upper left corner of the intersection: inter_x1=max(box1_x1,box2_x1)

[0106] Coordinate of the upper left corner of the intersection: inter_y1=max(box1_y1,box2_y1)

[0107] Coordinates of the lower right corner of the intersection: inter_x2=min(box1_x2,box2_x2)

[0108] Coordinate of the lower right corner of the intersection: inter_y2=min(box1_y2,box2_y2)

[0109] Intersection area inter_area=max(0,inter_x2−inter_x1)×max(0,inter_y2−inter_y1)

[0110] In the above formula, x1, y1, w1, and h1 are the coordinates of the center point and the width and height of frame 1; x2, y2, w2, and h2 are the coordinates of the center point and the width and height of frame 2.

[0111] The overlap ratio is obtained by dividing the intersection area by the area of ​​the target box to be judged.

[0112] If there is no overlap, the matching in step S3 is considered successful, and the human target frame with the successfully matched target frame sequence number is regarded as the successfully matched target frame;

[0113] If there is overlap, it means that the result of the target frame serial number matching may not be credible; execute step S322, that is, match through the human body feature vector.

[0114] In another possible implementation, when determining the tracking point position in S3, the following technical solution may be used:

[0115] If the match is successful, the midpoint of the line connecting the center points of the target frames of the two objects to be tracked is used as the tracking point position;

[0116] If the partial match is successful, the center point of the successfully matched target frame is used as the tracking point position.

[0117] Preferably, considering that target loss is common, a fault tolerance mechanism for temporary target loss can be introduced when partial matching is successful, namely:

[0118] Determine whether there is a successful matching record for the target to be tracked that failed to match in the previous preset frame:

[0119] If it exists, the midpoint of the line connecting the center point of the successfully matched target frame and the center point of the target frame of the target to be tracked that failed to match the last time is used as the tracking point position;

[0120] Specifically, considering that the stability of the algorithm may cause temporary loss of the target, a buffer fault tolerance mechanism can be established: when one of the targets fails to match within a short period of time, it is considered that the loss is due to algorithm reasons rather than the target leaving the shooting screen. The position of its last successful match is regarded as its real-time position, and the midpoint of the line connecting the center point of the target frame at the time of the last successful match and the center point of the successfully matched target frame is used as the tracking point position.

[0121] If it does not exist, the center point of the successfully matched target frame will be used as the tracking point position.

[0122] The preset frame rate can be adjusted dynamically, for example, it can be set to 40 frames or 60 frames, depending on the frame rate of the shooting device. In practice, it is generally believed that if the target is not detected for more than 1.5 to 2 seconds, the algorithm stability problem can be ruled out and it can be judged that the target has left the current screen.

[0123] Figure 2 FIG is a block diagram of an automatic tracking shooting device 400 according to an embodiment of the present application. Figure 2 As shown, the automatic tracking shooting device 400 includes a shooting module 401, an analysis module 402, and a rotation module 403. The automatic tracking shooting device 400 is used to implement the aforementioned automatic shooting and tracking method.

[0124] Optionally, the automatic tracking shooting device 400 can be an independent device, a component included in an independent device, or a combination of multiple different independent devices. In one possible implementation, the shooting module 401 is implemented as an independent smart terminal or camera, the analysis module 402 is implemented as another independent smart terminal or smart hardware, and the rotation module 403 is implemented as a pan-tilt head or a rotatable bracket. In another possible implementation, the shooting module 401 is implemented as a camera module in a smart terminal, the analysis module 402 is implemented as a computing module in the same smart terminal, and the rotation module 403 is implemented as an intelligent pan-tilt head or a rotatable bracket. In yet another possible implementation, the shooting module 401 is implemented as a camera module in an integrated intelligent camera device, the analysis module 402 is implemented as a computing module in the same integrated intelligent camera device, and the rotation module 403 is implemented as a motion mechanism in the same integrated intelligent camera device.

[0125] Among them, the shooting module 401 is physically connected to the rotation module 403, and the physical connection can be a fixed connection or a detachable connection. All connection methods that can ensure that the rotation module 403 can directly or indirectly affect the movement of the shooting module 401 by physical means are within the protection scope of this application. The shooting module 401 is suitable for continuously acquiring real-time shooting images and sending the real-time shooting images to the analysis module 402. Optionally, the shooting module 401 may not have the active information sending function, but the analysis module 402 actively acquires the real-time shooting images from the shooting module 401. The analysis module 402 is suitable for executing the method in the embodiment of the present application based on the real-time shooting images, calculating the tracking point position in real time, and generating a rotation instruction based on the tracking point position, and sending it to the rotation module 403. The rotation module 403 is suitable for horizontal rotation, that is, rotation around the vertical axis (z-axis), and is suitable for changing the angle of the shooting module 401 according to the rotation instruction.

[0126] In one implementation, when the tracking point is to the left of the screen's centerline, the rotation module is controlled to move to the left; when the tracking point is to the right of the screen's centerline, the rotation module is controlled to move to the right. Preferably, the rotation module's speed can also be controlled based on the distance between the tracking point and the screen's centerline: a greater distance indicates a greater rotation amplitude, i.e., a faster rotation speed is required.

[0127] Figure 3 A schematic diagram of the structure of a terminal device or server suitable for implementing an embodiment of the present application is shown.

[0128] like Figure 3 As shown, the terminal device or server includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 into the random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the terminal device or server are also stored. The CPU 501, ROM 502 and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0129] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output section 507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a LAN card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed, so that computer programs read therefrom can be installed into the storage section 508 as needed.

[0130] In particular, according to an embodiment of the present application, the above method flow steps can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a machine-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present application are executed.

[0131] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the aforementioned module, program segment or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0133] The units or modules described in the embodiments of the present application may be implemented in software or hardware. The units or modules described may also be provided in a processor. The names of these units or modules do not, in certain circumstances, limit the units or modules themselves.

[0134] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments, or may exist independently and not be incorporated into the electronic device. The computer-readable storage medium stores one or more programs, which, when used by one or more processors, execute the methods described in the present application.

[0135] As another aspect, an embodiment of the present application further provides a computer program product, which implements any of the above-mentioned embodiment methods when the computer program / instructions are executed by a processor.

[0136] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of application involved in this application is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the aforementioned application concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A dual-target automatic tracking and shooting method, characterized in that: include: S1: Determine two targets to be tracked in the real-time shooting picture; S2: Obtaining feature vector queues of the two targets to be tracked respectively through a human feature algorithm, and obtaining reference target frame serial numbers of the two targets to be tracked respectively through a target detection algorithm; S3: matching the target frame in each frame of the subsequent real-time shooting picture based on the feature vector queue and the reference target frame sequence number; If the match is successful or partially successful, the tracking point position is determined based on the position of the successfully matched target frame; If the matching fails, the fixed tracking point position is the center of the current image; S4: When the continuous preset frame matching fails, the focus is adjusted to expand the field of view of the real-time shooting picture.

2. The method according to claim 1, wherein The S3 step includes: S31: Detect the current frame using a target detection algorithm to obtain target frame sequence numbers of all human targets in the current frame; S32: For each target to be tracked, match the target frame serial numbers of all human targets with the reference target frame serial number of the target to be tracked.

3. The method according to claim 2, wherein The step S32 includes: S321: Acquire feature vectors of all human targets in the current frame using a human feature algorithm; For each target to be tracked, if the matching in step S32 fails, execute S322: match the feature vectors of all human targets with the feature vector queue of the target to be tracked one by one. If there is a match whose vector Euclidean distance is less than the threshold, it is considered that the matching in step S3 is successful, and the human target frame whose feature vector is successfully matched is used as the target frame with successful matching, and the reference target frame serial number is updated to the target frame serial number of the human target whose feature vector is successfully matched.

4. The method according to claim 3, wherein When there is a match whose vector Euclidean distance is less than the threshold, the following steps are performed: S3221: Increasing the weight of the corresponding feature vector that has been successfully matched in the feature vector queue; S3222: Reduce the weights of other feature vectors in the feature vector queue; S3223: Add the successfully matched feature vector to the feature vector queue and assign an initial weight.

5. The method according to claim 3, wherein The step S32 further includes: For each target to be tracked, if the match in step S32 is successful, then execute S323: determine whether the human target frame whose target frame sequence number matches successfully overlaps with other target frames: If there is no overlap, the matching in step S3 is considered successful, and the human target frame with the successfully matched target frame sequence number is regarded as the successfully matched target frame; If there is overlap, execute steps S321 and S322.

6. The method according to any one of claims 1 to 5, wherein If the match is successful or partially successful, the tracking point position is determined according to the position of the successfully matched target frame, including: If the match is successful, the midpoint of the line connecting the center points of the target frames of the two objects to be tracked is used as the tracking point position; If the partial match is successful, the center point of the successfully matched target frame is used as the tracking point position.

7. The method according to claim 6, wherein If the partial match is successful, the center point of the successfully matched target frame is used as the tracking point position, including: Determine whether there is a successful matching record for the target to be tracked that failed to match in the previous preset frame: If it exists, the midpoint of the line connecting the center point of the successfully matched target frame and the center point of the last successfully matched target frame of the target to be tracked that failed to match is taken as the tracking point position; If it does not exist, the center point of the successfully matched target frame will be used as the tracking point position.

8. An automatic tracking shooting device, characterized in that: The system comprises a shooting module, an analysis module and a rotation module, wherein the shooting module is physically connected to the rotation module, the rotation module is adapted to rotate horizontally, the shooting module is adapted to continuously acquire real-time shooting images and send the real-time shooting images to the analysis module; the analysis module is adapted to execute the method according to any one of claims 1 to 7 to calculate the position of the tracking point, generate a rotation instruction based on the position of the tracking point, and send the rotation instruction to the rotation module; the rotation module is adapted to change the angle of the shooting module according to the rotation instruction.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images