Tracking and shooting method and device based on binocular camera, equipment and medium

Abstract

The embodiment of the invention discloses a tracking and shooting method and device based on a binocular camera, equipment and a medium, and relates to the field of artificial intelligence, and the method comprises the steps: shooting a first video stream through a first camera; performing image analysis on the first video stream to identify a target object in the first video stream, and continuously calculating position information of the target object; based on the position information of the target object, controlling the second camera to carry out orientation, so that the second camera is continuously aligned with the target object and carries out shooting to obtain a second video stream; and continuously storing the first video stream and the second video stream. According to the invention, the problems of tracking stability and picture definition in moving target shooting can be effectively solved without manual intervention, and the shooting quality and the automation degree are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a tracking and shooting method and device based on a binocular camera, equipment and a medium. BACKGROUND

[0002] In the current field of sports shooting and video recording, users often need to continuously follow and shoot targets (such as people or objects) in motion.

[0003] Traditional shooting methods usually rely on manual holding or fixed positions, which are difficult to maintain stable tracking when the target moves quickly, and are likely to cause the target to be lost or the picture to be blurred, affecting the shooting effect. In addition, ordinary camera equipment often cannot balance the picture field of view and detail clarity when shooting distant targets, even if it has zoom function, it is often difficult to capture high-quality close-up pictures due to focusing speed or resolution limitations.

[0004] Therefore, the prior art urgently needs an overall solution that can automatically complete target tracking and zoom shooting without human intervention. SUMMARY

[0005] The embodiments of the present application provide a tracking and shooting method, device, equipment and medium based on a binocular camera, and aim to solve the technical problem of how to automatically complete target tracking and zoom shooting without human intervention.

[0006] In a first aspect, the embodiments of the present application provide a tracking and shooting method based on a binocular camera, which includes a first camera for shooting a first video stream and a second camera for shooting a second video stream, and the method includes: shooting a first video stream by the first camera; performing image analysis on the first video stream to identify a target object in the first video stream and continuously calculate the position information of the target object; based on the position information of the target object, controlling the second camera to be oriented so that the second camera continuously aims at the target object and shoots to obtain a second video stream; continuously storing the first video stream and the second video stream.

[0007] Further technical solutions are that the method further includes: real-time analysis of video content and audio information collected by the binocular camera to detect a preset type of highlight event; based on the detected highlight event, extracting a corresponding video segment from the first video stream and / or the second video stream, and automatically editing to generate a target video.

[0008] Further, the technical solutions are as follows: the image analysis on the first video stream is performed to identify a target object in the first video stream, including: image features of at least one moving object are extracted by analyzing image frames of the first video stream, and the image features include contour, color and texture features; the image features are input into a pre-trained identification model to identify the target object by the identification model.

[0009] Further, the technical solutions are as follows: the position information of the target object is continuously calculated, including: motion vectors of pixel points of the target object in continuous image frames of the first video stream are calculated by an optical flow method; a motion model of the target object is established based on the motion vectors; a position of the target object in a subsequent image frame of the first video stream is predicted according to the motion model, and the predicted position is corrected by a Kalman filtering algorithm to obtain the position information.

[0010] Further, the technical solutions are as follows: the highlight event includes: a specific gesture or action of the target object is identified; a specific trajectory or relative position of the target object is identified; a motion speed of the target object in a picture exceeds a threshold value is detected; a specific sound pattern in an environmental sound is identified; a shooting instruction sent by a user through a remote control device is received.

[0011] Further, the technical solutions are as follows: the specific gesture or action of the target object is identified, including: contour features of the target object in continuous image frames of the first video stream or a second video stream are extracted; a motion trajectory of a key joint is obtained by identifying a position change of the key joint through establishment of a skeleton model of the target object; the motion trajectory of the key joint is matched with a pre-stored standard action template, and when a matching degree exceeds a set threshold value, it is determined that the specific gesture or action is identified.

[0012] Further, the technical solutions are as follows: the target video is automatically generated by clipping, including: a plurality of video clips extracted are spliced in time sequence; transition special effects are added between adjacent video clips; background music is matched or added for the generated video.

[0013] In a second aspect, the embodiments of the present application also provide a tracking and shooting device based on a binocular camera, which comprises units for executing the above method.

[0014] In a third aspect, the embodiments of the present application also provide a computer device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the above method when executing the computer program.

[0015] In a fourth aspect, the embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program can implement the above method when executed by a processor.

[0016] The embodiments of the present application provide a tracking and shooting method, device, equipment and medium based on a binocular camera. The method comprises: shooting a first video stream through a first camera; performing image analysis on the first video stream to identify a target object in the first video stream and continuously calculating position information of the target object; controlling a second camera to be oriented based on the position information of the target object, so that the second camera continuously aims at the target object and shoots to obtain a second video stream; and continuously storing the first video stream and the second video stream. The present application realizes full-process automatic shooting through the cooperative working mechanism of the binocular camera. The first camera shooting the first video stream ensures that the target is always within the monitoring range, avoiding tracking loss; the continuous calculation of the target position based on image analysis provides accurate guidance for zoom control of the second camera; the second camera automatically adjusts the direction and focal length according to the position information, synchronously capturing close-up pictures of the target; and the continuous storage of the double video streams provides complete data support for post-processing. The method effectively solves the problems of tracking stability and picture clarity in shooting of a moving target without manual intervention, significantly improving the shooting quality and automation degree. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 A flowchart of a tracking and shooting method based on a binocular camera provided by the embodiments of the present application is shown. Figure 2 A schematic block diagram of a computer device provided by the embodiments of the present application is shown. DETAILED DESCRIPTION

[0019] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described, obviously, the described embodiments are a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0020] It should be understood that the terms "comprise" and "include" as used in the specification and the appended claims indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0021] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] It should be further understood that the term "and / or" as used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0023] As used in the present application specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrases "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [a described condition or event]" or "in response to detecting [a described condition or event]" depending on the context.

[0024] Referring to Figure 1 The embodiments of the present application provide a tracking and shooting method based on a binocular camera, the binocular camera comprising a first camera for shooting a first video stream and a second camera for shooting a second video stream, the method comprising the following steps: S1, shooting a first video stream by the first camera.

[0025] In a specific implementation, the first video stream is captured by a first camera, which can be a wide-angle camera, for example, and the present application does not specifically limit it. This step ensures complete coverage of the shooting scene, so that the target object can still be kept in the monitoring field of view when it moves in a larger range, effectively avoiding the phenomenon of losing the target due to rapid movement or sudden turning. For example, in sports event shooting, a wide-angle camera can continuously capture the movement trajectory of the athlete in the entire field, providing a complete scene context for subsequent processing.

[0026] S2, image analysis is performed on the first video stream to identify the target object in the first video stream, and the position information of the target object is continuously calculated.

[0027] In a specific implementation, in the process of image analysis of the first video stream to identify the target object, the system can accurately distinguish the target from the background interference by analyzing the motion features and apparent features in the video sequence. This image content-based analysis method ensures the reliability of target recognition. The step of continuously calculating the position information of the target object further enhances the tracking ability of the system. By establishing a target motion model and updating the position data in real time, the system can predict the movement trend of the target, thereby providing accurate guidance for the control of the second camera. This continuous position calculation mechanism enables the system to maintain stable tracking effect when facing complex motion patterns such as target speed change and direction change.

[0028] In some preferred embodiments, the above step of "image analysis of the first video stream to identify the target object in the first video stream" specifically includes the following steps: by analyzing the image frames of the first video stream, image features of at least one moving object are extracted, including contour, color and texture features; the image features are input into a pre-trained recognition model to identify the target object by the recognition model.

[0029] In a specific implementation, the target detection method based on image feature extraction and pre-trained recognition model improves the recognition accuracy and robustness of the system in complex environments. Specifically, the system extracts multi-dimensional image features such as contour, color and texture of moving objects by analyzing the image frames of the first video stream. These features can effectively characterize the visual attributes of the target and to some extent resist the influence of factors such as light changes and background interference. For example, contour features can be used to distinguish between the human body and the surrounding environment, color features can help track a specific dressed target in a multi-person scene, and texture features can further improve the granularity of recognition.

[0030] Subsequently, the system inputs the extracted image features into a pre-trained recognition model, which is trained by a large amount of data and can efficiently classify and recognize the input features, so as to accurately determine the category and position of the target object. This method of combining low-level image features and high-level model reasoning enables the system to adapt to different shooting scenes and target types, such as recognizing athletes in outdoor sports or recognizing specific animals in pet following. This method significantly improves the accuracy and adaptability of target recognition while ensuring real-time performance through the synergistic effect of features and models.

[0031] In some preferred embodiments, the above step "continuously calculating the position information of the target object" specifically includes the following steps: calculating the motion vector of the pixel point of the target object in the continuous image frames of the first video stream by the optical flow method; establishing a motion model of the target object based on the motion vector; predicting the position of the target object in the subsequent image frames of the first video stream according to the motion model, and correcting the predicted position by the Kalman filtering algorithm to obtain the position information.

[0032] In specific implementation, by using the motion estimation method combining the optical flow method and the Kalman filtering, high-precision prediction and stable tracking of the position information of the target object are realized. The optical flow method can accurately reflect the short-term motion trend of the target object by calculating the motion vector of the pixel point in the continuous image frames, and provides data basis for establishing the motion model. The motion model constructed based on the motion vector can further describe the motion law of the target, such as uniform speed, acceleration or curve motion, thereby supporting reasonable prediction of its future position.

[0033] However, due to the influence of factors such as image noise and occlusion, simply relying on prediction may lead to error accumulation. Therefore, the system introduces the Kalman filtering algorithm to correct the predicted position. This algorithm dynamically adjusts the estimation result by combining the predicted value and the actual observation value, significantly reducing the uncertainty of position estimation. For example, after the target is temporarily occluded, the system can still recover its accurate position through Kalman filtering, avoiding tracking loss. This method of combining short-term motion analysis and long-term state estimation enables the system to maintain the continuity and stability of tracking in complex scenes, and is particularly suitable for applications where the target moves quickly or is disturbed.

[0034] S3, based on the position information of the target object, controlling the second camera to be oriented so that the second camera continuously aims at the target object and takes pictures to obtain a second video stream.

[0035] In a specific implementation, the step of controlling the second camera to orient and zoom based on the target position information is one of the core innovations of the present application. The second camera can be a zoom camera, and the present application does not make specific limitations. Alternatively, the second camera can be a fixed-focus camera, and the present application does not make specific limitations. The second camera automatically adjusts the shooting direction and focal length based on the position data provided by the first camera, always keeps the target object in the center of its image, and obtains a close-up image of the target object through optical zoom. This dual-camera cooperative working mechanism allows the system to simultaneously consider the overall scene and local details. For example, in stage performance photography, a wide-angle lens can be used to show the overall stage effect, and a zoom lens can be used to capture the facial expressions and subtle movements of the performers.

[0036] In an embodiment, the implementation of the above step S3 is as follows: First, a coordinate mapping relationship between the image coordinate system of the first camera and the gimbal control system of the second camera is established. The correspondence between the field of view of the first camera and the gimbal rotation angle of the second camera is determined through calibration processing, so that any pixel position in the first camera image can be converted into the azimuth and elevation control parameters required to drive the second camera gimbal to rotate.

[0037] Further, a control instruction is generated based on the real-time position information of the target object. The current position coordinates of the target object in the first camera image coordinate system are converted into corresponding gimbal control parameters through the coordinate mapping relationship, and are output to the gimbal driving mechanism of the second camera, so that the shooting direction of the second camera is continuously aligned with the target object.

[0038] Further, based on the size change of the target object in the image or the preset shooting requirement (for example, the area occupied by the target object in the image needs to be within a preset proportion range, for example, greater than 50%), the required zoom factor is calculated. By analyzing the pixel size change trend of the target object in consecutive image frames, or according to the user's preset shooting close-up level, the optical zoom factor of the second camera is determined, and a corresponding zoom control signal is generated to drive the zoom motor of the second camera to perform zoom operation.

[0039] Further, during tracking, the motion state of the target object is continuously monitored, and the control parameters are dynamically adjusted. According to the motion speed and direction of the target object, the possible position of the target object at the next moment is predicted, and the pointing and zoom parameters of the second camera are adjusted in advance accordingly, to ensure smooth tracking when the target object moves quickly, and to avoid the target object moving out of the shooting range of the second camera.

[0040] Furthermore, a closed-loop control mechanism is used to correct tracking errors in real time. The position of the target object in the actual captured image of the second camera is compared with the expected position. The gimbal control parameters and zoom parameters are dynamically adjusted according to the position deviation to form a feedback control loop, ensuring that the second camera can continuously and stably aim at the target object and perform high-definition shooting.

[0041] S4, continuously store the first video stream and the second video stream.

[0042] In practice, the continuous storage of dual video streams provides ample data support for subsequent video processing and applications. The first video stream records the target's movement trajectory and surrounding environmental information, while the second video stream provides high-definition details of the target. This dual recording method allows users to flexibly choose the viewing angle as needed later, or to fuse the video footage from the two perspectives. Furthermore, the simultaneous storage of dual video streams also provides a data foundation for multi-view video analysis.

[0043] In some preferred embodiments, the method further includes the following steps: real-time analysis of video content and audio information captured by the binocular camera, detection of key events of a preset type; based on the detected key events, extraction of corresponding video segments from the first video stream and / or the second video stream, and automatic editing to generate a target video.

[0044] In its implementation, an event-detection-based automatic video editing function was introduced, further enhancing the intelligence and efficiency of video content production. By analyzing the video content and audio information (first and second video streams) captured by the binocular cameras in real time, the system can automatically identify preset types of exciting events, such as high-difficulty movements in sports, celebratory moments, or cheers in the environment. This multimodal event detection mechanism allows the system to accurately capture highlights that users may be interested in, without relying on manual labeling or post-production review of all materials. After detecting an exciting event, the system automatically extracts the corresponding segments from the stored first and second video streams and intelligently edits and splices them according to the event type to generate a coherent and smooth target video. For example, in a skiing scene, the system can automatically extract video segments before, during, and after an athlete completes a jump, and combine them into a complete highlight reel. This method not only significantly reduces the time cost and technical threshold for post-production editing but also ensures the relevance and watchability of the output content through an event-driven editing strategy, achieving fully automated processing from shooting to final product.

[0045] In some preferred embodiments, the highlight event comprises: recognizing a specific gesture or action made by the target object; recognizing a specific trajectory or relative position of the target object; detecting that the motion speed of the target object in the picture exceeds a threshold; recognizing a specific sound pattern in the environmental sound; receiving a shooting instruction sent by the user through a remote control device.

[0046] In specific implementation, by defining the detection conditions of multiple types of highlight events, the perception dimension and response capability of the system in video content analysis are expanded. Specifically, the system can capture the behavior highlights of the target object by recognizing specific gestures or actions of the target object, such as recognizing jumping actions in dance shooting or recording standard technical actions in sports training; by recognizing specific trajectories or relative positions of the target object, such as recognizing the motion path with typical characteristics such as the parabolic trajectory of shooting in sports events or the trajectory of shooting in sports events. At the same time, by monitoring the relative position relationship between multiple target objects in real time, the system can judge the spatial configuration changes with specific meanings, such as the key events of the football crossing the goal line and the basketball passing through the basket; by detecting that the motion speed of the target object in the picture exceeds a threshold, the high-speed motion instant can be automatically marked, such as the car rushing to the line or the athlete sprinting; by recognizing specific patterns in the environmental sound, the system can respond to acoustic events in the scene, such as audience applause, explosion sound or instrument playing, thereby assisting the judgment of highlight moments in the audio level; in addition, by receiving the shooting instruction sent by the user through the remote control device, the system also supports manual intervention and personalized marking, so that the user can actively trigger recording at the key moment. This multi-modal event detection mechanism enables the system to comprehensively cover multiple information dimensions such as vision, motion and hearing, thereby flexibly adapting to user needs in different application scenarios and improving the comprehensiveness and accuracy of event detection.

[0047] Specifically, in an embodiment: The recognition method of the specific gesture or action of the target object is as follows: first, the contour features of the target object are extracted through image segmentation technology, and on this basis, a human body skeleton joint model is established to track the spatial position changes of the key joints in real time. By analyzing the relative position relationship and motion trajectory between the joints in the continuous frame sequence, the motion parameters representing the gesture features are extracted. The motion parameters are matched with the pre-stored standard action templates through dynamic time warping, and when the similarity exceeds a set threshold, it is determined that the specific gesture or action is recognized.

[0048] Further, the recognition method of the specific trajectory or relative position of the target object is as follows: Firstly, motion trajectory analysis is performed to sample and fit the position sequence of the target object in consecutive image frames. A kinematic model is established to describe the motion trajectory characteristics of the target object, including trajectory curvature, motion acceleration, and azimuth rate of change, etc. Polynomial fitting or spline interpolation methods are used to smooth the discrete position points, obtaining a continuous motion trajectory curve. The motion trajectory curve is compared with pre-stored typical trajectory templates, including parabolic trajectory, straight line trajectory, and specific turning mode. For the recognition of relative position relationship, a spatial relationship model between multiple target objects is established. Through the conversion relationship from image coordinates to world coordinates, the relative distance, relative angle, and position relationship between different target objects are calculated. The key area boundary markers are set, and when the position relationship between the target object and the key area meets the preset conditions, the corresponding event judgment is triggered. For example, by establishing a spatial model of the goal area, when the whole football crosses the goal line, it is determined as a goal event; by simultaneously monitoring the player's posture and the position of the racket, when the specific spatial configuration relationship is met, it is determined as a smash action. In the implementation process, a multi-target tracking algorithm is used to maintain the identity of different target objects, ensuring the continuity of the trajectory data and the position relationship. Through prediction algorithms such as Kalman filtering, the motion state of the target object is short-term predicted, improving the accuracy and timeliness of trajectory recognition. At the same time, a confidence evaluation mechanism is established to evaluate the reliability of the recognition result, and when the confidence reaches a predetermined threshold, the highlight event marker is triggered.

[0049] Further, the detection method of the target object motion speed is as follows: based on the position coordinate change of the target in consecutive image frames, the instantaneous motion speed in the image coordinate system is calculated. Through coordinate transformation, the pixel speed in the image coordinate system is converted into the motion speed in the actual physical space, and the speed value is compared with the preset speed threshold. When it is detected that the motion speed of the target object continuously exceeds the threshold for a predetermined length of time, it is determined that a high-speed motion event has occurred.

[0050] Further, the recognition method of specific sound patterns in environmental sound is as follows: environmental audio signals are collected through a microphone array, and the audio signals are preprocessed and feature extracted to obtain the frequency spectrum features and time domain features of the sound. Pattern recognition methods are used to match and identify the sound features with pre-stored typical sound templates, including cheers, applause, specific passwords, etc. When the matching degree with any sound template reaches the confidence threshold, it is determined that a specific sound pattern event is detected.

[0051] Further, the user remote control shooting instruction receiving mode is as follows: receiving a control signal sent by a user operating a remote control device through a wireless communication module, the control signal being transmitted using a specific encoding protocol. The received signal is decoded and verified to confirm the validity and type of the instruction. When a valid shooting trigger instruction is identified, a corresponding highlight event marker is immediately generated and the event is stored in association with a current timestamp.

[0052] The above-mentioned highlight event detection modules operate independently and in parallel, and when any module detects a highlight event of a corresponding type, a uniform format event record is generated, including event type, timestamp, confidence, and other metadata. All detected events are recorded in an event log, providing a basis for subsequent video segment extraction and editing.

[0053] In some preferred embodiments, the above step of "identifying a specific gesture or action made by the target object" specifically includes the following steps: extracting contour features of the target object in consecutive image frames of the first video stream or the second video stream; identifying position changes of key joints by establishing a skeleton model of the target object to obtain motion trajectories of the key joints; and matching the motion trajectories of the key joints with pre-stored standard action templates, and determining that a specific gesture or action is identified when the matching degree exceeds a set threshold.

[0054] In a specific implementation, by introducing a gesture recognition method based on skeleton model and action template matching, fine analysis and accurate determination of complex actions of the target object are achieved. The system first extracts contour features of the target object in consecutive image frames to preliminarily determine its shape range and posture change. On this basis, by establishing a skeleton model of the target object, the system can further identify position changes of key joints (such as shoulders, elbows, knees, etc.) and construct motion trajectories of the joints. This skeleton representation method abstracts complex shape actions into spatiotemporal motion of the joints, significantly reducing the dimension and computational complexity of action analysis. Subsequently, the system matches the joint motion trajectories with pre-stored standard action templates, determines whether a specific gesture or action is identified by calculating the similarity between the two. For example, in fitness follow-up shooting, the system can automatically record the user's training actions and detect their completion quality by matching standard action templates such as squats and push-ups. This method realizes in-depth understanding of action structure and dynamic characteristics through layer-by-layer analysis from contour to skeleton and from trajectory to template, improving the accuracy and interpretability of action recognition.

[0055] Specifically, in an embodiment, the specific implementation of the specific gesture or action recognition includes the following steps: Firstly, a contour feature extraction process is performed, and a foreground region of a target object is separated from continuous image frames by a background difference method or an inter-frame difference method. The foreground region is subjected to a morphological process to eliminate noise interference, and then a complete contour feature of the target object is extracted by an edge detection algorithm. Convex hull analysis is performed based on the contour feature to determine a body boundary key point of the target object, thereby providing basic data for skeleton modeling.

[0056] Further, a skeleton model of the target object is established, a thinning algorithm is used to iteratively erode the contour feature, and outer layer pixels are gradually stripped until a skeleton structure with a single-pixel width is obtained. Key joint points, including limb end points, branch points and center points, are identified based on the skeleton structure, and a topological structure describing the connection relationship of the joint points is established. A complete skeleton model representation is constructed by analyzing the relative positions and connection relationships between adjacent joint points.

[0057] Further, joint motion trajectory tracking is performed to maintain the consistent correspondence of the joint points in a sequence of continuous image frames. A trajectory association algorithm based on motion coherence is used to establish a trajectory tracking chain according to the position distance, motion direction and speed characteristics of the joint points in adjacent frames. For trajectory interruption caused by temporary occlusion, trajectory prediction and re-association mechanisms are used to restore tracking continuity, thereby forming stable joint motion trajectory data.

[0058] Further, an action template matching operation is performed to calculate the similarity between the obtained joint motion trajectory and a pre-stored standard action template. The standard action template is obtained by statistical learning of a plurality of sample sequences, and includes time sequence data of joint positions and corresponding allowable deviation ranges. A first feature vector of the standard action template is obtained, and a second feature vector of the joint motion trajectory is obtained, a cosine similarity between the first feature vector and the second feature vector is calculated, and a similarity score is further determined based on the cosine similarity.

[0059] Further, action determination is performed according to the matching result, and when the similarity score exceeds a set threshold, a specific gesture or action is confirmed to be recognized. The category identifier, occurrence timestamp and confidence score of the action are recorded, and a corresponding action recognition event signal is generated. The system also includes a template updating mechanism that adaptively optimizes the standard action template based on continuous successful recognition results, thereby improving the accuracy and adaptability of subsequent recognition.

[0060] During the entire recognition process, the system continuously monitors the stability of the skeleton model and the tracking quality of the joint points, and when model degradation or tracking failure is detected, a re-initialization process is automatically triggered to ensure the continuity and reliability of action recognition.

[0061] In some preferred embodiments, the above step of "automatically generating the target video through editing" specifically comprises the following steps: splicing the extracted multiple video clips in chronological order; adding transition effects between adjacent video clips; and matching or adding background music for the generated video.

[0062] In specific implementation, by specifying the specific process of automatically generating the target video through editing, the professional nature and completeness of the output video in terms of content organization and audio-visual experience are ensured. The system first splices the extracted multiple video clips in chronological order, thereby constructing a narrative structure that conforms to the logic of event occurrence, avoiding perceptual discontinuity caused by chaotic clip order. On this basis, the system adds transition effects, such as fade-in, fade-out, sliding, or zooming, between adjacent video clips, making the switching between different scenes more natural and smooth, and improving the coherence and visual appeal of the video. In addition, the system matches or adds background music for the generated video, enhancing the emotional expression and atmosphere rendering of the content through audio-visual combination. For example, when generating a sports highlight, the system can select music with a lively rhythm to highlight the dynamic feeling; when generating a parent-child daily video, it can be matched with a warm and soothing background music. This automated editing scheme that integrates clip splicing, transition processing, and music accompaniment not only significantly reduces the user's operation burden, but also ensures the artistic quality of the output content through technical means, enabling ordinary users to easily produce video works with a professional feel.

[0063] The embodiment of the present application proposes a tracking and shooting method based on a binocular camera. The binocular camera includes a first camera for shooting a first video stream and a second camera for shooting a second video stream. The method comprises: shooting a first video stream through a first camera; performing image analysis on the first video stream to identify a target object in the first video stream and continuously calculating the position information of the target object; based on the position information of the target object, controlling the second camera to be oriented so that the second camera continuously aims at the target object and shoots, obtaining a second video stream; continuously storing the first video stream and the second video stream. The present application realizes full-process automatic shooting through the cooperative working mechanism of the binocular camera. The first camera shooting the first video stream ensures that the target is always within the monitoring range, avoiding tracking loss; based on image analysis, the position of the target is continuously calculated to provide accurate guidance for zoom control of the second camera; the second camera automatically adjusts the direction and focal length according to the position information, synchronously capturing close-up pictures of the target; the continuous storage of the dual video stream provides complete data support for post-processing. Without human intervention, the method effectively solves the problems of tracking stability and picture clarity in moving target shooting, significantly improving the shooting quality and automation level.

[0064] Corresponding to the above tracking and shooting method based on the binocular camera, the application also provides a tracking and shooting device based on the binocular camera. The tracking and shooting device based on the binocular camera includes units for executing the above tracking and shooting method based on the binocular camera, and can be configured in a desktop computer, a tablet computer, a laptop computer, and the like. Specifically, the tracking and shooting device based on the binocular camera includes: a shooting unit for shooting a first video stream through a first camera; a recognition unit for performing image analysis on the first video stream to identify a target object in the first video stream and continuously calculate position information of the target object; a following unit for controlling the second camera to be oriented based on the position information of the target object, so that the second camera continuously aims at the target object and shoots to obtain a second video stream; a storage unit for continuously storing the first video stream and the second video stream.

[0065] In some preferred embodiments, the application further includes: a detection unit for analyzing video content and audio information collected by the binocular camera in real time to detect a preset type of highlight event; an extraction unit for extracting a corresponding video segment from the first video stream and / or the second video stream based on the detected highlight event, and automatically generating a target video through editing.

[0066] In some preferred embodiments, the image analysis on the first video stream to identify the target object in the first video stream includes: extracting image features of at least one moving object by analyzing image frames of the first video stream, the image features including contour, color, and texture features; inputting the image features into a pre-trained recognition model to identify the target object by the recognition model.

[0067] In some preferred embodiments, the continuous calculation of the position information of the target object includes: calculating a motion vector of a pixel point of the target object in consecutive image frames of the first video stream by an optical flow method; establishing a motion model of the target object based on the motion vector; predicting a position of the target object in a subsequent image frame of the first video stream according to the motion model, and correcting the predicted position by a Kalman filtering algorithm to obtain the position information.

[0068] In some preferred embodiments, the highlight event includes: identifying that the target object makes a specific gesture or action; identifying that the target object has a specific trajectory or relative position; detecting that the motion speed of the target object in the picture exceeds a threshold value; identifying a specific sound pattern in the ambient sound; receiving a shooting instruction sent by the user through the remote control device.

[0069] In some preferred embodiments, the identifying that the target object makes a specific gesture or action specifically includes: extracting the contour features of the target object in the continuous image frames of the first video stream or the second video stream; establishing a skeleton model of the target object to identify the position changes of the key joints and obtain the motion trajectory of the key joints; matching the motion trajectory of the key joints with a pre-stored standard action template, and determining that a specific gesture or action is identified when the matching degree exceeds a set threshold.

[0070] In some preferred embodiments, the automatic clipping generates a target video, including: splicing the extracted multiple video clips in chronological order; adding transition effects between adjacent video clips; matching or adding background music for the generated video.

[0071] It should be noted that those skilled in the art can clearly understand the specific implementation process of the above-mentioned tracking and shooting device based on the binocular camera and each unit, which can refer to the corresponding description in the foregoing method embodiments. For the convenience and brevity of description, it will not be repeated here.

[0072] The above-mentioned tracking and shooting device based on the binocular camera can be realized in the form of a computer program, which can run on a computer device as shown in Figure 2 .

[0073] Please refer to Figure 2 , Figure 2 is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 500 can be a terminal or a server, wherein the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a personal digital assistant, and a wearable device, etc. The server can be a stand-alone server or a server cluster composed of multiple servers.

[0074] The computer device 500 comprises a processor 502, a memory and a network interface 505 connected by a system bus 501, wherein the memory can comprise a non-volatile storage medium 503 and an internal memory 504.

[0075] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032, when executed, can cause the processor 502 to perform a tracking and shooting method based on a binocular camera.

[0076] The processor 502 is configured to provide computing and control capabilities to support the operation of the entire computer device 500.

[0077] The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503, and the computer program 5032, when executed by the processor 502, can cause the processor 502 to perform a tracking and shooting method based on a binocular camera.

[0078] The network interface 505 is configured to perform network communication with other devices. Those skilled in the art can understand that the above structure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device 500 to which the scheme of the present application is applied. The specific computer device 500 can comprise more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0079] The processor 502 is configured to run the computer program 5032 stored in the memory to implement the following steps: shooting a first video stream through a first camera; performing image analysis on the first video stream to identify a target object in the first video stream and continuously calculating position information of the target object; controlling the second camera to be oriented based on the position information of the target object, so that the second camera continuously aims at the target object and shoots to obtain a second video stream; continuously storing the first video stream and the second video stream.

[0080] In some preferred embodiments, the method further comprises: analyzing video content and audio information collected by the binocular camera in real time to detect a preset type of highlight event; extracting a corresponding video segment from the first video stream and / or the second video stream based on the detected highlight event, and automatically generating a target video by editing.

[0081] In some preferred embodiments, the image analysis on the first video stream to identify a target object in the first video stream comprises: extracting image features of at least one moving object in the first video stream, the image features comprising contour, color and texture features; inputting the image features into a pre-trained recognition model to identify the target object by the recognition model.

[0082] In some preferred embodiments, the continuous calculation of the position information of the target object comprises: calculating a motion vector of a pixel point of the target object in consecutive image frames of the first video stream by an optical flow method; establishing a motion model of the target object based on the motion vector; predicting a position of the target object in a subsequent image frame of the first video stream according to the motion model, and correcting the predicted position by a Kalman filtering algorithm to obtain the position information.

[0083] In some preferred embodiments, the highlight event comprises: identifying that the target object makes a specific gesture or action; identifying a specific trajectory or relative position of the target object; detecting that a motion speed of the target object in the picture exceeds a threshold; identifying a specific sound pattern in the environmental sound; receiving a shooting instruction sent by a user through a remote control device.

[0084] In some preferred embodiments, the identification that the target object makes a specific gesture or action specifically comprises: extracting contour features of the target object in consecutive image frames of the first video stream or the second video stream; identifying a position change of a key joint by establishing a skeleton model of the target object to obtain a motion trajectory of the key joint; matching the motion trajectory of the key joint with a pre-stored standard action template, and determining that a specific gesture or action is identified when a matching degree exceeds a set threshold.

[0085] In some preferred embodiments, the automatic clipping generates a target video, comprising: splicing the extracted multiple video clips in chronological order; adding a transition special effect between adjacent video clips; matching or adding background music for the generated video.

[0086] It should be understood that, in the embodiments of the present application, the processor 502 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0087] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to realize the process steps of the above-mentioned embodiments.

[0088] Therefore, the present application also provides a storage medium. The storage medium can be a computer-readable storage medium. The storage medium stores a computer program. The computer program is executed by a processor to make the processor perform the following steps: capturing a first video stream by a first camera; performing image analysis on the first video stream to identify a target object in the first video stream and continuously calculate position information of the target object; controlling the second camera to be oriented based on the position information of the target object, so that the second camera continuously aims at the target object and captures a second video stream; continuously storing the first video stream and the second video stream.

[0089] In some preferred embodiments, the method further comprises: analyzing video content and audio information collected by the binocular camera in real time to detect a preset type of highlight event; extracting a corresponding video segment from the first video stream and / or the second video stream based on the detected highlight event, and automatically generating a target video by editing.

[0090] In some preferred embodiments, the image analysis on the first video stream to identify a target object in the first video stream comprises: extracting image features of the at least one moving object by analyzing image frames of the first video stream, the image features comprising contour, color and texture features; inputting the image features into a pre-trained recognition model to recognize the target object by the recognition model.

[0091] In some preferred embodiments, the continuously calculating the position information of the target object comprises: calculating a motion vector of a pixel point of the target object in consecutive image frames of the first video stream by an optical flow method; establishing a motion model of the target object based on the motion vector; predicting a position of the target object in a subsequent image frame of the first video stream according to the motion model, and correcting the predicted position by a Kalman filtering algorithm to obtain the position information.

[0092] In some preferred embodiments, the highlight event comprises: recognizing that the target object makes a specific gesture or action; recognizing a specific trajectory or relative position of the target object; detecting that a motion speed of the target object in the picture exceeds a threshold value; recognizing a specific sound pattern in the environmental sound; receiving a shooting instruction sent by a user through a remote control device.

[0093] In some preferred embodiments, the recognizing that the target object makes a specific gesture or action specifically comprises: extracting contour features of the target object in consecutive image frames of the first video stream or the second video stream; recognizing a position change of a key joint by establishing a skeleton model of the target object to obtain a motion trajectory of the key joint; matching the motion trajectory of the key joint with a pre-stored standard action template, and determining that a specific gesture or action is recognized when a matching degree exceeds a set threshold value.

[0094] In some preferred embodiments, the automatically clipping to generate a target video comprises: splicing the extracted multiple video clips in chronological order; adding a transition special effect between adjacent video clips; matching or adding background music for the generated video.

[0095] The storage medium is a physical, non-transient storage medium, for example, can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various physical storage media that can store program codes. The computer-readable storage medium can be non-volatile or volatile.

[0096] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description in a general manner. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0097] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0098] The steps in the method embodiments of the present application can be adjusted, combined and deleted in sequence according to actual needs. The units in the device embodiments of the present application can be combined, divided and deleted according to actual needs. In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0099] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a terminal or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application.

[0100] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0101] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

[0102] The above descriptions are only the specific embodiments of the application, but the protection scope of the application is not limited to this. Any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the application, and these modifications or replacements should be covered in the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. A tracking and shooting method based on a binocular camera, characterized in that, The binocular camera includes a first camera for capturing a first video stream and a second camera for capturing a second video stream, the method comprising: The first video stream is captured using the first camera; Image analysis is performed on the first video stream to identify target objects in the first video stream, and the location information of the target objects is continuously calculated; Based on the location information of the target object, the second camera is controlled to orient itself so that it continuously points at the target object and captures images, thereby obtaining a second video stream; The first video stream and the second video stream are continuously stored.

2. The tracking and shooting method based on a binocular camera according to claim 1, characterized in that, The method further includes: Real-time analysis of video content and audio information captured by binocular cameras to detect key events of preset types; Based on the detected exciting events, corresponding video segments are extracted from the first video stream and / or the second video stream, and automatically edited to generate the target video.

3. The tracking and shooting method based on a binocular camera according to claim 1, characterized in that, The step of performing image analysis on the first video stream to identify target objects in the first video stream includes: By analyzing the image frames of the first video stream, image features of at least one moving object are extracted, including contour, color, and texture features; The image features are input into a pre-trained recognition model so that the recognition model can identify the target object.

4. The tracking and shooting method based on a binocular camera according to claim 1, characterized in that, The continuous calculation of the target object's location information includes: The motion vectors of the target object's pixels in consecutive image frames of the first video stream are calculated using optical flow. A motion model of the target object is established based on the motion vectors; The position of the target object in subsequent image frames of the first video stream is predicted based on the motion model, and the predicted position is corrected by a Kalman filter algorithm to obtain the position information.

5. The tracking and shooting method based on a binocular camera according to claim 2, characterized in that, The exciting events mentioned include: It can identify specific poses or movements made by the target object. Identify the specific trajectory or relative position of the target object; The target object's movement speed in the frame exceeds a threshold; Identify specific sound patterns in ambient sounds; Received a shooting command from the user via remote control device.

6. The tracking and shooting method based on a binocular camera according to claim 5, characterized in that, The identification of a target object making a specific pose or movement specifically includes: Extract the contour features of the target object in consecutive image frames of the first or second video stream; By establishing a skeleton model of the target object, identifying the positional changes of key joints, and obtaining the motion trajectory of the key joints; The motion trajectory of the key joint is matched with the pre-stored standard action template. When the matching degree exceeds the set threshold, it is determined that a specific posture or action has been identified.

7. The tracking and shooting method based on a binocular camera according to claim 2, characterized in that, The automatic editing to generate the target video includes: The extracted video clips are spliced ​​together in chronological order; Add transition effects between adjacent video clips; Match or add background music to the generated video.

8. A tracking and shooting device based on a binocular camera, characterized in that, Includes a unit for performing the method as described in any one of claims 1-7.

9. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-7.

Images