A method and system for shooting a video of a trajectory of an object

Abstract

The application belongs to the technical field of motion trajectory shooting, and particularly provides a kind of object motion trajectory video shooting method and system, which comprises the following steps: S1, collecting the motion picture of a target object, generating a picture queue composed of multiple frames of images according to the motion picture, and screening multiple trajectory generation pictures from the picture queue; S2, identifying the position of the target object on the trajectory generation picture and obtaining the contour of the target object; S3, cutting the target object on the trajectory generation picture according to the contour, taking the motion posture picture of the target object obtained by cutting as a trajectory picture, and pasting each trajectory picture on the corresponding position of the image after the time point of the trajectory generation picture in the picture queue, to generate a motion trajectory picture of the target object. The application presents the posture of several motion points in the competition process of an athlete in front of the audience, so that the audience can more deeply and intuitively understand the competition state of the athlete.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of motion trajectory shooting, in particular to a motion trajectory picture shooting method and system. BACKGROUND

[0002] In the process of broadcasting competitive sports, how to present the athlete's competition picture from multiple angles in front of the television audience is one of the key points of the breakthrough of the broadcasting technology. The traditional broadcasting method cannot simultaneously show the motion trajectory of the athlete at each time node, and it is difficult to enable the audience to more deeply and intuitively understand the competition state of the athlete. SUMMARY

[0003] The purpose of the present application is also to provide an object motion trajectory video shooting method and system to solve the above technical problems.

[0004] An object motion trajectory video shooting method, comprising:

[0005] Step S1, collecting a motion picture of a target object, generating an image queue composed of multiple frames of images according to the motion picture, and screening multiple frames of trajectory generation images from the image queue;

[0006] Step S2, identifying the position of the target object on the trajectory generation image and obtaining the contour of the target object;

[0007] Step S3, cutting the target object on the trajectory generation image according to the contour, taking the motion posture picture of the target object obtained by cutting as a trajectory picture, pasting each trajectory picture at the corresponding position of the image after the time point of the trajectory generation image in the image queue at all time points, and generating a motion trajectory picture of the target object.

[0008] Preferably, step S1 comprises:

[0009] Step S11, taking the image in which the target object first appears in the image queue as a trajectory starting frame, and recording the time point of the trajectory starting frame;

[0010] Step S12, taking the trajectory starting frame as a reference frame, taking the images of the first n frames relative to the reference frame in the image queue as a training set, and taking the images of the last m frames relative to the reference frame in the image queue and the trajectory starting frame as the data set, wherein n and m are positive integers;

[0011] Step S13, extracting one frame of image every set time interval from the data set to obtain multiple frames of trajectory generation images.

[0012] Preferably, step S2 comprises:

[0013] Step S21, modeling the background picture of the target object by using a Gaussian mixture model, and modeling the background picture of each image in the training set by using a Gaussian mixture model for simulation training;

[0014] Step S22, performing Gaussian mixture model matching on the pixels of each trajectory generation image, and screening out the background picture and the picture of the target object according to the matching result;

[0015] Step S23, filtering out non-target object noise interference in the image;

[0016] Step S24, performing morphological image processing on the picture of the target object to identify the contour of the target object.

[0017] Preferably, step S22 specifically includes:

[0018] Step S221, if the pixel value of the pixel can match at least one Gaussian distribution in the Gaussian mixture model, determining the pixel as a pixel of the background picture; otherwise, performing step S222;

[0019] Step S222, determining the pixel as a pixel of the picture of the target object.

[0020] Preferably, the process of identifying the non-target object in step S23 includes:

[0021] determining the target object identified in the marked area of the image as the non-target object by selecting a scene restriction interference object to appear only in the marked area; and / or;

[0022] identifying the motion range of the moving object in the training set, marking the area corresponding to the motion range in the data set as a motion area, and determining the target object identified in the motion area as the non-target object; and / or;

[0023] determining the contour size of the target object, and determining the target object with different contour sizes as the non-target object; and / or;

[0024] pre-judging the motion trend of the target object, and determining the target object not conforming to the motion trend as the non-target object.

[0025] Preferably, step S3 includes:

[0026] Step S31, polling to read the images in the data set and judging whether the reading is successful, if yes, performing step S32; otherwise, performing step S4;

[0027] Step S32, judging whether the current read image is the track generation image, if yes, executing step S2 and intercepting the track image, and storing the track image and the position of the track image in a list, then executing step S33; otherwise, directly executing step S33.

[0028] Step S33, obtaining the track image and the position of the track image in the list, pasting all the track images in the list to the current frame image according to the position of the track image, and then executing step S31.

[0029] Preferably, after step S3, further comprising:

[0030] Step S4, composing all the processed images into a video for rendering and playing.

[0031] A system for shooting a video of a motion track of an object, applied to the method for shooting a video of a motion track of an object, comprising:

[0032] a camera for shooting the motion track of the target object, and obtaining a plurality of images of the motion picture;

[0033] a recording system connected to a video signal output end of the camera, for recording the images in the camera;

[0034] a track picture generation system connected to an image output end of the recording system, for generating a motion track picture of the target object;

[0035] a time synchronization system connected to the camera, the recording system and the track picture generation system, for synchronizing the time among the camera, the recording system and the track picture generation system.

[0036] Preferably, the recording system comprises:

[0037] a camera interface module connected to a video signal output end of the camera;

[0038] a recording module connected to the camera interface module, for storing the images and the time points of the images;

[0039] a frame picture extraction module connected to the track picture generation system, for extracting and outputting the corresponding images to the track picture generation system according to the instruction of the track picture generation system.

[0040] Preferably, the camera interface module is adapted to the model of the camera, and the model of the camera comprises a broadcast television camera and a network camera.

[0041] The beneficial effects of the present application: due to the adoption of the above technical solutions, the present application presents the posture of several movement points of the athlete in the competition process in front of the audience, so that the audience can better understand the competition state of the athlete. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 A step schematic diagram of the motion trajectory video shooting method in the embodiment of the present application;

[0043] Figure 2 A step schematic diagram of step S1 in the embodiment of the present application;

[0044] Figure 3 A step schematic diagram of step S2 in the embodiment of the present application;

[0045] Figure 4 A step schematic diagram of step S22 in the embodiment of the present application;

[0046] Figure 5 A step schematic diagram of step S3 in the embodiment of the present application;

[0047] Figure 6 A flow chart of the motion trajectory video shooting method in the embodiment of the present application;

[0048] Figure 7 A schematic diagram of the motion trajectory video shooting system in the embodiment of the present application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0050] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0051] The present application will be further described below with reference to the drawings and specific embodiments, but not as a limitation of the present application.

[0052] A motion trajectory video shooting method, as shown in Figure 1 includes:

[0053] Step S1, collecting the motion picture of the target object, generating a image queue composed of multiple frames of images according to the motion picture, and screening multiple frames of trajectory generation images in the image queue;

[0054] Step S2, identifying the position of the target object on the trajectory generation image and obtaining the contour of the target object;

[0055] Step S3, cutting the target object on the trajectory generation image according to the contour to obtain the motion posture picture of the target object as the trajectory picture, and pasting each trajectory picture on the corresponding position of the image after the time point of the trajectory generation image in the image queue to generate the motion trajectory picture of the target object.

[0056] Preferably, the technical solution in the present application is applicable to sports event broadcasting, and specifically, the motion trajectory shooting method is used to present the postures of the athletes at several motion points in the competition to the audience, so that the audience can have a deeper and more intuitive understanding of the competition state of the athletes. For example, a series of postures of a freestyle skiing athlete in the competition are shot on the East Olympic venue to form the motion trajectory of the athlete.

[0057] Specifically, the motion trajectory shooting method used in the present embodiment is a single camera shooting method, that is, the whole process of the competition of the athlete is recorded by one camera 1, then the athlete is identified in one frame picture in which the trajectory needs to be presented according to a fixed time interval, and the image of the athlete cut according to the contour is taken as the trajectory picture, which is pasted on the corresponding position of the subsequent frame picture of the trajectory picture. Finally, the processed frame pictures are composed to render and output the video.

[0058] In a more preferred embodiment, as shown in Figure 2 Step S1 includes:

[0059] Step S11, taking the image in which the target object appears for the first time in the image queue as the trajectory starting frame, and recording the time point of the trajectory starting frame;

[0060] Step S12, taking the trajectory starting frame as the reference frame, taking the images of the first n frames relative to the reference frame in the image queue as the training set, and taking the images of the last m frames relative to the reference frame and the trajectory starting frame in the image queue as the data set, wherein n and m are positive integers;

[0061] Step S13, extracting one frame image every set time interval from the data set to obtain multiple trajectory generation images.

[0062] Preferably, n frame pictures are pushed back from the first frame time point of the trajectory generation to be the background training set of the hybrid Hough model. At this time, the pictures of the images are generally the running-up of the athlete, and the athlete has not appeared in the picture, so all objects in the picture of the image in the training set can be used as the background for training. Once the athlete appears in the picture, that is, the first frame of the trajectory generation, the athlete can be identified and used as the picture of the target object.

[0063] In a preferred embodiment, as shown in Figure 3 Step S2 comprises:

[0064] Step S21, modeling the background picture of the target object by using a Gaussian mixture model, and modeling the background picture of each image in the training set by using a Gaussian mixture model;

[0065] Step S22, performing Gaussian mixture model matching on the pixels of each trajectory generated image, and screening out the background picture and the picture of the target object according to the matching result;

[0066] Step S23, filtering out the noise interference of the non-target object in the image.

[0067] Step S24, performing morphological image processing on the picture of the target object, and identifying the contour of the target object; preferably, performing image morphological operation processing to make the identified picture of the target object clearer, and then identifying the contour of the target object and the circumscribed rectangle of the contour.

[0068] In a preferred embodiment, as shown in Figure 4 Step S22 specifically comprises:

[0069] Step S221, if the pixel value of the pixel can match at least one Gaussian distribution in the Gaussian mixture model, determining the pixel as a pixel of the background picture; otherwise, performing step S222;

[0070] Step S222, determining the pixel as a pixel of the picture of the target object.

[0071] Specifically, the core technology of the motion trajectory shooting is to accurately identify the athlete in each frame of picture. The technical solution in the present application realizes background modeling of the motion picture based on the Gaussian mixture model, and reduces the picture noise by a certain algorithm optimization, so as to accurately locate the athlete in the picture. The basic principle of the Gaussian mixture model is that, before detecting the picture of the target object, the background is trained first, a Gaussian mixture model is used to simulate the background in each frame of image, and the number of Gaussian mixtures of each background can be self-adaptive; then the pixels of the subsequent images are matched by using the Gaussian mixture model, if the pixel value can match one of the Gaussian distributions, it is considered as the background picture, otherwise it is considered as the picture of the target object, and since the Gaussian mixture model is constantly updated and learned in the whole process, it has certain robustness to the dynamic background.

[0072] Specifically, the picture of the target object identified by step 2 is interfered by certain noise, that is, if there are other moving objects in the background during the movement of the athlete, the identification of the athlete will be interfered, such as the moving judge or reporter, the wind vane or flag fluttering in the wind, the shadow of the athlete on the ground, etc. The technical solution in the embodiment can effectively avoid the noise interference of non-target objects and improve the identification accuracy of the target object.

[0073] In a preferred embodiment, the process of identifying non-target objects in step S23 includes:

[0074] Camera scene selection optimization: by selecting a scene to limit the interference objects to appear in a certain area of the shooting picture, and marking the certain area, then through post-processing, the target objects identified in the marked area are determined as non-target objects; and / or;

[0075] Training set optimization: identify the area and range of moving objects from the training set of the previous n frames before the trajectory is generated. At this time, there is no athlete in the training set, so the moving objects are usually non-target objects. The area corresponding to the movement range in the data set is marked as a movement area, and the target objects identified in the movement area are determined as non-target objects; and / or;

[0076] Target object contour size optimization: after the camera scene is selected, the size range of the athlete appearing in the picture can be determined. In post-processing, target objects larger or smaller than the size range are non-target objects; and / or;

[0077] Movement trend optimization: in competitive sports, the movement trend of the athlete is certain, and as time changes, the movement trend of the athlete in the picture is fixed. In post-processing, target objects that do not conform to the movement trend are determined as non-target objects.

[0078] In a preferred embodiment, as shown in Figure 5 step S3 includes:

[0079] Step S31, poll the images in the data set and judge whether the reading is successful. If yes, execute step S32; otherwise, execute step S4;

[0080] Step S32, judge whether the currently read image is a trajectory generation image. If yes, execute step S2 and intercept the trajectory picture, and store the trajectory picture and the position of the trajectory picture in a list, and then execute step S33; otherwise, directly execute step S33;

[0081] Step S33, the trajectory map in the list and the position of the trajectory map are acquired, and all the trajectory maps in the list are pasted to the current frame image according to the position of the trajectory map, and step S31 is continued to be executed.

[0082] Specifically, in the embodiment, the images in the data set are extracted at a set time interval, and the extracted images are used as the step of generating the trajectory image. Instead of generating the image queue again after extraction, the polling method is used to complete the processing of the trajectory image, as shown in Figure 6 For all images in the data set, the frame images are read in time sequence in turn. When the current image is read, it is judged whether the image is a trajectory generation image according to the time interval. If yes, the position of the athlete in the trajectory generation image is identified, the athlete contour is acquired and the athlete posture picture is intercepted, and then each trajectory map and the trajectory map position identified are stored in the list. Further, the athlete posture picture and the athlete position are acquired from the list, and each trajectory map in the list is added to the corresponding position of the current frame image.

[0083] If the current read image is not a trajectory generation image, the athlete posture picture and the athlete position are directly acquired from the list, and each trajectory map in the list is added to the corresponding position of the current frame image.

[0084] In a more preferred embodiment, please continue to refer to Figure 1 As shown in the figure, after step S3, it further includes:

[0085] Step S4, all the processed images are composed into a video for rendering and playing.

[0086] An object motion trajectory video shooting system is applied to the motion trajectory video shooting method in any embodiment, and includes:

[0087] A camera 1 is used to shoot the motion trajectory of a target object, and obtain multiple frame images of the motion picture of the target object. Preferably, the camera 1 in the motion trajectory video shooting system can be a professional camera for broadcasting television, or a network camera, and the specific model can be determined according to the specific broadcast requirements.

[0088] A recording system 2 is connected to the video signal output end of the camera 1, and is used to record the images in the camera 1.

[0089] A trajectory picture generation system 3 is connected to the image output end of the recording system 2, and is used to generate the motion trajectory picture of the target object.

[0090] The time synchronization system 4 is connected with the camera 1, the recording system 2 and the trajectory picture generation system 3, and is used for synchronizing the time among the camera 1, the recording system 2 and the trajectory picture generation system 3 at the frame level, so as to ensure that the obtained image sequence has no time error.

[0091] In a preferred embodiment, the recording system 2 comprises:

[0092] The camera interface module is connected with the video signal output end of the camera 1; preferably, the camera interface module is matched with the model of the camera 1; if it is a professional camera for broadcast television, the SDI signal is transmitted into the recording system 2, and the recording system 2 needs to be equipped with an SDI acquisition card, and the interface module is developed according to the SDK of the acquisition card; if it is a network camera, the camera interface module is developed according to the network protocol of the camera.

[0093] The recording module is connected with the camera interface module, and is used for storing the image and the time point of the image; in the embodiment, the video frame picture obtained from the interface module is stored in the memory by the queue mode, and the time of each frame is calculated and recorded; the queue length can be within 2000 frames, i.e. 40 seconds.

[0094] The frame picture extraction module is connected with the trajectory picture generation system 3, and extracts the corresponding image according to the instruction of the trajectory picture generation system 3 and outputs the image to the trajectory picture generation system 3 for contour recognition and rendering processing.

[0095] Preferably, the trajectory picture generation system 3 in the embodiment is oriented to the operator, and is responsible for the functions of picture preview, dotting, video frame sequence request, trajectory picture generation and result picture output.

[0096] In a preferred embodiment, the working process of the object motion trajectory video shooting system is as follows:

[0097] First step: observing the real-time motion picture of the athlete, and confirming the first trajectory generation time point;

[0098] Second step: according to the first trajectory generation time point, sending the request instruction of taking the first 40 frames of image as the training set and the last 160 frames of image as the data set to the front-end recording system 2;

[0099] Third step: the recording system 2 obtains the 200 frames of instruction, obtains the frame sequence of the corresponding time point from the queue as the image queue, and sends the image queue to the trajectory picture generation system 3;

[0100] Fourth step: after the trajectory picture generation system 3 receives the 200 frames of result picture, the result picture is processed by recognition and rendering into a video frame;

[0101] Fifth step: according to an output instruction, the identified generated video frame sequence is played through a broadcast interface.

[0102] The above description is merely preferred embodiments of the present application, and is not intended to limit the embodiments and protection scope of the present application. It should be noted that, for those skilled in the art, any equivalent replacements and obvious changes made according to the content of the present application should be included in the protection scope of the present application.

Claims

1. A method for capturing video of an object's motion trajectory, characterized in that, include: Step S1: Acquire motion images of the target object, and generate multi-frame image structures based on the motion images. An image queue is formed, and multiple frames of trajectory are selected from the image queue to generate an image; Step S2: Identify the position of the target object on the trajectory generated image and obtain the outline of the target object; Step S3: The target object on the trajectory generation image is cropped according to its contour. The motion posture image of the target object obtained by cropping is used as a trajectory map. Each trajectory map is pasted onto the corresponding position of all time points in the image queue that are after the time point of the trajectory generation image, so as to generate the motion trajectory image of the target object. Step S1 includes: Step S11: Take the image of the first appearance of the target object in the image queue as the trajectory start frame, and record the time point of the trajectory start frame. Step S12: Using the trajectory start frame as a reference frame, the images of the first n frames in the image queue relative to the reference frame are used as a training set, and the images of the last m frames in the image queue relative to the reference frame and the trajectory start frame are used as a dataset, where n and m are both positive integers. Step S13: Extract one frame of image from the dataset at set time intervals to obtain multiple frames of trajectory generation images; Step S2 includes: Step S21: The background image of the target object is modeled using a Gaussian mixture model, and the background image of each image in the training set is simulated and trained using a Gaussian mixture model. Step S22: Perform Gaussian mixture model matching on each pixel of the trajectory generated image, and filter out the background image and the target object image based on the matching result; Step S23: Filter out noise interference from non-target objects in the image; Step S24: Perform morphological image processing on the image of the target object to identify the outline of the target object.

2. The motion trajectory video shooting method according to claim 1, characterized in that, Step S22 specifically includes: Step S221: If the pixel value of the pixel can match at least one Gaussian distribution in the Gaussian mixture model, the pixel is determined as a pixel of the background image; otherwise, proceed to step S222. Step S222: Determine the pixel as a pixel of the image of the target object.

3. The motion trajectory video shooting method according to claim 1, characterized in that, The process of identifying the non-target object in step S23 includes: By selecting a scene size that restricts interfering objects to appearing only in marked areas of the image, the target objects identified in the marked areas are determined as non-target objects; and / or; Identify the motion range of moving objects in the training set, mark the region in the dataset corresponding to the motion range as a motion region, and determine the target object identified in the motion region as the non-target object; and / or; Determine the outline size of the target object, and identify target objects with different outline sizes as non-target objects; and / or; The motion trend of the target object is determined in advance, and the target objects that do not conform to the motion trend are identified as non-target objects.

4. The motion trajectory video shooting method according to claim 2, characterized in that, Step S3 includes: Step S31: Poll and read the images in the dataset and determine whether the reading was successful. If yes, proceed to step S32; otherwise, proceed to step S4. Step S32: Determine whether the currently read image is the trajectory generated image. If so, execute step S2 and extract the trajectory image, and store the trajectory image and its position in a list, then execute step S33; otherwise, directly execute step S33. Step S33: Obtain the trajectory map in the list and the position of the trajectory map, paste all the trajectory maps in the list onto the current frame image according to the position of the trajectory map, and continue to execute step S31.

5. The motion trajectory video shooting method according to claim 1, characterized in that, Step S3 is followed by: Step S4: Combine all the processed images into a video for rendering and playback.

6. A video capture system for object motion trajectory, applied to the motion trajectory video capture method according to any one of claims 1-5, characterized in that, include: A camera is used to capture the motion trajectory of the target object and obtain multiple frames of the motion image. A recording system, connected to the video signal output terminal of the camera, is used to record the images from the camera; A trajectory image generation system, connected to the image output terminal of the recording system, is used to generate motion trajectory images of the target object; A time synchronization system, connecting the camera, the recording system, and the trajectory image generation system, is used to synchronize the time between the camera, the recording system, and the trajectory image generation system.

7. The motion trajectory video shooting system according to claim 6, characterized in that, The recording system includes: The camera interface module is connected to the video signal output terminal of the camera; The recording module is connected to the camera interface module and is used to store the images and the time points of the images; The frame extraction module is connected to the trajectory image generation system and extracts the corresponding image according to the instructions of the trajectory image generation system and outputs it to the trajectory image generation system.

8. The motion trajectory video shooting system according to claim 7, characterized in that, The camera interface module is compatible with the camera model, which includes broadcast television cameras and network cameras.

Images