Method, device and terminal device for generating camera-moving video images

Abstract

The present application provides a method, device and terminal equipment for generating camera movement video images, which are applicable to the field of image communication technology. The method includes: generating initial camera movement video image information, football position coordinate information, football movement information, player position coordinate information and player movement information based on panoramic video, historical camera movement image frames, position detection model and movement state detection model, and obtaining camera movement center coordinate information; calculating zoom factor information based on camera movement center coordinate information, panoramic video image size information and video image reference point information; generating target camera movement video image information based on camera movement center coordinate information, panoramic video size, multi-dimensional space conversion graphic information, zoom factor information and video image rendering model. The present application automatically detects targets on the court, calculates camera movement center coordinates and zoom factors, and generates camera movement video images through multi-dimensional space conversion and rendering, thereby improving camera movement flexibility, broadcast image quality and audience viewing experience.

Description

Technical Field

[0001] The present application belongs to the field of image communication technology, and in particular relates to a method, device and terminal device for generating a camera-moving video image. Background Art

[0002] In the existing technology, ball game broadcasts rely on fixed pan-tilt heads or cameras with preset rotation angles, or use player and ball detection algorithm models to obtain a detection rectangle, then calculate frame coordinates and camera movement coordinates based on simple calculations and threshold judgments, and determine the camera movement rectangle range in combination with the court detection area, or use video acquisition, processing and transmission modules, combined with task tracking recognition and dynamic lens adjustment functions to achieve event broadcasts.

[0003] However, traditional broadcast systems rely on fixed pan / tilt heads or cameras with preset rotation angles, which are physically limited in perspective. This can easily lead to keyframe loss and blind spots during football matches. Furthermore, the equipment is costly, and the mechanical structure is prone to wear, lag, and even failure, increasing operational costs and risks. Frame and camera coordinates are calculated based on simple calculations and thresholds, but lack the ability to perceive the dynamic rhythm of the game. This can lead to abrupt or delayed camera switches during fast breaks or intense defenses. Furthermore, camera angle control relies solely on a fixed target position, failing to consider other dynamic factors and failing to meet the audience's perspective requirements for different scenes. Broadcasting through the video acquisition, processing, and transmission modules typically involves only single-dimensional adjustments, failing to achieve pitch angle adjustment. Consequently, the broadcast image lacks three-dimensionality and depth perception, and cannot adapt to the complexity of the target's motion state through camera switching and dynamic zooming. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a method, apparatus and terminal device for generating camera movement video images, aiming to solve the problems of mechanical limitations and high operation and maintenance costs in the existing camera movement methods; the camera movement strategy is simple, difficult to adapt to dynamic scenes, and the captured images are inaccurate; the camera movement method is single, and it is impossible to achieve multi-angle and dynamic zooming, thereby reducing the audience's viewing experience of the game.

[0005] A first aspect of an embodiment of the present application provides a method for generating a camera movement video image, comprising:

[0006] Obtain panoramic video information and historical camera frame information;

[0007] Generate initial camera movement video frame information, ball position coordinate information, ball movement information, player position coordinate information, and player movement information based on the panoramic video information, historical camera movement frame information, a preset position detection model, and a preset movement state detection model;

[0008] Analyzing and processing the soccer ball position coordinate information, soccer ball movement information, multiple player position coordinate information, and multiple player movement information based on the initial camera movement video frame information and the historical camera movement frame information to obtain camera movement center coordinate information;

[0009] Calculating zoom factor information based on the camera center coordinate information, preset panoramic video screen size information, and multiple preset video screen reference point information;

[0010] Calculating the spatial conversion coordinate information of the camera movement center according to the camera movement center coordinate information, the preset panoramic video screen size information, and the preset multi-dimensional space conversion graphic information;

[0011] Target camera movement video picture information is generated according to the camera movement center space conversion coordinate information, zoom factor information and a preset video picture rendering model.

[0012] A second aspect of the embodiments of the present application provides a device for generating a camera movement video, including:

[0013] An information acquisition module is used to obtain panoramic video information and historical camera movement frame information;

[0014] an initial camera movement video image information generation module, configured to generate initial camera movement video image information, ball position coordinate information, ball movement information, player position coordinate information, and player movement information based on the panoramic video information, historical camera movement frame information, a preset position detection model, and a preset movement state detection model;

[0015] a camera movement center coordinate information calculation module, configured to analyze and process the ball position coordinate information, ball movement information, multiple player position coordinate information, and multiple player movement information based on the initial camera movement video frame information and historical camera movement frame information to obtain camera movement center coordinate information;

[0016] A zoom factor information calculation module, configured to calculate zoom factor information based on the camera center coordinate information, preset panoramic video screen size information, and a plurality of preset video screen reference point information;

[0017] A camera movement center space conversion coordinate information calculation module, used to calculate the camera movement center space conversion coordinate information based on the camera movement center coordinate information, preset panoramic video screen size information and preset multi-dimensional space conversion graphic information;

[0018] The target camera movement video picture information generation module is used to generate the target camera movement video picture information according to the camera movement center space conversion coordinate information, zoom factor information and a preset video picture rendering model.

[0019] A third aspect of an embodiment of the present application provides a terminal device, which includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements the steps of the method for generating a moving camera video image as described in the first aspect above.

[0020] Compared with the prior art, the beneficial effects of the embodiments of the present application are: by automatically detecting the position information and movement status of the target in each frame of the panoramic video information of the ball game, based on the setting of complex rules, the position information and movement status of the target are used to dynamically calculate the camera center coordinates and zoom multiples of each frame of the camera movement picture, and then through multi-dimensional space conversion and depth rendering model processing, the camera movement video picture is generated frame by frame, thereby achieving a three-dimensional and dynamic camera movement effect, improving the flexibility and accuracy of automatic camera movement, and at the same time improving the quality level of the broadcast picture, bringing a more immersive and high-quality viewing experience to the audience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 This is a schematic diagram of the implementation process of the method for generating a moving camera video provided in Example 1 of the present application;

[0023] Figure 2 This is a schematic diagram of the implementation flow of the method for generating a moving camera video provided in Example 2 of the present application;

[0024] Figure 3 This is a schematic diagram of the implementation flow of the method for generating a camera-moving video image provided in Example 3 of the present application;

[0025] Figure 4 This is a schematic diagram of the implementation flow of the method for generating a camera movement video image provided in the fourth embodiment of the present application;

[0026] Figure 5 This is a schematic diagram of the implementation flow of the method for generating a camera movement video provided in Example 5 of the present application;

[0027] Figure 6 This is a schematic diagram of the implementation flow of the method for generating a camera-moving video image provided in Example 6 of the present application;

[0028] Figure 7 This is a schematic diagram of the implementation flow of the method for generating a camera movement video image provided in Example 7 of the present application;

[0029] Figure 8 This is a schematic diagram of the implementation flow of the method for generating a camera movement video provided in Example 8 of the present application;

[0030] Figure 9 Schematic diagram of the structure of the camera movement video image generation device provided in an embodiment of the present application;

[0031] Figure 10 It is a schematic diagram of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0033] In order to illustrate the technical solution described in this application, specific embodiments are provided below.

[0034] Figure 1 The following is a flowchart of the method for generating a camera-moving video image according to the first embodiment of the present application, which is described in detail as follows:

[0035] Step S101, obtaining panoramic video information and historical camera movement frame information.

[0036] In this embodiment, the panoramic video information can be captured by a panoramic camera. The panoramic camera can be mounted in a suitable location on the court, such as a high-point fixed mount, to ensure that the camera's field of view fully covers key areas of the court. Understandably, the captured panoramic video data requires preliminary processing, such as noise removal and color correction, to ensure the accuracy of subsequent operations such as position detection. The historical camera movement frame information can be the broadcast image information preceding the current frame, including the previous frame information.

[0037] Step S102, based on the panoramic video information, historical camera frame information, a preset position detection model and a preset movement state detection model, generates initial camera video frame information, football position coordinate information, football movement information, player position coordinate information and player movement information.

[0038] In this embodiment, the preset position detection model can be an existing target detection model, specifically, a combination of a YOLOv8 model and a DeepSORT model. The YOLOv8 model is used for target detection, i.e., identifying and detecting the football and players in the panoramic video information, while the DeepSORT model is used for target tracking, i.e., determining the position coordinate information of the football and players in the panoramic video information. The football's movement direction and the player's movement direction can then be calculated based on the football's position coordinate information and the player's position coordinate information in the previous and next frames. Therefore, the position detection model can be used to determine the football's position coordinates and the player's position coordinates in the panoramic video information. The preset movement state detection model can be designed based on the optical flow method and used to detect the movement speed of the football and players on the field, thereby determining the football's movement information and the player's movement information in the panoramic video information. The football's movement information includes the football's movement speed and the football's movement direction, and the player's movement information includes the player's movement speed and the player's movement direction. The data used to train the position detection model can be obtained by collecting data from 45 games totaling 72 hours. The collected game videos are randomly intercepted to obtain 10,000 panoramic images. These panoramic images are then annotated to clarify the location, category, and other information of the target, thereby obtaining data for training the position detection model. The initial camera movement video image information can be generated based on historical camera movement frame information or the previous frame of the camera movement. This is used to determine whether the ball is tracked by the camera movement image in the subsequent step of determining the camera movement center.

[0039] In this embodiment, the YOLOv8 algorithm can be used to train a model on a training set of annotated panoramic images. The model is fed a panoramic image, and the model learns image features to identify the position and category of the player and ball, outputting detection boxes and confidence scores. During training, parameters are adjusted to optimize detection accuracy and speed. The trained YOLOv8 model then performs object detection on the current frame, obtaining the player and ball's detection boxes and confidence scores. The detection results are then fed into the DeepSORT tracking model to track the player and ball, and their coordinate positions within the two-dimensional panoramic video are calculated. Optical flow is then used to estimate pixel motion between consecutive frames, determining the speed of the ball and player on the image plane. This speed is then converted to actual physical motion speed based on the image size and camera parameters. Finally, the movement direction of the ball and player is calculated by comparing the position difference between the previous and current frames, simplifying the movement direction into two directions: positive and negative.

[0040] Step S103: Analyze and process the football position coordinate information, football movement information, multiple player position coordinate information, and multiple player movement information based on the initial camera movement video frame information and historical camera movement frame information to obtain camera movement center coordinate information.

[0041] In this embodiment, the camera movement center is the visual center of the camera lens during a football match broadcast, and the camera movement center coordinate information refers to the coordinate information of the point in the camera movement image where the camera movement center is located. This can be accomplished by first determining whether the coordinate information of the ball in the current frame of the panoramic video information falls within the initial camera movement video image information. If the coordinate information of the ball in the current frame falls within the initial camera movement video image information, the specific camera movement center coordinate information is determined based on the ball movement information, or the ball movement speed information in the ball movement information. If the coordinate information of the ball in the current frame does not fall within the initial camera movement video image information, it indicates that the ball needs to be repositioned, and the camera movement center coordinate information is re-determined based on the ball coordinate information. This can be accomplished by determining whether the ball coordinate information exists in historical camera movement image frame information or in the previous frame information of the historical camera movement image frame information. The camera movement center coordinate information is then determined based on the determination result, combined with the player position coordinate information and the player movement information.

[0042] Step S104 , calculating zoom factor information according to the camera center coordinate information, preset panoramic video screen size information, and multiple preset video screen reference point information.

[0043] In this embodiment, the preset panoramic video screen size information may include the width and height information of the panoramic video screen, and may also include aspect ratio information, where the aspect ratio information may be determined by the hardware device performing the broadcast. The preset video screen reference point information may be a manually set initial point of the two-dimensional panoramic image, as well as a distal end point and a proximal end point of the panoramic image from the initial point. Zoom factor information may refer to a key parameter used to adjust the zoom effect of the broadcast screen during football match broadcasts. Its setting needs to comprehensively consider normal situations and special scenarios to ensure that the audience can watch the game clearly. Under normal circumstances, the default width zoom of the lens is 0.5 times the width of the original image. This setting is intended to ensure a stable image, fully present game information, and meet the basic viewing needs of the audience. In special scenarios, when the camera center is far away from the initial point of the image, the width zoom factor needs to be dynamically adjusted based on the distance between the camera center and the reference point of the video screen. A plane rectangular coordinate system is established with the lower left corner of the two-dimensional panoramic image as the origin of the coordinate system. The near boundary distance and the far boundary distance can be set. When the camera center reaches the near boundary distance from the initial point of the image, the width zoom is 0.5 times; when the camera center reaches or exceeds the far boundary distance from the initial point of the image, the width zoom is reduced to 0.2 times. In the intermediate state between the two, a linear interpolation formula can be used to determine the zoom factor.

[0044] Step S105 , calculating the camera movement center space conversion coordinate information according to the camera movement center coordinate information, the preset panoramic video screen size information and the preset multi-dimensional space conversion graphic information.

[0045] In this embodiment, it is understandable that during the broadcast process, three-dimensional scene modeling is required to simulate the movement and perspective changes of the camera in the broadcast of sports events, so it is necessary to set a 3D model that conforms to the 180-degree panoramic picture broadcast perspective, that is, the preset multi-dimensional space conversion graphic information. The preset multi-dimensional space conversion graphic information can use a semi-cylinder model to map the 180-degree panoramic image captured by the panoramic camera to the inner surface of the semi-cylinder model. At the same time, the symmetrical center point of the entire cylinder in the three-dimensional space is set as the camera position, so that the visualization effect can be presented to any position on the court, as the camera perspective of the broadcast, and provided to the audience for watching the event. It is understandable that the two-dimensional camera movement center coordinate information is mapped to the three-dimensional space through the preset multi-dimensional space conversion graphic information, and the obtained coordinate information is used as the camera movement center space conversion coordinate information.

[0046] Step S106 , generating target camera movement video picture information according to the camera movement center space conversion coordinate information, the zoom factor information, and a preset video picture rendering model.

[0047] In this embodiment, the preset video rendering model can be set up based on Three.js, a JavaScript-based library that enables staff to quickly complete broadcast image rendering without in-depth knowledge of the underlying details of WebGL. This can be achieved by inputting the camera center spatial conversion coordinate information and zoom factor information into the preset video rendering model. The visual image input by the video rendering model, i.e., the target camera video image information, can be used to output to the terminal devices used by the audience for real-time broadcast of the event.

[0048] The method for generating camera movement video images provided in the embodiment of the present application automatically detects the position information and movement status of the target in each frame of the panoramic video information of the ball game, and based on the setting of complex rules, dynamically calculates the camera movement center coordinates and zoom factor of each frame of the camera movement image through the position information and movement status of the target. Then, through multi-dimensional space conversion and depth rendering model processing, the camera movement video images are generated frame by frame, thereby achieving a three-dimensional and dynamic camera movement effect, improving the flexibility and accuracy of automatic camera movement, and at the same time improving the quality level of the broadcast image, bringing a more immersive and high-quality viewing experience to the audience.

[0049] Figure 2The flowchart of the method for generating a camera movement video provided in the second embodiment of the present application is shown. The difference between the method and the first embodiment is as follows:

[0050] The historical camera movement picture frame information includes the previous frame video picture frame information; the previous frame video picture frame information includes the previous frame camera movement center coordinate information;

[0051] The football position coordinate information includes the current football position coordinate information and the previous frame football position coordinate information;

[0052] The football movement information includes the current football movement speed information, the previous frame football movement direction information and the previous frame football movement speed information;

[0053] The player movement information includes player movement direction information;

[0054] The step S103 specifically includes:

[0055] Step S201, determine whether the current football position coordinate information exists in the initial camera movement video image information; if so, proceed to step S202; if not, proceed to step S203.

[0056] In this embodiment, the current football position coordinate information refers to the football position coordinate information in the picture currently taken by the panoramic camera. If the current football position coordinate information exists in the initial camera movement video picture information, it means that the football has not been lost during the camera movement process. Therefore, the camera movement center coordinate information can be determined based on the football movement speed information; if the current football position coordinate information does not exist in the initial camera movement video picture information, it means that the football has moved during the camera movement process, and the football needs to be repositioned, and then the camera movement center is re-determined based on the positioning of the football. Therefore, it is necessary to determine whether the football position coordinate information exists in the previous frame camera movement picture, and then further determine the specific position coordinates of the camera movement center.

[0057] Step S202: obtaining the camera center coordinate information according to the current football movement speed information, the current football position coordinate information, the preset football speed threshold information and the preset player number parameter information.

[0058] In this embodiment, the preset ball speed threshold information may be 15 m / s. It will be appreciated that if the current ball speed information is less than the preset ball speed threshold information, it indicates that the current ball speed is low and the ball is unlikely to move widely. Therefore, a camera movement area may be determined using the position coordinates of multiple moving players, and the center of gravity of the camera movement area may be calculated as the camera movement center coordinate information. Alternatively, multiple dynamically moving coordinates may be determined, and the center of gravity of the multiple position coordinates may be calculated as the camera movement center coordinate information. The camera movement area may be a camera movement area composed of the ball coordinates and the coordinates of multiple players closest to the ball. The multiple player position coordinates closest to the ball may be the position coordinates of the eight players closest to the ball. If the current ball speed information is greater than or equal to the preset ball speed threshold information, it indicates that the current ball speed is high and is likely to be lost. Therefore, the current ball coordinate information may be used as the camera movement center coordinate information.

[0059] Step S203, obtaining the camera center coordinate information based on the previous frame video frame information, the previous frame football position coordinate information, the previous frame football movement speed information, the previous frame football movement direction information, the previous frame camera center coordinate information, the player position coordinate information, the player movement direction information and the preset football speed threshold.

[0060] In this embodiment, if the current football position coordinate information does not exist in the initial camera movement video picture information, it means that the camera movement process has already caused the football to move over a large range. Therefore, the football needs to be tracked again. It is necessary to determine whether the football position coordinate information exists in the previous frame camera movement picture, and then further determine the specific position coordinates of the camera movement center based on the previous frame football movement speed information, the previous frame football movement direction information, the previous frame camera movement center coordinate information, the player position coordinate information and the player movement direction information.

[0061] The camera movement video image generation method provided in the embodiment of the present application determines the camera movement center according to the ball speed when the ball is detected. When the ball speed is low, the center of gravity is determined by combining the ball and the player's position. When the ball speed is high, the ball coordinates are used as the center. This can ensure that the image contains both the dynamics of the ball and the situation of the players, and fully presents the key content of the game. When the ball is not detected, the camera movement center is determined based on the previous frame situation and the player's movement status, ensuring the continuity of the image and preventing the broadcast logic from being interrupted due to the temporary loss of the ball. Therefore, during the actual football game broadcast process, the camera movement flexibility is improved, the adaptability of the broadcast image to complex game conditions is improved, the best viewing angle is provided for the audience, and the broadcast quality and the audience's viewing experience are greatly improved.

[0062] Figure 3 The flowchart of the method for generating a camera-moving video provided in the third embodiment of the present application is shown. The difference between the method and the second embodiment is as follows:

[0063] The step S202 specifically includes:

[0064] Step S301, determining whether the current ball movement speed information is less than a preset ball speed threshold information; if so, proceeding to step S302; if not, proceeding to step S304.

[0065] In this embodiment, the preset ball speed threshold may be 15 m / s. If the current ball speed is less than the preset ball speed threshold, it indicates that the ball is currently moving at a low speed and is not undergoing extensive movement. The current ball position coordinates and the position coordinates of the multiple players closest to the ball can then be determined. The center of gravity of the current ball position coordinates and the multiple player position coordinates can then be calculated as the camera movement center coordinates. If the current ball speed is greater than or equal to the preset ball speed threshold, it indicates that the ball is currently moving at a high speed and is likely to undergo extensive movement. The current ball position coordinates can then be used as the camera movement center coordinates.

[0066] Step S302: Calculate the pixel distance between the current ball position coordinate information and the player position coordinate information.

[0067] In this embodiment, the distance value between the current football position coordinate information and the position coordinate information of all players can be calculated, or the straight-line distance can be calculated to measure the distance value between the pixel where the football is located and the pixels where all players are located in the current frame, that is, the pixel distance value.

[0068] Step S303: Obtain camera center coordinate information based on the current football position coordinate information, pixel distance value, player position coordinate information, and preset player number parameter information.

[0069] In this embodiment, the preset parameter information for the number of players may be 8. The pixel distance values may be arranged in ascending order, and the player position coordinate information corresponding to the first 8 pixel distance values may be screened out. The center of gravity coordinates are then calculated by combining these 8 player position coordinates with the current ball position coordinates, and the center of gravity coordinates are then used as the camera movement center coordinate information.

[0070] Step S304: Using the current soccer ball position coordinate information as the camera center coordinate information.

[0071] In this embodiment, if the current football movement speed information is greater than or equal to the preset football speed threshold information, it means that the current football speed is high and it is very easy to be lost, so the current football position coordinate information can be used as the camera center coordinate information.

[0072] The camera movement video image generation method provided by the embodiment of the present application, when the ball speed is low, by calculating the pixel distance between the ball and the player, selecting the specific player coordinates and the ball coordinates to obtain the center of gravity as the camera movement center, so that the broadcast image can take into account the dynamic changes of the ball and the player, present the details of the game to the audience, and let the audience fully understand the situation on the field in real time. When the ball speed is high, using the ball coordinates as the camera movement center can quickly capture the movement trajectory information of the ball, thereby avoiding the loss of key images, so as to achieve the adjustment of the camera movement center according to different ball speeds. When the ball speed is slow, the overall game situation is displayed through the broadcast image, and when the ball speed is fast, the high-speed movement of the football is displayed through the broadcast image, which meets the audience's viewing expectations of the game images, enhances the immersion and tension of the viewing, and improves the viewing and attractiveness of the broadcast images.

[0073] Figure 4 The flowchart of the method for generating a camera movement video provided in the fourth embodiment of the present application is shown. The difference between the fourth embodiment and the third embodiment is that:

[0074] The step S303 specifically includes:

[0075] Step S401 : Arrange the pixel distance values in ascending order to obtain arranged distance values.

[0076] In this embodiment, the pixel distance values may be arranged in ascending order, and the arranged pixel distance values are used as the arranged distance values.

[0077] Step S402: screening the arrangement distance values according to preset player number parameter information to obtain multiple player confirmation distance values.

[0078] In this embodiment, the preset player number parameter information may be 8. The arrangement distance values may be screened to obtain the first 8 arrangement distance values as the player confirmation distance values.

[0079] Step S403: extracting the player position coordinate information corresponding to the player confirmation distance value to obtain the camera movement center reference variable coordinate information.

[0080] In this embodiment, the preset player number parameter information can be 8, and the player position coordinate information corresponding to the first 8 pixel distance values is filtered out as the reference variable coordinate information of the camera movement center, which is used to calculate and determine the camera movement center coordinate information in subsequent steps.

[0081] Step S404, calculating the center of gravity of the camera movement center reference variable coordinate information and the current football position coordinate information to obtain the camera movement center coordinate information.

[0082] In this embodiment, the center of gravity of the camera movement center reference variable coordinate information and the current football position coordinate information may be calculated, and the calculated center of gravity may be used as the camera movement center coordinate information.

[0083] The camera movement video image generation method provided in the embodiment of the present application can take into account the dynamic movement of the football and the players while the positions of the football and the players are constantly changing, so that the selection of the camera movement center can allow the broadcast image to include the football and the key players around the football, thereby comprehensively presenting the tactical coordination, positional relationship and other game information between the players, so that the audience can quickly grasp the game situation through the broadcast image, and when the players are attacking or defending, the core area image can be accurately presented to avoid abrupt or delayed image switching, providing the audience with a smooth and focused viewing experience, greatly improving the broadcast quality and the audience's viewing experience.

[0084] Figure 5 The flowchart of the method for generating a camera-movement video provided in the fifth embodiment of the present application is shown. The difference between the method and the second embodiment is as follows:

[0085] The step S203 specifically includes:

[0086] Step S501, determining whether the previous frame soccer ball position coordinate information exists in the previous frame video frame information; if so, proceeding to step S502; if not, proceeding to step S506.

[0087] In this embodiment, if the ball's position coordinate information is not detected in the current frame, the camera center of the current broadcast frame needs to be determined based on the previous broadcast frame. If the ball's position coordinate information in the current frame exists in the previous video frame, it means that the ball can be tracked in the previous frame. In this case, the camera center is determined around the ball's current position in the previous frame to ensure that the camera can capture the ball's position. If the ball's position coordinate information in the current frame does not exist in the previous video frame, it means that the ball cannot be tracked in the previous frame. In this case, the camera center needs to be determined based on the player's movement direction to increase the probability that the camera can capture the ball's position.

[0088] Step S502, determining whether the ball movement speed information of the previous frame is less than a preset ball speed threshold; if so, proceeding to step S503; if not, proceeding to step S504.

[0089] In this embodiment, the preset football speed threshold can be 15m / s. If the football movement speed information of the previous frame is less than the preset football speed threshold, it means that the football movement speed in the previous frame is low, and the current football position is likely to be around the previous frame or blocked. In this case, the camera center can be kept unchanged, that is, the camera center of the previous frame is used as the camera center of the current frame. If the football movement speed information of the previous frame is greater than or equal to the preset football speed threshold, it means that the football movement speed in the previous frame is high, and the current football position is likely to be far away from the range covered by the previous frame. In this case, the position information and movement direction of the football in the previous frame are recorded. At the same time, in the current frame, based on the football position in the previous frame, along the direction of the football movement, a player at the edge of the direction is searched as a reference point. The position of the ball in the previous frame and the reference point of the current frame are used as the boundary, the middle area is used as the camera movement area, and the center of the camera movement area is selected as the camera movement center.

[0090] Step S503: Using the previous frame's camera movement center coordinate information as the camera movement center coordinate information.

[0091] In this embodiment, if the football movement speed information of the previous frame is less than the preset football speed threshold, it means that the football movement speed in the previous frame is relatively low, and the current football position is most likely around the previous frame or is blocked, so the camera center can be kept unchanged, that is, the camera center of the previous frame is used as the camera center of the current frame.

[0092] Step S504: obtaining camera movement area information according to the previous frame football movement direction information, player position coordinate information, and player movement direction information.

[0093] In this embodiment, if the football movement speed information of the previous frame is greater than or equal to the preset football speed threshold, it means that the football movement speed in the previous frame is relatively high, and the current football position is likely to be far away from the range covered by the previous frame. The position information and movement direction of the football in the previous frame are recorded. At the same time, in the current frame, based on the football position in the previous frame, along the direction of football movement, a player located at the edge of the direction is found as a reference point. The position of the ball in the previous frame and the reference point of the current frame are used as the boundary, and the middle area information is used as the camera movement area information.

[0094] Step S505: Calculate the midpoint coordinates of the camera movement area information to obtain the camera movement center coordinate information.

[0095] In this embodiment, the center coordinates of the camera movement area information are extracted as the camera movement center coordinate information.

[0096] Step S506: Calculate the player's movement direction ratio based on the player's movement direction information.

[0097] In this embodiment, if the ball was not detected in the previous frame, the movement direction information of all players is counted and the player movement direction ratio is calculated. For example, if there are m players moving to the left and n players moving to the right, the player movement direction ratio can be expressed as m / (n+m) and n / (n+m).

[0098] Step S507 , obtaining the camera center coordinate information according to the player movement direction information, the player movement direction ratio, the player position coordinate information, the preset player movement ratio threshold and the preset moving player number parameter information.

[0099] In this embodiment, the preset player movement ratio threshold may be 0.65 or 0.3. If the movement ratios of all players on the field are relatively balanced (the difference in ratios does not exceed the player movement ratio threshold), it indicates that there is no single-directional offensive or defensive trend on the field. In this case, the center of gravity can be calculated for all player coordinates and used as the camera movement center coordinate information for the current frame. If the number of players on the field moving in a certain direction is significantly higher than in other directions (the difference in ratios is greater than the preset player movement ratio threshold), it indicates that a clear offensive or defensive trend exists in a local area of the field. The coordinate information of players with consistent movement directions is then filtered, and the center of gravity of the filtered player coordinate information is calculated, and this center of gravity is used as the camera movement center coordinate information.

[0100] The method for generating a camera movement video provided in an embodiment of the present application is determined based on whether the ball was detected in the previous frame and the ball speed, which can ensure the continuity and stability of the broadcast image. Since the state of the ball in a football match is changeable, if the ball is detected in the previous frame, the ball speed is low and it is judged to be blocked, the camera movement center is kept unchanged, avoiding frequent changes in the image due to temporary obstruction of the ball. The audience can continue to pay attention to the image and not miss the actions of the surrounding players and changes in the situation; if the ball speed is high, the camera movement center is determined based on the position and movement direction of the ball in the previous frame and the edge players in the current frame, which can quickly track the high-speed movement trajectory of the ball. Even if the ball is partially blocked in the current frame or quickly crosses the screen, the camera can stably focus on the key area, thereby improving the adaptability of the camera movement image to complex game conditions, providing the audience with the best viewing angle, and improving the viewing experience.

[0101] Figure 6 The flowchart of the method for generating a camera movement video provided in the sixth embodiment of the present application is shown. The difference between the sixth embodiment and the fifth embodiment is that:

[0102] The player movement direction information includes first movement direction information and second movement direction information; the first movement direction information is opposite to the second movement direction information;

[0103] The step S507 specifically includes:

[0104] Step S601, determining whether the player movement direction ratio is less than a preset player movement ratio threshold; if so, proceeding to step S602; if not, proceeding to step S603.

[0105] In this embodiment, the preset player movement ratio threshold may be 0.65 or 0.3. When the player movement direction ratio is less than the preset player movement ratio threshold, it indicates that the proportion of players moving in both directions is less than the player movement ratio threshold, meaning there is no single-directional offensive or defensive trend on the field. In this case, the center of gravity can be calculated for all player coordinates and used as the camera movement center coordinate information for the current frame. When the player movement direction ratio is greater than or equal to the preset player movement ratio threshold, it indicates that among multiple players moving in both directions, the proportion of players in one direction is greater than the player movement ratio threshold, meaning there is a clear offensive or defensive trend in a local area on the field. The coordinate information of players with the same movement direction is then filtered, and the center of gravity of the filtered player coordinate information is calculated, and this center of gravity is used as the camera movement center coordinate information.

[0106] In this embodiment, optionally, the difference between the player number information in the first moving direction and the player number information in the second moving direction can be calculated, and the preset player movement ratio threshold can also be the threshold of the difference between the player number information in the first moving direction and the player number information in the second moving direction, wherein the preset player movement ratio threshold can be 0.65 or 0.3, and then the relationship between the difference between the player number information in the first moving direction and the player number information in the second moving direction and the player movement ratio threshold is determined to determine whether there is an obvious offensive and defensive trend on the field.

[0107] Step S602: Calculate the center of gravity of the player's position coordinate information to obtain the camera movement center coordinate information.

[0108] In this embodiment, when the ratio of the player movement direction is less than the preset player movement ratio threshold, it means that there is no single-directional offensive and defensive trend on the entire field. At this time, the center of gravity of all player coordinates is calculated and used as the camera center coordinate information of the current frame.

[0109] Step S603: Obtain the first moving direction player quantity information and the first moving direction player position information corresponding to the first moving direction information, and the second moving direction player quantity information and the second moving direction player position information corresponding to the second moving direction information, based on the player moving direction information.

[0110] In this embodiment, when the player movement direction ratio is greater than or equal to a preset player movement ratio threshold, it indicates that a significant offensive and defensive trend exists in a local area on the field. Therefore, the first movement direction player quantity information and the first movement direction player position information corresponding to the first movement direction information, as well as the second movement direction player quantity information and the second movement direction player position information corresponding to the second movement direction information, can be counted. The first movement direction information may be a left direction, and the second movement direction information may be a right direction. The first movement direction information may be the number of players moving to the left, and the first movement direction player position information may indicate the coordinates of players moving to the left. The second movement direction information may be the number of players moving to the right, and the second movement direction player position information may indicate the coordinates of players moving to the right.

[0111] Step S604, determining whether the information on the number of players in the first moving direction is greater than the information on the number of players in the second moving direction; if so, proceeding to step S605; if not, proceeding to step S606.

[0112] In this embodiment, when the information about the number of players in the first moving direction is greater than the information about the number of players in the second moving direction, it is necessary to further determine whether there is an obvious offensive and defensive trend in the situation on the field.

[0113] Step S605 : filtering the player position information in the first moving direction according to the preset parameter information of the number of sports players to obtain the camera movement center calculation variable information.

[0114] In this embodiment, the preset parameter information for the number of moving players can be half of the number of players moving along the first moving direction, or half of the number of players on the entire court. Alternatively, after filtering out the position information of all players moving along the first moving direction, the first half of the players on the entire court who are located at the front of the court moving along the first moving direction are selected, and the coordinate information of the first half of the players on the entire court is used as the variable information for camera center calculation. In the subsequent center of gravity calculation, the calculated center of gravity is used as the variable information for camera center calculation.

[0115] Step S606: Filter the player position information in the second moving direction according to the preset parameter information of the number of sports players to obtain the camera movement center calculation variable information.

[0116] In this embodiment, the preset parameter information for the number of moving players can be half of the number of players moving along the second moving direction, or half of the number of players on the entire court. Alternatively, after filtering out the position information of all players moving along the second moving direction, the first half of the players on the entire court moving along the second moving direction are filtered out, and the coordinate information of the first half of the players on the entire court is used as the variable information for camera center calculation. In the subsequent center of gravity calculation, the calculated center of gravity is used as the variable information for camera center calculation.

[0117] Step S607: Calculate the center of gravity of the camera movement center calculation variable information to obtain the camera movement center coordinate information.

[0118] In this embodiment, all camera center calculation variable information is used to calculate the center of gravity coordinates, and the calculated center of gravity is used as the camera center coordinate information, that is, the coordinate information of the center of gravity is used as the camera center coordinate information of the current frame to realize the broadcast of football matches.

[0119] The camera movement video image generation method provided in the embodiment of the present application can comprehensively reflect the offensive and defensive situation on the court by judging the direction of player movement, thereby calculating the center of gravity of multiple players in the front position, and using the center of gravity as the camera movement center, so that the dynamic situation of players in key offensive or defensive positions can be presented in the broadcast screen, so as to accurately capture the core area of the game and accurately present the audience with the exciting moments such as confrontation and cooperation in the core area and the direction of the game, thereby improving the flexibility of the camera movement broadcast screen, the broadcast quality and the audience's viewing experience.

[0120] Figure 7 The flowchart of the method for generating a camera movement video provided in the seventh embodiment of the present application is shown. The difference between the method and the first embodiment is as follows:

[0121] The preset panoramic video screen size information includes preset panoramic video screen aspect ratio information;

[0122] The mirror movement center coordinate information includes the mirror movement center horizontal coordinate information and the mirror movement center vertical coordinate information;

[0123] The preset video picture reference point information includes preset reference original point information, preset reference near end point information and preset reference far end point information;

[0124] The zoom factor information includes width zoom factor information and height zoom factor information;

[0125] The step S104 specifically includes:

[0126] Step S701 : Calculate the proximal boundary distance according to the preset reference original point information and the preset reference proximal endpoint information.

[0127] In this embodiment, the two-dimensional panoramic image may be a rectangle having a width and a height. A rectangular coordinate system may be established with the lower left corner of the two-dimensional panoramic image as the coordinate system origin, where the X-axis represents the right side and the Y-axis represents the top side. The preset reference origin point information may be the midpoint on the base of the rectangle, that is, the point whose horizontal coordinate is half the width of the base of the rectangle and whose vertical coordinate is zero. The preset reference proximal endpoint may be manually set, with the horizontal coordinate being half the width of the base of the rectangle and the vertical coordinate being half the height of the rectangle. The Pythagorean theorem may be used to calculate the straight-line distance between the reference origin point and the reference proximal endpoint using the horizontal and vertical coordinate information of the preset reference origin point information and the preset reference proximal endpoint information, and use this as the proximal boundary distance.

[0128] Step S702 : Calculate the far-end boundary distance according to the preset reference original point information and the preset reference far-end point information.

[0129] In this embodiment, the preset reference proximal endpoint may be manually set, the horizontal coordinate may be the base width, and the vertical coordinate may be 0. The Pythagorean theorem may be used to calculate the straight-line distance between the reference origin point and the reference distal endpoint using the preset horizontal and vertical coordinate information of the reference origin point and the preset reference distal endpoint to serve as the distal boundary distance.

[0130] Step S703: Calculate an initial distance value according to the camera center coordinate information and the preset reference original point information.

[0131] In this embodiment, the straight-line distance between the mirror movement center coordinate information and the reference original point information may be calculated based on the Pythagorean theorem as the initial distance value.

[0132] Step S704 : Calculate width scaling factor information according to the initial distance value, the near-end boundary distance, and the far-end boundary distance.

[0133] In this embodiment, the width scaling factor information may be calculated as follows:

[0134]

[0135] Among them, ux represents the width scaling factor information, d represents the initial distance value, and satisfies d=min(d, ), d=max( , ),in, represents the near-end boundary distance, Indicates the far boundary distance.

[0136] Step S705 : Calculate height scaling factor information according to the width scaling factor information and preset panoramic video aspect ratio information.

[0137] In this embodiment, the height scaling factor information may be calculated as follows:

[0138] ux

[0139] Here, k represents preset panoramic video aspect ratio information, which may be determined by the size of the broadcasting device and is used to represent the aspect ratio of the broadcasting device.

[0140] The camera movement video image generation method provided in the embodiment of the present application dynamically adjusts the zoom factor according to the distance between the camera movement center and the reference original point when the camera movement center is far away from the image initial point, ensuring that the audience can clearly see the details of the game through the broadcast image. At the same time, since the football game scene is complex and changeable, the focus of attention is different at different game moments. By dynamically adjusting the zoom factor, it can flexibly adapt to the game situation at different game moments, and always provide the audience with the most appropriate viewing angle during the game, so that the broadcast image is more adapted to the game rhythm and audience needs.

[0141] Figure 8 The flowchart of the method for generating a camera-moving video provided in the eighth embodiment of the present application is shown. The difference between the eighth embodiment and the seventh embodiment is that:

[0142] The preset panoramic video screen size information includes preset panoramic video screen width information and preset panoramic video screen height information;

[0143] The preset multi-dimensional space conversion graphic information includes a preset multi-dimensional space cylinder radius and a preset multi-dimensional space cylinder height;

[0144] The spatial conversion coordinate information of the camera movement center includes the horizontal axis coordinate of the camera movement center multi-dimensional space, the vertical axis coordinate of the camera movement center multi-dimensional space, and the vertical axis coordinate of the camera movement center multi-dimensional space;

[0145] The step S105 specifically includes:

[0146] Step S801 : normalizing the horizontal coordinate information of the camera movement center according to preset panoramic video width information to obtain a normalized horizontal coordinate of the camera movement center.

[0147] In this embodiment, the preset panoramic video screen width information refers to the width information of the panoramic video image, which may be determined by the size of the hardware device. The normalized horizontal coordinate of the camera center may be calculated by dividing the horizontal coordinate information of the camera center by the preset panoramic video screen width information.

[0148] Step S802 : normalizing the vertical coordinate information of the camera movement center according to the preset panoramic video picture height information to obtain the normalized vertical coordinate of the camera movement center.

[0149] In this embodiment, the preset panoramic video screen height information refers to the height information of the panoramic video image, which may be determined by the size of the hardware device. The normalized vertical coordinate of the camera center may be calculated by dividing the vertical coordinate information of the camera center by the preset panoramic video screen height information.

[0150] Step S803 , performing space angle conversion processing according to the normalized horizontal coordinate of the camera movement center to obtain the space angle of the camera movement center.

[0151] In this embodiment, the normalized horizontal coordinate of the mirror movement center can be Multiply them together to calculate the spatial angle of the camera center.

[0152] Step S804 , obtaining the horizontal axis coordinate of the multi-dimensional space of the camera movement center according to the camera movement center space angle and the preset multi-dimensional space cylinder radius.

[0153] In this embodiment, the preset multi-dimensional space cylinder radius can be manually set or automatically generated by Three.js. The camera center's spatial angle can be used as the independent variable of a cosine function to calculate the cosine function value, which is then multiplied by the preset multi-dimensional space cylinder radius, with the multiplication result being used as the multi-dimensional space horizontal axis coordinate of the camera center.

[0154] Step S805 , obtaining the multi-dimensional space vertical axis coordinate of the camera movement center according to the camera movement center space angle and the preset multi-dimensional space cylinder radius.

[0155] In this embodiment, the spatial angle of the camera movement center can be used as the independent variable of the sine function to calculate the sine function value, and then the sine function value is multiplied by the preset multi-dimensional space cylinder radius, and the multiplication result is used as the multi-dimensional space vertical axis coordinate of the camera movement center.

[0156] Step S806 , calculating the multi-dimensional space vertical axis coordinate of the camera movement center according to the normalized vertical coordinate of the camera movement center and the preset multi-dimensional space cylinder radius.

[0157] In this embodiment, the normalized vertical coordinate of the camera movement center may be multiplied by a preset multi-dimensional space cylinder radius, and the multiplication result is used as the vertical axis coordinate of the camera movement center in the multi-dimensional space.

[0158] The camera movement video image generation method provided in the embodiment of the present application converts the target coordinates in the two-dimensional panoramic image into three-dimensional space, so that the virtual camera can accurately track the target, simulate the real broadcast perspective, enhance the audience's immersion in watching the game through the broadcast image, and let the audience feel a more realistic sense of three-dimensionality and depth.

[0159] Corresponding to the method of the above embodiment, Figure 9A structural block diagram of the camera movement video image generation device provided in an embodiment of the present application is shown. For the sake of convenience, only the parts related to the embodiment of the present application are shown. Figure 9 The exemplary camera movement video image generation device may be the execution subject of the camera movement video image generation method provided in the aforementioned embodiment 1.

[0160] Reference Figure 9 , the camera movement video image generating device includes:

[0161] An information acquisition module 910 is used to acquire panoramic video information and historical camera movement frame information;

[0162] An initial camera movement video image information generation module 920 is configured to generate initial camera movement video image information, ball position coordinate information, ball movement information, player position coordinate information, and player movement information based on the panoramic video information, historical camera movement frame information, a preset position detection model, and a preset movement state detection model.

[0163] A camera movement center coordinate information calculation module 930 is configured to analyze and process the ball position coordinate information, ball movement information, multiple player position coordinate information, and multiple player movement information based on the initial camera movement video frame information and historical camera movement frame information to obtain camera movement center coordinate information;

[0164] A zoom factor information calculation module 940 is configured to calculate zoom factor information based on the camera center coordinate information, preset panoramic video screen size information, and multiple preset video screen reference point information;

[0165] A camera movement center space conversion coordinate information calculation module 950 is used to calculate the camera movement center space conversion coordinate information based on the camera movement center coordinate information, preset panoramic video screen size information and preset multi-dimensional space conversion graphic information; and

[0166] The target camera movement video picture information generation module 960 is used to generate the target camera movement video picture information according to the camera movement center space conversion coordinate information, zoom factor information and a preset video picture rendering model.

[0167] The process of each module in the camera motion video generation device provided in the embodiment of the present application realizing its own function can be specifically referred to the aforementioned Figure 1 The description of the first embodiment will not be repeated here.

[0168] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0169] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0170] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0171] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0172] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish descriptions and should not be understood as indicating or implying relative importance. It should also be understood that although the terms "first", "second", etc. are used in the text to describe various elements in some embodiments of the present application, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first table can be named a second table, and similarly, a second table can be named a first table without departing from the scope of the various described embodiments. Both the first table and the second table are tables, but they are not the same table.

[0173] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0174] The method for generating a moving camera video provided in the embodiments of the present application can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). The embodiments of the present application do not impose any restrictions on the specific type of terminal device.

[0175] For example, the terminal device can be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a wireless modem card, a TV set-top box (STB), customer premise equipment (CPE) and / or other devices for communicating on a wireless system and a next-generation communication system, such as a mobile terminal in a 5G network or a mobile terminal in a future evolved Public Land Mobile Network (PLMN) network.

[0176] As an example and not a limitation, when the terminal device is a wearable device, the wearable device can also be a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are full-featured, large in size, and can achieve complete or partial functions without relying on smartphones, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0177] Figure 10This is a schematic diagram of the structure of a terminal device provided by an embodiment of the present application. Figure 10 As shown, the terminal device 100 of this embodiment includes: at least one processor 1000 ( Figure 10 Only one is shown), a memory 1001, wherein the memory 1001 stores a computer program 1002 that can be run on the processor 1000. When the processor 1000 executes the computer program 1002, the steps in the above-mentioned embodiments of the method for generating a moving mirror video are implemented, such as Figure 1 Alternatively, when the processor 1000 executes the computer program 1002, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 9 Functions of modules 910 to 960 are shown.

[0178] The terminal device 100 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 1000 and a memory 1001. Those skilled in the art will understand that Figure 10 It is only an example of the terminal device 100 and does not constitute a limitation of the terminal device 100. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include an input and sending device, a network access device, a bus, etc.

[0179] The processor 1000 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0180] In some embodiments, the memory 1001 may be an internal storage unit of the terminal device 100, such as a hard disk or memory of the terminal device 100. The memory 1001 may also be an external storage device of the terminal device 100, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the terminal device 100. Furthermore, the memory 1001 may include both an internal storage unit of the terminal device 100 and an external storage device. The memory 1001 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory 1001 may also be used to temporarily store data that has been sent or is to be sent.

[0181] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0182] An embodiment of the present application also provides a terminal device, which includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor. When the processor executes the computer program, the terminal device implements the steps of any of the above-mentioned method embodiments.

[0183] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0184] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0185] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the above-mentioned embodiment method by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0186] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0187] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0188] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0189] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for generating a camera-moving video image, characterized in that: include: Obtain panoramic video information and historical camera frame information; Generate initial camera movement video frame information, ball position coordinate information, ball movement information, player position coordinate information, and player movement information based on the panoramic video information, historical camera movement frame information, a preset position detection model, and a preset movement state detection model; Analyzing and processing the ball position coordinate information, the ball movement information, the multiple player position coordinate information, and the multiple player movement information based on the initial camera movement video frame information and the historical camera movement frame information to obtain camera movement center coordinate information; Calculating zoom factor information based on the camera center coordinate information, preset panoramic video screen size information, and multiple preset video screen reference point information; Calculating the spatial conversion coordinate information of the camera movement center according to the camera movement center coordinate information, the preset panoramic video screen size information, and the preset multi-dimensional space conversion graphic information; Generate target camera movement video image information according to the camera movement center space conversion coordinate information, zoom factor information and a preset video image rendering model; The historical camera movement picture frame information includes the previous frame video picture frame information; The previous frame video frame information includes the previous frame mirror movement center coordinate information; The football position coordinate information includes the current football position coordinate information and the previous frame football position coordinate information; The football movement information includes current football movement speed information, previous frame football movement direction information and previous frame football movement speed information; The player movement information includes player movement direction information; The step of analyzing and processing the football position coordinate information, the football movement information, the multiple player position coordinate information, and the multiple player movement information based on the initial camera movement video frame information and the historical camera movement frame information to obtain the camera movement center coordinate information specifically includes: Determine whether the current football position coordinate information exists in the initial camera movement video image information; If yes, obtaining the camera center coordinate information according to the current football movement speed information, the current football position coordinate information, the preset football speed threshold information and the preset player number parameter information; If not, obtaining the camera center coordinate information based on the previous frame video frame information, the previous frame football position coordinate information, the previous frame football movement speed information, the previous frame football movement direction information, the previous frame camera center coordinate information, the player position coordinate information, the player movement direction information, and a preset football speed threshold; The preset multi-dimensional space conversion graphic information includes a preset multi-dimensional space cylinder radius and a preset multi-dimensional space cylinder height.

2. The method for generating a camera-movement video according to claim 1, wherein: The step of obtaining the camera center coordinate information according to the current football movement speed information, the current football position coordinate information, the preset football speed threshold information, and the preset player number parameter information specifically includes: Determining whether the current football movement speed information is less than a preset football speed threshold information; If yes, then calculate the pixel distance between the current football position coordinate information and the player position coordinate information; obtain the camera center coordinate information based on the current football position coordinate information, the pixel distance, the player position coordinate information, and the preset player number parameter information; If not, the current football position coordinate information is used as the camera center coordinate information.

3. The method for generating a camera-movement video according to claim 2, wherein: The step of obtaining the camera movement center coordinate information according to the current football position coordinate information, the pixel distance value, the player position coordinate information, and the preset player number parameter information specifically includes: Arrange the pixel distance values in ascending order to obtain an arranged distance value; Filtering the arrangement distance values according to preset player number parameter information to obtain multiple player confirmation distance values; Extracting player position coordinate information corresponding to the player confirmation distance value to obtain camera movement center reference variable coordinate information; Calculate the center of gravity of the camera movement center reference variable coordinate information and the current football position coordinate information to obtain the camera movement center coordinate information.

4. The method for generating a camera-movement video according to claim 1, wherein: The step of obtaining the camera center coordinate information based on the previous frame video frame information, the previous frame football position coordinate information, the previous frame football movement speed information, the previous frame football movement direction information, the previous frame camera center coordinate information, the player position coordinate information, the player movement direction information, and a preset football speed threshold specifically includes: Determine whether the previous frame football position coordinate information exists in the previous frame video picture frame information; If yes, when the football movement speed information of the previous frame is less than the preset football speed threshold, the camera movement center coordinate information of the previous frame is used as the camera movement center coordinate information; When the football movement speed information of the previous frame is greater than or equal to a preset football speed threshold, obtaining camera movement area information based on the football movement direction information of the previous frame, the player position coordinate information, and the player movement direction information; calculating the midpoint coordinates of the camera movement area information to obtain camera movement center coordinate information; If not, the player movement direction ratio is calculated according to the player movement direction information; the camera center coordinate information is obtained according to the player movement direction information, the player movement direction ratio, the player position coordinate information, the preset player movement ratio threshold and the preset moving player number parameter information.

5. The method for generating a camera-movement video according to claim 4, wherein: The player movement direction information includes first movement direction information and second movement direction information; The first moving direction information is opposite to the second moving direction information; The step of obtaining the camera center coordinate information according to the player movement direction information, the player movement direction ratio, the player position coordinate information, the preset player movement ratio threshold, and the preset parameter information of the number of moving players specifically includes: Determining whether the player movement direction ratio is less than a preset player movement ratio threshold; If yes, then calculate the center of gravity of the player's position coordinate information to obtain the camera movement center coordinate information; If not, obtaining the first moving direction player quantity information and the first moving direction player position information corresponding to the first moving direction information, and the second moving direction player quantity information and the second moving direction player position information corresponding to the second moving direction information according to the player moving direction information; Determining whether the information about the number of players in the first moving direction is greater than the information about the number of players in the second moving direction; If yes, then filtering the player position information in the first moving direction according to the preset parameter information of the number of sports players to obtain the camera movement center calculation variable information; If not, filtering the player position information in the second moving direction according to the preset parameter information of the number of sports players to obtain the camera movement center calculation variable information; Calculate the center of gravity of the camera movement center calculation variable information to obtain the camera movement center coordinate information.

6. The method for generating a camera-movement video according to claim 1, wherein: The preset panoramic video screen size information includes preset panoramic video screen aspect ratio information; The mirror movement center coordinate information includes the mirror movement center horizontal coordinate information and the mirror movement center vertical coordinate information; The preset video picture reference point information includes preset reference original point information, preset reference near end point information and preset reference far end point information; The zoom factor information includes width zoom factor information and height zoom factor information; The step of calculating the zoom factor information according to the camera center coordinate information, the preset panoramic video screen size information, and the plurality of preset video screen reference point information specifically includes: Calculate the proximal boundary distance based on the preset reference original point information and the preset reference proximal end point information; Calculate the far-end boundary distance based on the preset reference original point information and the preset reference far-end point information; Calculating an initial distance value based on the mirror movement center coordinate information and preset reference original point information; The width scaling factor information is calculated according to the initial distance value, the near-end boundary distance, and the far-end boundary distance; and the height scaling factor information is calculated according to the width scaling factor information and preset panoramic video picture aspect ratio information.

7. The method for generating a camera-movement video according to claim 6, wherein: The preset panoramic video screen size information includes preset panoramic video screen width information and preset panoramic video screen height information; The spatial conversion coordinate information of the camera movement center includes the horizontal axis coordinate of the camera movement center multi-dimensional space, the vertical axis coordinate of the camera movement center multi-dimensional space, and the vertical axis coordinate of the camera movement center multi-dimensional space; The step of calculating the spatial conversion coordinate information of the camera movement center according to the camera movement center coordinate information, the preset panoramic video screen size information, and the preset multi-dimensional space conversion graphic information specifically includes: Normalizing the horizontal coordinate information of the camera movement center according to preset panoramic video picture width information to obtain a normalized horizontal coordinate of the camera movement center; Normalizing the vertical coordinate information of the camera movement center according to the preset panoramic video picture height information to obtain the normalized vertical coordinate of the camera movement center; Performing a spatial angle conversion process according to the normalized horizontal coordinate of the camera movement center to obtain the spatial angle of the camera movement center; Obtaining the multi-dimensional space horizontal axis coordinate of the camera movement center according to the camera movement center space angle and a preset multi-dimensional space cylinder radius; Obtaining the multi-dimensional space vertical axis coordinate of the camera movement center according to the camera movement center space angle and a preset multi-dimensional space cylinder radius; The multi-dimensional space vertical axis coordinate of the camera movement center is calculated according to the normalized vertical coordinate of the camera movement center and the preset multi-dimensional space cylinder radius.

8. A camera motion video generation device, characterized in that: include: An information acquisition module is used to obtain panoramic video information and historical camera movement frame information; an initial camera movement video image information generation module, configured to generate initial camera movement video image information, ball position coordinate information, ball movement information, player position coordinate information, and player movement information based on the panoramic video information, historical camera movement frame information, a preset position detection model, and a preset movement state detection model; a camera movement center coordinate information calculation module, configured to analyze and process the ball position coordinate information, ball movement information, multiple player position coordinate information, and multiple player movement information based on the initial camera movement video frame information and historical camera movement frame information to obtain camera movement center coordinate information; A zoom factor information calculation module, configured to calculate zoom factor information based on the camera center coordinate information, preset panoramic video screen size information, and a plurality of preset video screen reference point information; a camera movement center space conversion coordinate information calculation module, configured to calculate the camera movement center space conversion coordinate information based on the camera movement center coordinate information, preset panoramic video screen size information, and preset multi-dimensional space conversion graphic information; and A target camera movement video image information generation module is used to generate target camera movement video image information based on the camera movement center space conversion coordinate information, zoom factor information and a preset video image rendering model; The historical camera movement picture frame information includes the previous frame video picture frame information; The previous frame video frame information includes the previous frame mirror movement center coordinate information; The football position coordinate information includes the current football position coordinate information and the previous frame football position coordinate information; The football movement information includes current football movement speed information, previous frame football movement direction information and previous frame football movement speed information; The player movement information includes player movement direction information; The step of analyzing and processing the football position coordinate information, the football movement information, the multiple player position coordinate information, and the multiple player movement information based on the initial camera movement video frame information and the historical camera movement frame information to obtain the camera movement center coordinate information specifically includes: Determine whether the current football position coordinate information exists in the initial camera movement video image information; If yes, obtaining the camera center coordinate information according to the current football movement speed information, the current football position coordinate information, the preset football speed threshold information and the preset player number parameter information; If not, obtaining the camera center coordinate information based on the previous frame video frame information, the previous frame football position coordinate information, the previous frame football movement speed information, the previous frame football movement direction information, the previous frame camera center coordinate information, the player position coordinate information, the player movement direction information, and a preset football speed threshold; The preset multi-dimensional space conversion graphic information includes a preset multi-dimensional space cylinder radius and a preset multi-dimensional space cylinder height.

9. A terminal device, characterized in that: The terminal device includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images