Mirror moving video picture generation method and device and terminal equipment

Abstract

The invention provides a moving mirror video picture generation method and device and terminal equipment, and is suitable for the technical field of image communication. The method comprises the following steps: according to a panoramic video, a historical moving mirror picture frame, a position detection model and a moving state detection model, obtaining a moving mirror picture frame; generating initial mirror movement video picture information, football position coordinate information, football movement information, player position coordinate information and player movement information, and obtaining mirror movement center coordinate information; calculating zooming multiple information according to the mirror moving center coordinate information, the panoramic video picture size information and the video picture reference point information; and generating target mirror moving video picture information according to the mirror moving center coordinate information, the panoramic video size, the multi-dimensional space conversion graphic information, the zooming multiple information and the video picture rendering model. According to the application, the court target is automatically detected, the mirror moving center coordinate and the zoom multiple are calculated, the mirror moving video picture is generated through multi-dimensional space conversion and rendering, and the mirror moving flexibility, the rebroadcast picture quality and the crowd watching experience are improved.

Description

Technical Field

[0001] This application belongs to the technical field of image communication, and particularly relates to a method, device, and terminal device for generating a panning video screen. Background Art

[0002] In the prior art, the live broadcast of ball games relies on a fixed pan-tilt head or a camera with a preset rotation angle, or uses a player and ball detection algorithm model to obtain a detection rectangle, and then calculates the frame coordinates and panning coordinates based on simple calculations and threshold judgments, combines the detected area of the stadium to determine the panning rectangle range, or realizes the live broadcast of the event through a video acquisition, processing, and transmission module, combined with task tracking recognition and dynamic lens adjustment functions.

[0003] However, the fixed pan-tilt head or camera with a preset rotation angle relied on by the traditional live broadcast system has a perspective limitation in its physical structure, and problems such as key frame loss and blind spots in the picture are likely to occur in football games. At the same time, the equipment cost is high, the mechanical structure is prone to wear, jamming, or even failure, increasing the operation and maintenance costs and operation risks; calculating the frame coordinates and panning coordinates based on simple calculations and threshold judgments lacks the perception of the dynamic rhythm of the game. When there is a fast break or a tight defense, the lens switching is rigid or delayed, and the panning angle adjustment only depends on the fixed target position, without comprehensively considering other dynamic factors, and cannot meet the perspective needs of the audience for different scenarios; the live broadcast through the video acquisition, processing, and transmission module usually only involves single-dimensional adjustment, cannot achieve pitch angle adjustment, the live broadcast picture lacks a sense of three-dimensionality and depth, and cannot perform panning switching and dynamic zooming according to the complexity of the motion state of the ball game target. Summary of the Invention

[0004] In view of this, the embodiments of this application provide a method, device, and terminal device for generating a panning video screen, aiming to solve the problems in the existing panning methods that there are mechanical limitations and high operation and maintenance costs in the rotation and movement of the lens; the panning strategy is simple, it is difficult to adapt to dynamic scenarios, and the captured pictures are inaccurate; the panning method is single, and multi-angle and dynamic zooming cannot be achieved, thus reducing the viewing experience of the audience for the ball game.

[0005] The first aspect of the embodiments of this application provides a method for generating a panning video screen, including: Obtaining panoramic video information and historical panning video frame information; Generating initial panning video screen information, football position coordinate information, football movement information, player position coordinate information, and player movement information according to the panoramic video information, historical panning video frame information, a preset position detection model, and a preset movement state detection model; Analyze and process the football position coordinate information, football movement information, multiple player position coordinate information, and multiple player movement information based on the initial camera movement video frame information and historical camera movement frame information to obtain the camera movement center coordinate information; Calculate the zoom factor information based on the camera movement center coordinate information, the preset panoramic video frame size information, and multiple preset video frame reference point information; Calculate the camera movement center space transformation coordinate information based on the camera movement center coordinate information, the preset panoramic video frame size information, and the preset multi-dimensional space transformation graphic information; Generate the target camera movement video frame information based on the camera movement center space transformation coordinate information, the zoom factor information, and the preset video frame rendering model.

[0006] The second aspect of the embodiments of the present application provides a camera movement video frame generation device, including: An information acquisition module for acquiring panoramic video information and historical camera movement frame information; An initial camera movement video frame information generation module for generating initial camera movement video frame information, football position coordinate information, football movement information, player position coordinate information, and player movement information based on the panoramic video information, historical camera movement frame information, a preset position detection model, and a preset movement state detection model; A camera movement center coordinate information calculation module for analyzing and processing the football position coordinate information, football movement information, multiple player position coordinate information, and multiple player movement information based on the initial camera movement video frame information and historical camera movement frame information to obtain the camera movement center coordinate information; A zoom factor information calculation module for calculating the zoom factor information based on the camera movement center coordinate information, the preset panoramic video frame size information, and multiple preset video frame reference point information; A camera movement center space transformation coordinate information calculation module for calculating the camera movement center space transformation coordinate information based on the camera movement center coordinate information, the preset panoramic video frame size information, and the preset multi-dimensional space transformation graphic information; A target camera movement video frame information generation module for generating the target camera movement video frame information based on the camera movement center space transformation coordinate information, the zoom factor information, and the preset video frame rendering model.

[0007] The third aspect of the embodiments of the present application provides a terminal device, which includes a memory and a processor. A computer program is stored on the memory and can run on the processor. When the processor executes the computer program, the steps of the camera movement video frame generation method described in the first aspect above are implemented.

[0008] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: By automatically detecting the position information and movement state of the target in each frame of the panoramic video information of the ball game, based on the set complex rules, through the position information and movement state of the target, dynamically calculate the camera center coordinates and zoom ratio of each frame of the camera movement screen, and then through multi-dimensional space conversion and deep rendering model processing, generate the camera movement video screen frame by frame, so as to achieve a three-dimensional and dynamic camera movement effect, improve the flexibility and accuracy of automatic camera movement, and at the same time improve the quality level of the broadcast screen, bringing a more immersive and high-quality viewing experience to the audience. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0010] Figure 1 It is a schematic flowchart of the implementation of the camera movement video screen generation method provided in the first embodiment of the present application; Figure 2 It is a schematic flowchart of the implementation of the camera movement video screen generation method provided in the second embodiment of the present application; Figure 3 It is a schematic flowchart of the implementation of the camera movement video screen generation method provided in the third embodiment of the present application; Figure 4 It is a schematic flowchart of the implementation of the camera movement video screen generation method provided in the fourth embodiment of the present application; Figure 5 It is a schematic flowchart of the implementation of the camera movement video screen generation method provided in the fifth embodiment of the present application; Figure 6 It is a schematic flowchart of the implementation of the camera movement video screen generation method provided in the sixth embodiment of the present application; Figure 7 It is a schematic flowchart of the implementation of the camera movement video screen generation method provided in the seventh embodiment of the present application; Figure 8 It is a schematic flowchart of the implementation of the camera movement video screen generation method provided in the eighth embodiment of the present application; Figure 9 It is a schematic structural diagram of the camera movement video screen generation device provided in the embodiments of the present application; Figure 10 It is a schematic diagram of the terminal device provided in the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0011] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0012] In order to illustrate the technical solutions described in the present application, the following will be described through specific embodiments.

[0013] Figure 1 The implementation flowchart of the panning video frame generation method provided in Embodiment 1 of the present application is shown and described in detail as follows: Step S101, obtain panoramic video information and historical panning frame information.

[0014] In this embodiment, the panoramic video information can be obtained by shooting with a panoramic camera. Among them, the panoramic camera can be installed at a suitable position on the stadium, such as a high-point fixed bracket, etc., to ensure that the camera's field of view can completely cover the key areas of the stadium. It can be understood that the collected panoramic video data needs to be preliminarily processed, such as removing noise interference and color correction, to ensure the accuracy of operations such as position detection in subsequent steps. The historical panning frame information can be the broadcast frame information before the current frame, including the previous frame information.

[0015] Step S102, generate initial panning video frame information, football position coordinate information, football movement information, player position coordinate information, and player movement information according to the panoramic video information, historical panning frame information, a preset position detection model, and a preset movement state detection model.

[0016] In this embodiment, the preset position detection model can be an existing object detection model. Specifically, it can be composed of the YOLOv8 model and the DeepSORT model. Among them, the YOLOv8 model is used for object detection, that is, to identify and detect footballs and players in the panoramic video information. The DeepSORT model is used for object tracking, that is, to determine the position coordinate information of footballs and players in the panoramic video information. Furthermore, the football movement direction and the player movement direction can be calculated based on the position coordinate information of the football and the player in the previous frame and the next frame. Therefore, the position coordinate of the football and the position coordinate of the player in the panoramic video information can be determined through the position detection model. The preset movement state detection model can be designed based on the optical flow method, and is used to detect the movement speed of footballs and players on the field, so as to determine the football movement information and the player movement information in the panoramic video information. Among them, the football movement information includes the football movement speed and the football movement direction, and the player movement information includes the player movement speed and the player movement direction. Among them, the data for training the position detection model can be obtained by collecting 45 game data for a total of 72 hours, and randomly intercepting 10,000 panoramic pictures from the collected game videos, and then performing data annotation on the intercepted panoramic pictures to clarify information such as the position and category of the target, so as to obtain the data for training the position detection model. The initial camera movement video frame information can be generated based on the historical camera movement frame information, and can be generated based on the previous frame of camera movement picture, and is used to judge whether the football is tracked by the camera movement picture in the subsequent step of determining the camera movement center.

[0017] In this embodiment, the YOLOv8 algorithm can be used to train the model with the labeled panoramic picture training set. Input the panoramic picture, and the model learns the picture features to identify the positions and categories of players and footballs, outputs the detection frame and the confidence score, and adjusts the parameters during training to optimize the detection accuracy and speed. Furthermore, use the trained YOLOv8 model to perform object detection on the current frame image, obtain the detection frame and the confidence score of the players and footballs, and then input the detection results into the DeepSORT tracking model to track the players and footballs, and calculate the coordinate positions of the players and footballs in the two-dimensional panoramic video accordingly. Then, use the optical flow method to estimate the pixel movement between consecutive frames, obtain the movement speed of the football and the player on the image plane, and convert it into the actual physical movement speed in combination with the picture size and camera parameters. Finally, by comparing the position differences of the football and the player in the previous frame and the current frame, calculate the movement directions of the football and the player, and simplify the movement directions into two positive and negative directions.

[0018] Step S103, analyze and process the football position coordinate information, the football movement information, the multiple player position coordinate information, and the multiple player movement information according to the initial camera movement video frame information and the historical camera movement frame information, to obtain the camera movement center coordinate information.

[0019] In this embodiment, the camera movement center is the determined visual center of the camera lens in the live broadcast of a football game. The coordinate information of the camera movement center refers to the coordinate information of the point where the camera movement center is located in the camera movement video frame. It can be to first determine whether the football coordinate information of the current frame in the panoramic video information falls within the initial camera movement video frame information. If the football coordinate information of the current frame falls within the initial camera movement video frame information, then the specific coordinate information of the camera movement center can be determined according to the football movement information, and it can be to determine the coordinate information of the camera movement center according to the football movement speed information in the football movement information. If the football coordinate information of the current frame does not fall within the initial camera movement video frame information, it indicates that the football needs to be repositioned, and then the coordinate information of the camera movement center can be re-determined according to the football coordinate information situation. It can be to determine whether there is football coordinate information in the historical camera movement video frame information, and it can be to determine whether there is football coordinate information in the previous frame information of the historical camera movement video frame information, and then, according to the judgment result, in combination with the player position coordinate information and the player movement information, the determination of the coordinate information of the camera movement center is carried out.

[0020] Step S104, calculate the zoom factor information according to the coordinate information of the camera movement center, the preset panoramic video frame size information, and a plurality of preset video frame reference point information.

[0021] In this embodiment, the preset panoramic video frame size information may include the width information and height information of the panoramic video frame, or may include the aspect ratio information, where the aspect ratio information may be determined by the hardware device for the live broadcast. The preset video frame reference point information may be the artificially set initial point of the two-dimensional panoramic image, as well as the far-end point and near-end point from this initial point in the panoramic image. The zoom factor information may refer to the key parameter for adjusting the zoom effect of the live broadcast video frame in the live broadcast of a football game, and its setting needs to comprehensively consider normal situations and special scenarios to ensure that the audience can watch the game clearly. Among them, in normal situations, the default width of the camera lens is scaled to 0.5 times the original width. This setting is aimed at ensuring the stability of the video frame, presenting the game information completely, and meeting the basic viewing needs of the audience. In special scenarios, when the camera movement center is far from the initial point of the image, the width zoom factor needs to be dynamically adjusted according to the distance between the camera movement center and the video frame reference point. A plane rectangular coordinate system is established with the lower left corner of the two-dimensional panoramic image as the coordinate origin, and the near boundary distance and far boundary distance can be set. When the distance between the camera movement center and the initial point of the image reaches the near boundary distance, the width is scaled to 0.5 times; when the distance between the camera movement center and the initial point of the image reaches or is greater than the far boundary distance, the width zoom factor is reduced to 0.2 times. When it is in the intermediate state between the two, the linear interpolation formula can be used to determine the zoom factor.

[0022] Step S105: Calculate the coordinate information of the camera movement center in the transformed space based on the coordinate information of the camera movement center, the preset size information of the panoramic video screen, and the preset multi-dimensional space transformation graphic information.

[0023] In this embodiment, it can be understood that during the live broadcast process, 3D scene modeling is required to simulate the movement and perspective changes of the camera in the live broadcast of sports events. Then, a 3D model that conforms to the 180-degree panoramic picture live broadcast perspective needs to be set, that is, the preset multi-dimensional space transformation graphic information. The preset multi-dimensional space transformation graphic information can adopt a semi-cylindrical model, which is used to map the 180-degree panoramic image collected by the panoramic camera to the inner surface of the semi-cylindrical model. At the same time, the symmetric center point of the entire cylinder in the three-dimensional space is set as the camera position, so that the visualization effect can present any position on the stadium as the perspective of the live broadcast camera for the audience to watch the event. It can be understood that through the preset multi-dimensional space transformation graphic information, the two-dimensional coordinate information of the camera movement center is mapped into the three-dimensional space, and the obtained coordinate information is used as the coordinate information of the camera movement center in the transformed space.

[0024] Step S106: Generate the target camera movement video screen information based on the coordinate information of the camera movement center in the transformed space, the zoom factor information, and the preset video screen rendering model.

[0025] In this embodiment, the preset video screen rendering model can be set based on Three.js. Among them, Three.js is a library based on JavaScript, which enables the staff to quickly complete the rendering of the live broadcast screen without in-depth understanding of the underlying details of WebGL. It can be by inputting the coordinate information of the camera movement center in the transformed space and the zoom factor information into the preset video screen rendering model. The visualized screen input by the video screen rendering model is used as the target camera movement video screen information, which can be output to the terminal device used by the audience for real-time live broadcast of the event.

[0026] The method for generating the camera movement video screen provided by the embodiment of the present application automatically detects the position information and movement state of the target in each frame of the panoramic video information of the ball game. Based on the set complex rules, through the position information and movement state of the target, the coordinate of the camera movement center and the zoom factor of each frame of the camera movement screen are dynamically calculated. Then, through multi-dimensional space transformation and deep rendering model processing, the camera movement video screen is generated frame by frame, so as to achieve a three-dimensional and dynamic camera movement effect, improve the flexibility and accuracy of automatic camera movement, and at the same time improve the quality level of the live broadcast screen, bringing a more immersive and high-quality viewing experience to the audience.

[0027] Figure 2The figure shows an implementation flowchart of the method for generating a camera movement video screen provided in the second embodiment of the present application. The difference from the first embodiment above is as follows: The historical camera movement screen frame information includes the previous frame video screen frame information; the previous frame video screen frame information includes the previous frame camera movement center coordinate information; The football position coordinate information includes the current football position coordinate information and the previous frame football position coordinate information; The football movement information includes the current football movement speed information, the previous frame football movement direction information, and the previous frame football movement speed information; The player movement information includes the player movement direction information; Step S103 specifically includes: Step S201: Determine whether the current football position coordinate information exists in the initial camera movement video screen information; if so, proceed to step S202; if not, proceed to step S203.

[0028] In this embodiment, the current football position coordinate information refers to the football position coordinate information in the picture captured by the current panoramic camera. If the current football position coordinate information exists in the initial camera movement video screen information, it means that the football has not been lost during the camera movement. Therefore, the camera movement center coordinate information can be determined according to the football movement speed information; if the current football position coordinate information does not exist in the initial camera movement video screen information, it means that the football has moved during the camera movement and needs to be repositioned. Then, by repositioning the football, the camera movement center is determined again. Therefore, it is necessary to determine whether there is football position coordinate information in the previous frame camera movement screen, and then further determine the specific position coordinates of the camera movement center.

[0029] Step S202: Obtain the camera movement center coordinate information according to the current football movement speed information, the current football position coordinate information, the preset football speed threshold information, and the preset player number parameter information.

[0030] In this embodiment, the preset football speed threshold information can be 15 m / s. It can be understood that if the current football movement speed information is less than the preset football speed threshold information, it indicates that the current ball speed is relatively low and it is not easy for the football to have a large-scale transfer. Therefore, a shooting area can be determined based on the position coordinates of multiple moving players, and the center of gravity of this shooting area can be calculated as the shooting center coordinate information. It is also possible to determine multiple dynamically moving coordinates and then calculate the center of gravity of multiple position coordinates as the shooting center coordinate information. Among them, the shooting area can be a shooting area composed of the football coordinates and the coordinates of multiple players closest to the football. Among them, the position coordinates of multiple players closest to the football can be the position coordinates of 8 players closest to the football. If the current football movement speed information is greater than or equal to the preset football speed threshold information, it indicates that the current football speed is relatively high and it is easy to lose track. Therefore, the current football coordinate information can be used as the shooting center coordinate information.

[0031] Step S203: Obtain the shooting center coordinate information according to the previous frame video picture frame information, the previous frame football position coordinate information, the previous frame football movement speed information, the previous frame football movement direction information, the previous frame shooting center coordinate information, the player position coordinate information, the player movement direction information, and the preset football speed threshold.

[0032] In this embodiment, if the current football position coordinate information does not exist in the initial shooting video picture information, it indicates that the football has had a large-scale transfer during the shooting process. Therefore, it is necessary to re-track the football. Then, it is necessary to determine whether there is football position coordinate information in the previous frame shooting picture, and further determine the specific position coordinates of the shooting center according to the previous frame football movement speed information, the previous frame football movement direction information, the previous frame shooting center coordinate information, the player position coordinate information, and the player movement direction information.

[0033] The shooting video picture generation method provided by the embodiment of the present application determines the shooting center according to the ball speed when detecting the ball. When the ball speed is low, the center of gravity is determined by combining the ball and the player positions. When the ball speed is high, the ball coordinates are used as the center. This can ensure that the picture not only includes the dynamics of the ball but also takes into account the player situation, fully demonstrating the key content of the game. When the ball is not detected, the shooting center is determined based on the previous frame situation and the player movement state, ensuring the coherence of the picture and not interrupting the broadcast logic due to the temporary loss of the ball. Therefore, during the actual football game broadcast process, the shooting flexibility is improved, the adaptability of the broadcast picture to complex game situations is enhanced, the best viewing angle is provided for the audience, and the broadcast quality and the audience's viewing experience are greatly improved.

[0034] Figure 3 Fig. shows the implementation flowchart of the shooting video picture generation method provided by Embodiment 3 of the present application, and the difference from the above Embodiment 2 is as follows: The step S202 specifically includes: Step S301: Determine whether the current football movement speed information is less than the preset football speed threshold information. If so, proceed to Step S302; if not, proceed to Step S304.

[0035] In this embodiment, the preset football speed threshold information can be 15 m / s. If the current football movement speed information is less than the preset football speed threshold information, it indicates that the current football speed is relatively low and the football has not undergone a large-scale transfer. Then, the current football position coordinate information and the position coordinate information of multiple players closest to the football can be determined. Furthermore, the center of gravity of the current football position coordinate information and the position coordinate information of multiple players is calculated as the camera movement center coordinate information. If the current football movement speed information is greater than or equal to the preset football speed threshold information, it indicates that the current football speed is relatively high and is prone to large-scale transfer. Then, the current football position coordinate information can be used as the camera movement center coordinate information.

[0036] Step S302: Calculate the pixel distance value between the current football position coordinate information and the player position coordinate information.

[0037] In this embodiment, it can be to calculate the distance value between the current football position coordinate information and the position coordinate information of all players. It can be to calculate the straight-line distance, which is used to measure the distance value between the pixels where the football is located and the pixels where all players are located in the current frame of the image, that is, the pixel distance value.

[0038] Step S303: Obtain the camera movement center coordinate information based on the current football position coordinate information, the pixel distance value, the player position coordinate information, and the preset number of players parameter information.

[0039] In this embodiment, the preset number of players parameter information can be 8. It can be to sort the pixel distance values from smallest to largest, select the position coordinate information of the players corresponding to the first 8 pixel distance values, and then calculate the center of gravity coordinate of these 8 player position coordinate information and the current football position coordinate information. Then, this center of gravity coordinate is used as the camera movement center coordinate information.

[0040] Step S304: Use the current football position coordinate information as the camera movement center coordinate information.

[0041] In this embodiment, if the current football movement speed information is greater than or equal to the preset football speed threshold information, it indicates that the current football speed is relatively high and is extremely likely to be lost. Then, the current football position coordinate information can be used as the camera movement center coordinate information.

[0042] The camera movement video image generation method provided in the embodiment of the present application, when the ball speed is low, by calculating the pixel distance between the football and the player, selecting the specific player coordinates and the football coordinates to obtain the center of gravity as the camera movement center, so that the broadcast image can take into account the dynamic changes of the football and the player, present the game details to the audience, and let the audience fully understand the situation on the court in real time; when the ball speed is high, using the football coordinates as the camera movement center, it is possible to quickly capture the movement trajectory information of the football, thereby avoiding the loss of key images, so as to achieve the adjustment of the camera movement center according to different ball speeds, show the overall game situation through the broadcast image when the ball speed is slow, and show the high-speed movement of the football through the broadcast image when the ball speed is fast, which meets the audience's viewing expectations for the game images, enhances the immersion and tension of watching the game, and improves the viewing and attractiveness of the broadcast images.

[0043] Figure 4 The flowchart of the method for generating a moving camera video provided in the fourth embodiment of the present application is shown, which differs from the third embodiment in that: The step S303 specifically includes: Step S401, arranging the pixel distance values ​​in ascending order to obtain arranged distance values.

[0044] In this embodiment, the pixel distance values ​​may be arranged in ascending order, and the arranged pixel distance values ​​are used as the arranged distance values.

[0045] Step S402, screening the arrangement distance values ​​according to preset player number parameter information to obtain multiple player confirmation distance values.

[0046] In this embodiment, the preset player number parameter information may be 8. The arrangement distance values ​​may be screened to obtain the first 8 arrangement distance values ​​as the player confirmation distance values.

[0047] Step S403, extracting the player position coordinate information corresponding to the player confirmation distance value to obtain the camera movement center reference variable coordinate information.

[0048] In this embodiment, the preset player number parameter information can be 8, and the player position coordinate information corresponding to the first 8 pixel distance values ​​is filtered out as the camera movement center reference variable coordinate information, which is used to calculate and determine the camera movement center coordinate information in subsequent steps.

[0049] Step S404, calculating the center of gravity of the camera movement center reference variable coordinate information and the current football position coordinate information to obtain the camera movement center coordinate information.

[0050] In this embodiment, the center of gravity of the camera movement center reference variable coordinate information and the current football position coordinate information may be calculated, and the calculated center of gravity may be used as the camera movement center coordinate information.

[0051] The camera movement video frame generation method provided by the embodiments of the present application can take into account the dynamic movement of the football and the players while the positions of the football and the players are constantly changing, so that the selection of the camera movement center can make the broadcast frame include the football and the key players around the football, thereby comprehensively presenting the tactical cooperation, positional relationship and other game information among the players, enabling the audience to quickly grasp the game situation from the broadcast frame, and accurately presenting the core area frame when the players are attacking or defending, avoiding jerky or delayed picture switching, providing the audience with a smooth and focused viewing experience, and greatly improving the broadcast quality and the audience's viewing feeling.

[0052] Figure 5 The implementation flowchart of the camera movement video frame generation method provided by the fifth embodiment of the present application is shown. The difference from the second embodiment above is as follows: The step S203 specifically includes: Step S501, determining whether the football position coordinate information of the previous frame exists in the previous frame video frame information; if so, proceed to step S502; if not, proceed to step S506.

[0053] In this embodiment, if the football position coordinate information is not detected in the current frame image, the camera movement center of the current broadcast frame needs to be determined according to the previous broadcast frame. If the football position coordinate information of the current frame exists in the previous frame video frame information, it means that the football can be tracked in the previous frame image. Then, the camera movement center is determined around the football in the previous frame image to ensure that the camera movement frame can capture the football position. If the football position coordinate information of the current frame does not exist in the previous frame video frame information, it means that the football cannot be tracked in the previous frame image. Then, the camera movement center needs to be determined according to the movement direction of the players to increase the probability that the camera movement frame can capture the football position.

[0054] Step S502, determining whether the football movement speed information of the previous frame is less than a preset football speed threshold; if so, proceed to step S503; if not, proceed to step S504.

[0055] In this embodiment, the preset football speed threshold may be 15 m / s. If the football movement speed information in the previous frame is less than the preset football speed threshold, it indicates that the football movement speed in the previous frame is relatively low. Then, the current football position is most likely around the previous frame or blocked. Therefore, the camera movement center can be kept unchanged, that is, the camera movement center of the previous frame is used as the camera movement center of the current frame. If the football movement speed information in the previous frame is greater than or equal to the preset football speed threshold, it indicates that the football movement speed in the previous frame is relatively high. The current football position is most likely far from the range covered by the previous frame. Then, record the position information and movement direction of the football in the previous frame. At the same time, in the current frame, based on the football position in the previous frame along the football movement direction, find the player located at the edge of this direction as a reference point. Use the position of the football in the previous frame and the reference point in the current frame as boundaries, and the middle area as the camera movement area. Select the center of the camera movement area as the camera movement center.

[0056] Step S503: Use the coordinate information of the previous-frame camera movement center as the coordinate information of the camera movement center.

[0057] In this embodiment, if the football movement speed information in the previous frame is less than the preset football speed threshold, it indicates that the football movement speed in the previous frame is relatively low. Then, the current football position is most likely around the previous frame or blocked. Therefore, the camera movement center can be kept unchanged, that is, the camera movement center of the previous frame is used as the camera movement center of the current frame.

[0058] Step S504: Obtain the camera movement area information according to the previous-frame football movement direction information, player position coordinate information, and player movement direction information.

[0059] In this embodiment, if the football movement speed information in the previous frame is greater than or equal to the preset football speed threshold, it indicates that the football movement speed in the previous frame is relatively high. The current football position is most likely far from the range covered by the previous frame. Then, record the position information and movement direction of the football in the previous frame. At the same time, in the current frame, based on the football position in the previous frame along the football movement direction, find the player located at the edge of this direction as a reference point. Use the position of the football in the previous frame and the reference point in the current frame as boundaries, and the middle area information as the camera movement area information.

[0060] Step S505: Calculate the midpoint coordinates of the camera movement area information to obtain the coordinate information of the camera movement center.

[0061] In this embodiment, extract the center coordinates of the camera movement area information as the coordinate information of the camera movement center.

[0062] Step S506: Calculate the player movement direction ratio according to the player movement direction information.

[0063] In this embodiment, if the ball is not detected in the previous frame either, all the player movement direction information is counted, and the player movement direction ratio is calculated. For example, if there are m players moving to the left and n players moving to the right, the player movement direction ratio can be expressed as m / (n + m) and n / (n + m).

[0064] Step S507: Obtain the camera movement center coordinate information according to the player movement direction information, the player movement direction ratio, the player position coordinate information, a preset player movement ratio threshold, and preset moving player quantity parameter information.

[0065] In this embodiment, the preset player movement ratio threshold can be 0.65 or 0.3. If the ratios of all the players on the field in the movement direction are relatively balanced (the difference in ratios does not exceed the player movement ratio threshold), it indicates that there is no offensive or defensive trend in a single direction on the whole field. At this time, the center of gravity of all the player coordinates can be obtained and used as the camera movement center coordinate information for the current frame; when the number of players in a certain movement direction on the field is much higher than that in other directions (the difference in ratios is greater than the preset player movement ratio threshold), it indicates that there is an obvious offensive or defensive trend in a local area on the field. Then, the player coordinate information with the same movement direction is screened, and the center of gravity of the screened player coordinate information is obtained and used as the camera movement center coordinate information.

[0066] The camera movement video frame generation method provided by the embodiment of the present application is determined by combining whether the ball is detected in the previous frame and the ball speed, which can ensure the continuity and stability of the broadcast picture. Since the state of the ball in a football game is changeable, if the ball is detected in the previous frame, the ball speed is low and it is judged to be blocked, the camera movement center remains unchanged, avoiding frequent changes of the picture due to the ball being temporarily blocked, and the audience can continuously focus on the picture without missing the actions and situation changes of the surrounding players; if the ball speed is high, based on the position and movement direction of the ball in the previous frame, and combined with the edge players in the current frame to determine the camera movement center, it can quickly track the high-speed movement trajectory of the ball. Even if the ball is partially blocked or quickly passes through the picture in the current frame, the camera can still be stably focused on the key area, improving the adaptability of the camera movement picture to complex game situations, providing the best viewing angle for the audience, and enhancing the viewing experience.

[0067] Figure 6 The implementation flowchart of the camera movement video frame generation method provided by the sixth embodiment of the present application is shown, and its difference from the fifth embodiment above is that: The player movement direction information includes first movement direction information and second movement direction information; the first movement direction information is opposite to the second movement direction information; The step S507 specifically includes: Step S601: Judge whether the player movement direction ratio is less than a preset player movement ratio threshold; if so, go to step S602; if not, go to step S603.

[0068] In this embodiment, the preset player movement ratio threshold can be 0.65 or 0.3. When the player movement direction ratio is less than the preset player movement ratio threshold, it indicates that the ratios of players moving in both directions are less than the player movement ratio threshold, that is, there is no offensive or defensive trend in a single direction on the whole field. At this time, the centroid of all player coordinates can be calculated, and this centroid is used as the camera movement center coordinate information for the current frame. When the player movement direction ratio is greater than or equal to the preset player movement ratio threshold, it indicates that among the multiple players moving in both directions, the ratio of players in one direction is greater than the player movement ratio threshold, that is, there is an obvious offensive or defensive trend in a local area on the field. Then, the player coordinate information with the same movement direction is screened, and the centroid of the screened player coordinate information is calculated, and this center is used as the camera movement center coordinate information.

[0069] In this embodiment, alternatively, it can also be to calculate the difference between the number information of players in the first movement direction and the number information of players in the second movement direction. The preset player movement ratio threshold can also be the threshold of the difference between the number information of players in the first movement direction and the number information of players in the second movement direction. Among them, the preset player movement ratio threshold can take the value of 0.65 or 0.3. Then, the size relationship between the difference between the number information of players in the first movement direction and the number information of players in the second movement direction and the player movement ratio threshold is judged to determine whether there is an obvious offensive or defensive trend on the field.

[0070] Step S602: Calculate the centroid of the player position coordinate information to obtain the camera movement center coordinate information.

[0071] In this embodiment, when the player movement direction ratio is less than the preset player movement ratio threshold, it indicates that there is no offensive or defensive trend in a single direction on the whole field. At this time, the centroid of all player coordinates is calculated, and this centroid is used as the camera movement center coordinate information for the current frame.

[0072] Step S603: According to the player movement direction information, obtain the number information of players in the first movement direction corresponding to the first movement direction information, the position information of players in the first movement direction, the number information of players in the second movement direction corresponding to the second movement direction information, and the position information of players in the second movement direction.

[0073] In this embodiment, when the player movement direction ratio is greater than or equal to the preset player movement ratio threshold, it indicates that there is an obvious offensive and defensive trend in a local area on the field. Then, the number information of players in the first movement direction corresponding to the first movement direction information, the position information of players in the first movement direction, the number information of players in the second movement direction corresponding to the second movement direction information, and the position information of players in the second movement direction can be counted. Among them, the first movement direction information can be the left direction, the second movement direction information can be the right direction, the number information of players in the first movement direction corresponding to the first movement direction information can be the number of players moving to the left, the position information of players in the first movement direction represents the coordinates of players moving to the left, the number information of players in the second movement direction corresponding to the second movement direction information can be the number of players moving to the right, and the position information of players in the second movement direction represents the coordinates of players moving to the right.

[0074] Step S604, determine whether the number information of players in the first movement direction is greater than the number information of players in the second movement direction; if so, proceed to step S605; if not, proceed to step S606.

[0075] In this embodiment, when the number information of players in the first movement direction is greater than the number information of players in the second movement direction, it is necessary to further determine whether there is an obvious offensive and defensive trend in the on-field situation.

[0076] Step S605, according to the preset number parameter information of moving players, perform screening processing on the position information of players in the first movement direction to obtain the calculation variable information of the camera movement center.

[0077] In this embodiment, the preset number parameter information of moving players can be half of the number of players moving along the first movement direction, or half of the total number of players on the field. It can be to screen out all the position information of players moving along the first movement direction, and then screen out the players in the front half of the total number of players moving along the first movement direction, and use the coordinate information of these players in the front half of the total number as the calculation variable information of the camera movement center, and perform centroid calculation in the subsequent process, and use the calculated centroid as the calculation variable information of the camera movement center.

[0078] Step S606, according to the preset number parameter information of moving players, perform screening processing on the position information of players in the second movement direction to obtain the calculation variable information of the camera movement center.

[0079] In this embodiment, the preset number parameter information of the moving players can be half of the number of players moving along the second moving direction, or half of the total number of players on the field. It can be that after screening out all the position information of the players moving along the second moving direction, the first half of the total number of players moving along the second moving direction are screened out, and the coordinate information of these half of the total number of players is used as the calculation variable information of the camera movement center. Then, the center of gravity is calculated subsequently, and the calculated center of gravity is used as the calculation variable information of the camera movement center.

[0080] Step S607, calculate the center of gravity of the calculation variable information of the camera movement center to obtain the camera movement center coordinate information.

[0081] In this embodiment, the center of gravity coordinates of all the calculation variable information of the camera movement center are calculated, and the calculated center of gravity is used as the camera movement center coordinate information, that is, the coordinate information of this center of gravity is used as the camera movement center coordinate information of the current frame to realize the live broadcast of football matches.

[0082] The method for generating a camera movement video picture provided by the embodiment of the present application can comprehensively reflect the offensive and defensive postures on the field by judging the movement directions of the players, thereby calculating the center of gravity of multiple players in the front position and using this center of gravity as the camera movement center, so that the dynamic situations of the players in the key offensive or defensive positions can be presented by the live broadcast picture, accurately capture the core area of the game, and accurately present to the audience the exciting moments such as confrontations and cooperation in the core area and the game trend, improving the flexibility of the camera movement live broadcast picture, the live broadcast quality, and the viewing experience of the audience.

[0083] Figure 7 The implementation flowchart of the method for generating a camera movement video picture provided by the seventh embodiment of the present application is shown, and the difference from the first embodiment above is as follows: The preset panoramic video picture size information includes the preset panoramic video picture aspect ratio information; The camera movement center coordinate information includes the camera movement center abscissa information and the camera movement center ordinate information; The preset video picture reference point information includes the preset reference original point information, the preset reference near-end point information, and the preset reference far-end point information; The zoom ratio information includes the width zoom ratio information and the height zoom ratio information; The step S104 specifically includes: Step S701, calculate the proximal boundary distance according to the preset reference original point information and the preset reference near-end point information.

[0084] In this embodiment, the two-dimensional panoramic image can be rectangular. The rectangle has a width and a height. A plane rectangular coordinate system can be established with the lower left corner of the two-dimensional panoramic image as the origin of the coordinate system, where the X-axis represents the right side and the Y-axis represents the upper side. The preset reference original point information can be the midpoint on the bottom side of the rectangle, that is, the point with the abscissa being half of the width of the bottom side of the rectangle and the ordinate being zero. The preset reference near point can be set artificially. The abscissa can be half of the width of the bottom side of the rectangle, and the ordinate can be half of the height of the rectangle. The Pythagorean theorem can be used to calculate the straight-line distance between the reference original point and the reference near point through the abscissa and ordinate information of the preset reference original point information and the preset reference near point information, and use it as the near-end boundary distance.

[0085] Step S702, calculate the far-end boundary distance according to the preset reference original point information and the preset reference far point information.

[0086] In this embodiment, the preset reference near point can be set artificially. The abscissa can be the width of the bottom side, and the ordinate can be 0. The Pythagorean theorem can be used to calculate the straight-line distance between the reference original point and the reference far point through the abscissa and ordinate information of the preset reference original point information and the preset reference far point information, and use it as the far-end boundary distance.

[0087] Step S703, calculate the initial distance value according to the lens movement center coordinate information and the preset reference original point information.

[0088] In this embodiment, based on the Pythagorean theorem, the straight-line distance between the lens movement center coordinate information and the reference original point information can be calculated as the initial distance value.

[0089] Step S704, calculate the width scaling multiple information according to the initial distance value, the near-end boundary distance, and the far-end boundary distance.

[0090] In this embodiment, the calculation method of the width scaling multiple information can be:

[0091] where ux represents the width scaling multiple information, d represents the initial distance value, and d = min(d, ), d = max( , ), where represents the near-end boundary distance, represents the far-end boundary distance.

[0092] Step S705, calculate the height scaling multiple information according to the width scaling multiple information and the preset aspect ratio information of the panoramic video frame.

[0093] In this embodiment, the calculation method of the height scaling factor information may be as follows: ux where k represents the preset aspect ratio information of the panoramic video frame, which can be determined by the size of the broadcast device and is used to represent the aspect ratio of the broadcast device.

[0094] For the method for generating a panning video frame provided by the embodiments of the present application, when the panning center is far from the initial image point, the scaling factor is dynamically adjusted according to the distance between the panning center and the reference original point, ensuring that the audience can clearly see the game details through the broadcast screen. At the same time, due to the complex and changeable football game scenarios and different focuses of attention at different game moments, by dynamically adjusting the scaling factor, it can flexibly adapt to the game conditions at different game moments, and always provide the most suitable viewing angle for the audience during the game, making the broadcast screen more adaptable to the game rhythm and the needs of the audience.

[0095] Figure 8 FIG. shows the implementation flowchart of the method for generating a panning video frame provided by Embodiment 8 of the present application, and its difference from Embodiment 7 described above is as follows: The preset panoramic video frame size information includes the preset panoramic video frame width information and the preset panoramic video frame height information; The preset multi-dimensional space conversion graphic information includes the preset multi-dimensional space cylinder radius and the preset multi-dimensional space cylinder height; The panning center space conversion coordinate information includes the panning center multi-dimensional space horizontal axis coordinate, the panning center multi-dimensional space vertical axis coordinate, and the panning center multi-dimensional space vertical axis coordinate; Step S105 specifically includes: Step S801: Normalize the panning center abscissa information according to the preset panoramic video frame width information to obtain the normalized panning center abscissa.

[0096] In this embodiment, the preset panoramic video frame width information refers to the width information of the panoramic video image, which can be determined by the size of the hardware device. It can be calculated by dividing the panning center abscissa information by the preset panoramic video frame width information to obtain the normalized panning center abscissa.

[0097] Step S802: Normalize the panning center ordinate information according to the preset panoramic video frame height information to obtain the normalized panning center ordinate.

[0098] In this embodiment, the preset panoramic video frame height information refers to the height information of the panoramic video image, which can be determined by the size of the hardware device. It can be calculated by dividing the panning center ordinate information by the preset panoramic video frame height information to obtain the normalized panning center ordinate.

[0099] Step S803: Perform spatial angle conversion processing according to the normalized abscissa of the camera movement center to obtain the spatial angle of the camera movement center.

[0100] In this embodiment, it may be to multiply the normalized abscissa of the camera movement center by to calculate and obtain the spatial angle of the camera movement center.

[0101] Step S804: Obtain the multi-dimensional space horizontal axis coordinate of the camera movement center according to the spatial angle of the camera movement center and the preset multi-dimensional space cylinder radius.

[0102] In this embodiment, the preset multi-dimensional space cylinder radius can be set manually or automatically generated based on Three.js. It may be to use the spatial angle of the camera movement center as the independent variable of the cosine function to calculate the cosine function value, and then multiply the cosine function value by the preset multi-dimensional space cylinder radius, and the multiplication result is used as the multi-dimensional space horizontal axis coordinate of the camera movement center.

[0103] Step S805: Obtain the multi-dimensional space vertical axis coordinate of the camera movement center according to the spatial angle of the camera movement center and the preset multi-dimensional space cylinder radius.

[0104] In this embodiment, it may be to use the spatial angle of the camera movement center as the independent variable of the sine function to calculate the sine function value, and then multiply the sine function value by the preset multi-dimensional space cylinder radius, and the multiplication result is used as the multi-dimensional space vertical axis coordinate of the camera movement center.

[0105] Step S806: Calculate the multi-dimensional space vertical axis coordinate of the camera movement center according to the normalized ordinate of the camera movement center and the preset multi-dimensional space cylinder radius.

[0106] In this embodiment, it may be to multiply the normalized ordinate of the camera movement center by the preset multi-dimensional space cylinder radius, and the multiplication result is used as the multi-dimensional space vertical axis coordinate of the camera movement center.

[0107] The method for generating a camera movement video picture provided by the embodiment of the present application converts the target coordinates in the two-dimensional panoramic image into a three-dimensional space, enabling the virtual camera to accurately track the target, simulating a real broadcast perspective, enhancing the immersion of the audience watching the game through the broadcast picture, and making the audience feel a more real sense of three-dimensionality and depth.

[0108] Corresponding to the method in the above embodiment, Figure 9 The structural block diagram of the camera movement video picture generating device provided by the embodiment of the present application is shown. For the sake of convenience of description, only the parts related to the embodiment of the present application are shown. Figure 9 The exemplary camera movement video picture generating device may be the execution subject of the camera movement video picture generating method provided in the foregoing Embodiment 1.

[0109] Referring to Figure 9 , the camera movement video frame generating device includes: An information acquisition module 910, configured to acquire panoramic video information and historical camera movement frame information; An initial camera movement video frame information generating module 920, configured to generate initial camera movement video frame information, football position coordinate information, football movement information, player position coordinate information, and player movement information according to the panoramic video information, historical camera movement frame information, a preset position detection model, and a preset movement state detection model; A camera movement center coordinate information calculation module 930, configured to analyze and process the football position coordinate information, football movement information, multiple player position coordinate information, and multiple player movement information according to the initial camera movement video frame information and historical camera movement frame information to obtain camera movement center coordinate information; A zoom ratio information calculation module 940, configured to calculate zoom ratio information according to the camera movement center coordinate information, a preset panoramic video frame size information, and multiple preset video frame reference point information; A camera movement center space conversion coordinate information calculation module 950, configured to calculate camera movement center space conversion coordinate information according to the camera movement center coordinate information, a preset panoramic video frame size information, and a preset multi-dimensional space conversion graphic information; and A target camera movement video frame information generating module 960, configured to generate target camera movement video frame information according to the camera movement center space conversion coordinate information, zoom ratio information, and a preset video frame rendering model.

[0110] In the camera movement video frame generating device provided by the embodiments of the present application, the process of each module realizing its respective functions can be specifically referred to the description of the foregoing Figure 1 shown in Embodiment 1, which will not be elaborated here.

[0111] It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0112] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0113] It should also be understood that the term "and / or" as used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0114] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" depending on the context.

[0115] In addition, in the description of the specification and appended claims of this application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and should not be construed as indicating or implying relative importance. It should also be understood that although the terms "first", "second", etc. are used in the text in some embodiments of this application to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, the first table can be named the second table, and similarly, the second table can be named the first table without departing from the scope of the various described embodiments. The first table and the second table are both tables, but they are not the same table.

[0116] Reference to "one embodiment" or "some embodiments" or the like described in the specification of this application means that a particular feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0117] The camera movement video frame generation method provided by the embodiments of this application can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of this application do not impose any restrictions on the specific types of terminal devices.

[0118] For example, the terminal device can be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication capabilities, a computing device, or other processing devices connected to a wireless modem, a vehicle-mounted device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a wireless modem card, a television set top box (STB), a customer premise equipment (CPE), and / or other devices for communicating on a wireless system, as well as next-generation communication systems, such as mobile terminals in a 5G network or mobile terminals in a future evolved Public Land Mobile Network (PLMN) network.

[0119] By way of example and not limitation, when the terminal device is a wearable device, the wearable device can also be a general term for devices that apply wearable technology to the intelligent design of daily wear and develop wearable devices, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is either directly worn on the body or integrated into the user's clothes or accessories. A wearable device is not just a hardware device, but also realizes powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable intelligent devices include those with complete functions and large sizes that can achieve complete or partial functions without relying on a smart phone, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to cooperate with other devices such as smart phones, such as various smart bracelets and smart jewelry for physical sign monitoring.

[0120] Figure 10It is a schematic structural diagram of a terminal device provided by an embodiment of the present application. As Figure 10 shown, the terminal device 100 of this embodiment includes: at least one processor 1000 ( Figure 10 only one is shown in the figure), a memory 1001, and a computer program 1002 that can run on the processor 1000 is stored in the memory 1001. When the processor 1000 executes the computer program 1002, it implements the steps in the above-mentioned embodiments of each method for generating a video screen with camera movement, such as Figure 1 the steps S101 to S106 shown. Alternatively, when the processor 1000 executes the computer program 1002, it implements the functions of each module / unit in the above-mentioned device embodiments, such as Figure 9 the functions of the modules 910 to 960 shown.

[0121] The terminal device 100 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor 1000 and a memory 1001. Those skilled in the art can understand that Figure 10 this is only an example of the terminal device 100, and does not constitute a limitation on the terminal device 100. It may include more or fewer components than those shown in the figure, or combine some components, or different components. For example, the terminal device may further include an input and sending device, a network access device, a bus, etc.

[0122] The so-called processor 1000 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may also be any conventional processor, etc.

[0123] In some embodiments, the memory 1001 may be an internal storage unit of the terminal device 100, such as the hard disk or memory of the terminal device 100. The memory 1001 may also be an external storage device of the terminal device 100, such as a plug-in hard disk equipped on the terminal device 100, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 1001 may also include both the internal storage unit of the terminal device 100 and an external storage device. The memory 1001 is used to store an operating system, application programs, a BootLoader, data, and other programs, such as the program code of the computer program. The memory 1001 may also be used to temporarily store data that has been sent or will be sent.

[0124] In addition, in each embodiment of the present application, each functional unit may be integrated in a processing unit, may exist separately physically for each unit, or two or more units may be integrated in one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0125] An embodiment of the present application further provides a terminal device, which includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor. When the processor executes the computer program, the terminal device implements the steps in any of the above method embodiments.

[0126] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the steps in each of the above method embodiments.

[0127] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, it enables the terminal device to implement the steps in each of the above method embodiments when executed.

[0128] When the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0129] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0130] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] The above-described embodiments are only used to illustrate the technical solutions of this application, rather than to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for generating a video frame with camera movement, characterized in that, Including: Obtaining panoramic video information and historical camera movement frame information; Generating initial camera movement video frame information, football position coordinate information, football movement information, player position coordinate information, and player movement information according to the panoramic video information, historical camera movement frame information, a preset position detection model, and a preset movement state detection model; Analyzing and processing the football position coordinate information, football movement information, multiple player position coordinate information, and multiple player movement information according to the initial camera movement video frame information and historical camera movement frame information to obtain camera movement center coordinate information; Calculating zoom factor information according to the camera movement center coordinate information, preset panoramic video frame size information, and multiple preset video frame reference point information; Calculating camera movement center space conversion coordinate information according to the camera movement center coordinate information, preset panoramic video frame size information, and preset multi-dimensional space conversion graphic information; Generating target camera movement video frame information according to the camera movement center space conversion coordinate information, zoom factor information, and preset video frame rendering model.

2. The method for generating a camera movement video picture according to claim 1, wherein, The historical camera movement frame information includes previous frame video frame information; the previous frame video frame information includes previous frame camera movement center coordinate information; The football position coordinate information includes current football position coordinate information and previous frame football position coordinate information; The football movement information includes current football movement speed information, previous frame football movement direction information, and previous frame football movement speed information; The player movement information includes player movement direction information; The step of analyzing and processing the football position coordinate information, football movement information, multiple player position coordinate information, and multiple player movement information according to the initial camera movement video frame information and historical camera movement frame information to obtain camera movement center coordinate information specifically includes: Judging whether the current football position coordinate information exists in the initial camera movement video frame information; If so, obtaining camera movement center coordinate information according to the current football movement speed information, current football position coordinate information, preset football speed threshold information, and preset player number parameter information; If not, obtaining camera movement center coordinate information according to the previous frame video frame information, previous frame football position coordinate information, previous frame football movement speed information, previous frame football movement direction information, previous frame camera movement center coordinate information, player position coordinate information, player movement direction information, and preset football speed threshold.

3. The method for generating a panning video frame according to claim 2, wherein The step of obtaining camera movement center coordinate information according to the current football movement speed information, current football position coordinate information, preset football speed threshold information, and preset player number parameter information specifically includes: Judging whether the current football movement speed information is less than the preset football speed threshold information; If so, calculating the pixel distance value between the current football position coordinate information and the player position coordinate information; Obtaining camera movement center coordinate information according to the current football position coordinate information, pixel distance value, player position coordinate information, and preset player number parameter information; Otherwise, use the current football position coordinate information as the camera movement center coordinate information.

4. The method for generating a camera movement video frame according to claim 3, wherein, The step of obtaining the camera movement center coordinate information according to the current football position coordinate information, pixel distance value, player position coordinate information, and preset player number parameter information specifically includes: Perform an ascending order arrangement process on the pixel distance value to obtain an arranged distance value; According to the preset player number parameter information, screen the arranged distance value to obtain multiple player confirmation distance values; Extract the player position coordinate information corresponding to the player confirmation distance value to obtain the camera movement center reference variable coordinate information; Calculate the centroid of the camera movement center reference variable coordinate information and the current football position coordinate information to obtain the camera movement center coordinate information.

5. The method for generating a panning video picture according to claim 2, wherein, The step of obtaining the camera movement center coordinate information according to the previous frame video picture frame information, previous frame football position coordinate information, previous frame football movement speed information, previous frame football movement direction information, previous frame camera movement center coordinate information, player position coordinate information, player movement direction information, and preset football speed threshold specifically includes: Judge whether the previous frame football position coordinate information exists in the previous frame video picture frame information; If so, when the previous frame football movement speed information is less than the preset football speed threshold, use the previous frame camera movement center coordinate information as the camera movement center coordinate information; When the previous frame football movement speed information is greater than or equal to the preset football speed threshold, obtain the camera movement area information according to the previous frame football movement direction information, player position coordinate information, and player movement direction information; Calculate the midpoint coordinate of the camera movement area information to obtain the camera movement center coordinate information; If not, calculate the player movement direction ratio according to the player movement direction information; Obtain the camera movement center coordinate information according to the player movement direction information, player movement direction ratio, player position coordinate information, preset player movement ratio threshold, and preset moving player number parameter information.

6. The method for generating a camera movement video picture according to claim 5, wherein: The player movement direction information includes first movement direction information and second movement direction information; the first movement direction information is opposite to the second movement direction information; The step of obtaining the camera movement center coordinate information according to the player movement direction information, player movement direction ratio, player position coordinate information, preset player movement ratio threshold, and preset moving player number parameter information specifically includes: Judge whether the player movement direction ratio is less than the preset player movement ratio threshold; If so, calculate the centroid of the player position coordinate information to obtain the camera movement center coordinate information; If not, according to the player movement direction information, obtain the first movement direction player number information corresponding to the first movement direction information, the first movement direction player position information, the second movement direction player number information corresponding to the second movement direction information, and the second movement direction player position information; Judge whether the first movement direction player number information is greater than the second movement direction player number information; If so, perform screening processing on the first moving direction player position information according to the preset number parameter information of moving players to obtain the calculation variable information of the camera movement center; If not, perform screening processing on the second moving direction player position information according to the preset number parameter information of moving players to obtain the calculation variable information of the camera movement center; Calculate the center of gravity of the calculation variable information of the camera movement center to obtain the camera movement center coordinate information.

7. The method for generating a camera movement video picture according to claim 1, wherein The preset panoramic video picture size information includes the preset panoramic video picture aspect ratio information; The camera movement center coordinate information includes the camera movement center abscissa information and the camera movement center ordinate information; The preset video picture reference point information includes the preset reference original point information, the preset reference near-end point information, and the preset reference far-end point information; The scaling factor information includes the width scaling factor information and the height scaling factor information; The step of calculating the scaling factor information according to the camera movement center coordinate information, the preset panoramic video picture size information, and multiple preset video picture reference point information specifically includes: Calculate the near-end boundary distance according to the preset reference original point information and the preset reference near-end point information; Calculate the far-end boundary distance according to the preset reference original point information and the preset reference far-end point information; Calculate the initial distance value according to the camera movement center coordinate information and the preset reference original point information; Calculate the width scaling factor information according to the initial distance value, the near-end boundary distance, and the far-end boundary distance; Calculate the height scaling factor information according to the width scaling factor information and the preset panoramic video picture aspect ratio information.

8. The method for generating a camera movement video picture according to claim 7, wherein The preset panoramic video picture size information includes the preset panoramic video picture width information and the preset panoramic video picture height information; The preset multi-dimensional space conversion graphic information includes the preset multi-dimensional space cylinder radius and the preset multi-dimensional space cylinder height; The camera movement center space conversion coordinate information includes the camera movement center multi-dimensional space horizontal axis coordinate, the camera movement center multi-dimensional space vertical axis coordinate, and the camera movement center multi-dimensional space vertical axis coordinate; The step of calculating the camera movement center space conversion coordinate information according to the camera movement center coordinate information, the preset panoramic video picture size information, and the preset multi-dimensional space conversion graphic information specifically includes: Perform normalization processing on the camera movement center abscissa information according to the preset panoramic video picture width information to obtain the normalized abscissa of the camera movement center; Perform normalization processing on the camera movement center ordinate information according to the preset panoramic video picture height information to obtain the normalized ordinate of the camera movement center; Perform spatial angle conversion processing according to the normalized abscissa of the camera movement center to obtain the camera movement center spatial angle; Obtain the camera movement center multi-dimensional space horizontal axis coordinate according to the camera movement center spatial angle and the preset multi-dimensional space cylinder radius; Obtain the camera movement center multi-dimensional space vertical axis coordinate according to the camera movement center spatial angle and the preset multi-dimensional space cylinder radius; Calculate the vertical axis coordinate of the camera movement center in the multi-dimensional space according to the normalized vertical coordinate of the camera movement center and the preset radius of the multi-dimensional space cylinder.

9. A shooting movement video picture generating device, characterized in that It includes: An information acquisition module, configured to acquire panoramic video information and historical camera movement frame information; An initial camera movement video frame information generation module, configured to generate initial camera movement video frame information, football position coordinate information, football movement information, player position coordinate information, and player movement information according to the panoramic video information, historical camera movement frame information, a preset position detection model, and a preset movement state detection model; A camera movement center coordinate information calculation module, configured to analyze and process the football position coordinate information, football movement information, multiple player position coordinate information, and multiple player movement information according to the initial camera movement video frame information and historical camera movement frame information to obtain camera movement center coordinate information; A zoom factor information calculation module, configured to calculate zoom factor information according to the camera movement center coordinate information, preset panoramic video frame size information, and multiple preset video frame reference point information; A camera movement center space conversion coordinate information calculation module, configured to calculate camera movement center space conversion coordinate information according to the camera movement center coordinate information, preset panoramic video frame size information, and preset multi-dimensional space conversion graphic information; and A target camera movement video frame information generation module, configured to generate target camera movement video frame information according to the camera movement center space conversion coordinate information, zoom factor information, and a preset video frame rendering model.

10. A terminal device, characterized in that, The terminal device includes a memory and a processor. A computer program that can run on the processor is stored on the memory. When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images