Video playing method, electronic device, readable storage medium and program product

Abstract

The application provides a video playing method, an electronic device, a readable storage medium and a program product. The method comprises the following steps: determining a backtracking frame sequence corresponding to a target event from a video stream of a target area; the video stream comprises a plurality of video frames sorted by time, each video frame comprising a plurality of preset camera shots from different perspectives; the plurality of preset cameras are arranged around the target area and shoot the target area; the backtracking frame sequence comprises a video frame corresponding to the target event and a plurality of video frames before the occurrence of the target event; determining a target perspective related to the target event from a plurality of perspectives corresponding to each video frame of the backtracking frame sequence; determining a perspective switching path corresponding to the target event according to the target perspective of each video frame; and controlling a video player to sequentially play target pictures corresponding to a plurality of target perspectives in the perspective switching path. The method can improve the fluency of video playing.

Description

Technical Field

[0001] This application relates to video processing technology, and more particularly to a video playback method, electronic device, readable storage medium, and program product. Background Technology

[0002] When watching live video broadcasts of sports events, concerts, etc., there are often some exciting moments that attract viewers. However, a single shooting perspective often misses the entire process of the event, affecting the user's overall viewing experience.

[0003] Currently, the mainstream video playback methods mainly include: deploying multiple cameras to capture images of the surrounding area, forming a 360-degree panoramic video playback system, and then stitching the various video streams captured by the 360-degree panoramic video playback system into a complete panoramic image. When an exciting event occurs, users can switch to different perspectives by swiping or tapping the screen, achieving a "360-degree surround view" experience and enhancing the viewing experience.

[0004] However, in current video playback solutions, it takes a long time for users to manually switch perspectives, the perspective switching process is not smooth enough, and due to accidental operation when manually switching perspectives, the switched screen may have low relevance to the exciting event, resulting in the video playback process not being able to smoothly play the event screen that the user wants to watch. Summary of the Invention

[0005] This application provides a video playback method, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the correlation between the played video and the target event, thereby improving the smoothness of video playback.

[0006] This application provides a video playback method, the method including: The backtracking frame sequence corresponding to the target event is determined from the video stream of the target area; the video stream includes multiple video frames ordered by time, each video frame including images captured from different angles by multiple preset cameras; the multiple preset cameras are arranged around the target area and capture images of the target area; the backtracking frame sequence includes the video frame corresponding to the target event, as well as multiple video frames before the target event occurred. Identify the target viewpoint related to the target event from multiple viewpoints corresponding to each video frame in the retrospective frame sequence; Based on the target viewpoint of each video frame, determine the viewpoint switching path corresponding to the target event; Control the video player to sequentially play the target images corresponding to multiple target perspectives in the perspective switching path.

[0007] In the above scheme, the target viewpoint related to the target event is determined from multiple viewpoints corresponding to each video frame in the retrospective frame sequence, including: For each video frame in the backtracking frame sequence, the quality of the images from different perspectives in the video frame is evaluated to obtain the image quality score corresponding to each perspective. Based on the image quality score corresponding to each viewpoint, multiple candidate viewpoints are determined from multiple viewpoints; The target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame.

[0008] In the above scheme, the quality of images from different perspectives in the video frame is evaluated to obtain an image quality score corresponding to each perspective, including: For each viewpoint corresponding to a video frame, a first image quality score, a second image quality score, and a third image quality score are determined for the image corresponding to each viewpoint. The first image quality score represents the sharpness of the corresponding image; the second image quality score represents whether there is an object of interest corresponding to the target event in the corresponding image; and the third image quality score represents whether the object of interest in the corresponding image is in a preset position and whether the object of interest is obscured by interference. The image quality score for each viewpoint is determined based on the first image quality score, the second image quality score, and the third image quality score.

[0009] In the above scheme, the target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame, including: Each video frame corresponds to multiple candidate viewpoints, which are then identified as nodes in a pre-defined graph model. Determine two adjacent nodes among multiple nodes, and the distance between the two adjacent nodes; the two adjacent nodes represent two video frames that are ordered adjacent in the video stream; If the distance meets the preset distance constraint, then add an edge between the two adjacent nodes; Based on the preset graph model with added edges, the target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame.

[0010] In the above scheme, based on the preset graph model after adding edges, the target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame, including: Based on multiple nodes and multiple edges in the preset graph model after adding edges, determine multiple candidate paths in the preset graph model after adding edges; For each candidate path, determine the image quality score corresponding to each node in the candidate path and the distance between each pair of adjacent nodes in the candidate path; Based on the image quality scores corresponding to all nodes in the candidate path, the image quality index corresponding to the candidate path is determined, and based on the distance between all two adjacent nodes in the candidate path, the smoothness of the viewpoint switching corresponding to the candidate path is determined. Based on the image quality index and viewpoint switching smoothness corresponding to each candidate path, the target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame.

[0011] In the above scheme, the target event is the event identified by a pre-trained event classification model; wherein, the pre-trained event classification model is used to determine the event type of the event occurring in the video frame; Determine the backtracking frame sequence of the target event from the video stream of the target region, including: For each video frame in the video stream of the target region, feature extraction is performed on the video frame to obtain video frame features; Based on video frame features, the types of events occurring in video frames are determined using a pre-trained event classification model; If the event type is a target event, the sequence of backtracking frames corresponding to the target event is determined based on the video frames and the preset number of backtracking frames.

[0012] In the above scheme, the target event is an event determined based on user behavior information; user behavior information includes the user's historical preference tags or the user's instructions on the target object on the playback interface; The target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame, including: For each video frame, the target viewpoint related to the target event is determined from multiple candidate viewpoints based on user behavior information.

[0013] This application provides an electronic device, which includes: Memory is used to store executable instructions or computer programs. When a processor executes computer-executable instructions or computer programs stored in memory, it implements the video playback method provided in the embodiments of this application.

[0014] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the video playback method provided in this application.

[0015] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the video playback method provided in this application.

[0016] The solution in this application embodiment determines the backtracking frame sequence corresponding to the target event from the video stream of the target area. The video stream includes multiple video frames ordered by time, and each video frame includes images captured from different perspectives by multiple preset cameras. The multiple preset cameras are arranged around the target area and capture images of the target area. The backtracking frame sequence includes the video frame corresponding to the target event and multiple video frames before the target event occurs. The target perspective related to the target event is determined from the multiple perspectives corresponding to each video frame in the backtracking frame sequence. Based on the target perspective of each video frame, the perspective switching path corresponding to the target event is determined. The video player is controlled to sequentially play the target images corresponding to the multiple target perspectives in the perspective switching path. The entire video playback process does not require manual control of perspective changes by the user. When a target event occurs, the backtracking frame sequence corresponding to the target event is automatically determined, and the target perspective corresponding to the target event in each video frame is selected from the backtracking frame sequence. Then, based on the target perspective corresponding to each video frame, a perspective switching path with high coherence for the target event is determined, which improves the correlation between the played video and the target event. This solves the problems of multi-view video playback systems relying on manual intervention in perspective selection and failing to achieve smooth playback, thus improving the smoothness of video playback. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the video playback system architecture provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the video playback method provided in an embodiment of this application; Figure 3 This is a schematic diagram of multiple preset cameras arranged around a target area, as provided in the embodiments of this application.

[0018] Figure 4 This is a schematic diagram illustrating the playback principle of a full-screen player provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] When watching live video broadcasts of sports events, concerts, etc., there are often some exciting moments that attract viewers the most. For example, when watching live sports events, exciting moments such as goals and scores can attract viewers.

[0021] Currently, mainstream video playback methods primarily involve deploying multiple cameras to capture a surrounding scene, creating a 360-degree panoramic video playback system, and then stitching the various video streams captured by this system into a complete panoramic view. When an exciting event occurs, users can switch to different perspectives by swiping or tapping the screen to view the desired scene, achieving a "360-degree surround view" experience and enhancing the viewing experience. However, this 360-degree panoramic video playback system only provides a basic perspective switching function, requiring users to manually switch viewing angles. Manual switching is time-consuming, and errors during manual switching can lead to the switched scene having low relevance to the exciting event, failing to automatically and smoothly recommend viewing angles tailored to the target event.

[0022] With the development of deep learning, some sports live streaming platforms have begun to use AI (Artificial Intelligence) models to broadcast matches in real time. For example, when exciting events occur, object detection algorithms or models are used to identify ball trajectories, athlete movements, and other exciting information, and then mark the highlights accordingly. These markers can be used to generate edited match highlights for users to watch on demand. Commonly used models include object detection models (YOLO (You Only Look Once), Faster R-CNN (Region-based Convolutional Neural Network)) and action recognition models (TSN (Time-Sensitive Networking), SlowFast (Bimodal Convolutional Neural Network)). However, such systems typically focus on content analysis from a single perspective and fail to combine multi-view video data for perspective selection. As a result, when reviewing highlights, the limitations of a single perspective may lead to parts of the footage that are not captured, such as the trajectory of the ball or the movements of the athletes. This can result in situations where a player appears in one frame but not in the next, indicating that the AI ​​model's automatic switching of footage from a single perspective has low relevance to the target event, making it difficult for users to smoothly watch the highlights of the target event.

[0023] Therefore, the videos played by the current video playback solutions have low relevance to the content of the events that need to be viewed, making it impossible to watch the videos smoothly.

[0024] In view of the shortcomings of the above-mentioned related technologies, this application provides a video playback method that can improve the correlation between the played video and the target event, thereby improving the smoothness of video playback. To illustrate the technical solution of the present invention, specific embodiments are described below.

[0025] This application provides a video playback method applied to a video playback system 1000. The video playback system 1000 can be located on a server 101 or a playback terminal device 102. The playback terminal device 102 communicates with the server 101 via 5G or a dedicated network. The video playback system 1000 also includes a multi-camera module 103, which includes multiple preset cameras. These preset cameras can communicate with both the playback terminal device 102 and the server 101. The preset cameras are arranged around a target area 104, capturing images of the target area 104 and sending the captured images to the communicating server 101 or the playback terminal device 102. For example, when the number of cameras is 36, each video frame includes images captured by the 36 cameras from 36 different perspectives. Furthermore, the playback terminal device 102 is also equipped with a video player 105, used to render the images to be played on the playback interface of the playback terminal device 102.

[0026] like Figure 1 As shown, the video playback system 1000 is set up on server 101. The multi-camera module 103 has four cameras. The multi-camera module 103 communicates with the playback terminal device 102. The four cameras capture images of the target area in real time from different perspectives, and send these images to the playback terminal device 102. The playback terminal device 102 stitches the received images of the target area from different perspectives into a single video frame and controls the video player 105 to play this video frame. Simultaneously, the playback terminal device 102 uploads this video frame to server 101, and server 101 adds this video frame to the previously stored video stream of the target area. The video stream includes... Multiple preset cameras capture images from different angles simultaneously. The newly acquired video frame is analyzed to determine if a target event has occurred. If a target event has occurred, a backtracking frame sequence corresponding to the target event is determined from the multiple angles corresponding to the video stream of the target area. The backtracking frame sequence includes the video frame corresponding to the target event and multiple video frames before the target event occurred. The target angle related to the target event is determined from each video frame in the backtracking frame sequence. Based on the target angle of each video frame, the angle switching path corresponding to the target event is determined. Then, the server 101 controls the video player 105 of the playback terminal device 102 to sequentially play the target images corresponding to the multiple target angles in the angle switching path.

[0027] In practical applications, the playback terminal device includes, but is not limited to, mobile phones, tablets, or VR (Virtual Reality) glasses. Server 101 can be replaced by an edge computing device. Similar to server 101, the edge computing device is also responsible for receiving, splicing, and processing video frames.

[0028] Figure 2 A flowchart of a video playback method provided in this application embodiment, taking the method applied to server 101 as an example, the video playback method provided in this application includes S201~S204: S201, determine the backtracking frame sequence corresponding to the target event from the video stream of the target area.

[0029] The video stream includes multiple video frames ordered by time, each video frame including footage captured from different angles by multiple preset cameras; the multiple preset cameras are arranged around the target area and capture the target area; the backtracking frame sequence includes the video frame corresponding to the target event, as well as multiple video frames before the target event occurred.

[0030] In practical applications, the target area can be a stadium, examination room, concert hall, or other areas requiring monitoring. Multiple cameras are positioned around the target area, either clockwise or counter-clockwise. The combination of cameras should provide a 360° panoramic view of the target area, and the images captured by the cameras are uploaded in real-time to a communicating playback terminal or server. For example... Figure 3 The image shows 16 cameras, A1 to A16, arranged clockwise around the perimeter of the basketball court. The dotted lines indicate the shooting direction of the cameras. Each camera has a horizontal field of view of 10°. Each camera captures a picture of the basketball court, and the images captured by all the cameras form a 360° panoramic view of the basketball court.

[0031] The playback terminal device or server acquires real-time footage from multiple preset cameras and stitches it together to form the current video frame. For example, footage captured by 36 cameras simultaneously can be sequentially stitched together to form a large 6x6 image, which constitutes one video frame. If the preset cameras communicate with the playback terminal device, the playback terminal device also needs to upload the stitched video frame to the server. The server adds the acquired current video frame to the previously stored video stream of the target area. The video stream includes multiple video frames ordered by time, each containing footage of the target area captured by the preset cameras from different perspectives.

[0032] A target event refers to certain events occurring in a target area, such as a goal or a penalty awarded by the referee. This target event can be a pre-specified event, for example, pre-configured to automatically determine the corresponding backtracking frame sequence from the video stream of the target area when a goal occurs. Alternatively, it can be a user-specified event in real-time. User-specified events include, but are not limited to, events specifying a specific action, a specific person or object, or events specifying the execution of an action by a person or other object. When the user-specified event includes a goal-scoring action, if the user wants to rewatch the goal while watching the video, they can give an instruction to the goal event occurring in the video playback interface. The playback terminal device responds to this instruction by sending a timestamp and location information corresponding to the instruction to the server. Based on the timestamp and location information, the server determines the desired goal from the video frame in the target area's video stream that matches the timestamp information. The process involves several key aspects. First, the goal event is matched with the indicated location information, and the corresponding backtracking frame sequence is determined from the video stream of the target area. Second, when the user-specified event includes a specified event targeting a person or object, the user can give an instruction to a player on the video playback interface. The playback terminal device responds to this instruction by sending a timestamp and indicated location information to the server. Based on the timestamp and indicated location information, the server identifies the player in the video frame matching the timestamp from the video stream of the target area. The player's location information matches the indicated location information, and the event occurring in that video frame is determined as the target event. The corresponding backtracking frame sequence is then determined from the video stream of the target area. The process for determining the corresponding backtracking frame sequence for the target event is also described. The first two types of events are not elaborated upon here. In practical applications, the instruction includes, but is not limited to, voice input of a name, air gestures, mouse clicks, and touchscreen actions.

[0033] The backtracking frame sequence is a part of the video stream. The backtracking frame sequence refers to multiple video frames ordered in chronological order within a preset time period, that is, multiple video frames before the occurrence of the target event and the video frames corresponding to the target event. It can be considered as backtracking N frames from the time t when the target event occurs to obtain multiple video frames at time [tN, t].

[0034] In this embodiment of the application, taking the communication between multiple preset cameras and the server as an example, the multiple preset cameras are arranged around the target area and take pictures of the target area from different perspectives. The images captured by the multiple preset cameras in real time are uploaded to the server. The server stitches the images corresponding to the different perspectives into the current video frame. For example, when there are 16 cameras, the images corresponding to the 16 perspectives can be stitched into a 4x4 video frame in a 4x4 format, and the video frame is added to the video stream of the target area that was previously stored.

[0035] The server identifies the current video frame acquired in real time, determines whether a target event has occurred within that frame, and if so, identifies the video frame as the one corresponding to the target event, records it as 't', and then traces back N preset video frames from the video stream of the target area. Each video frame includes footage captured by multiple preset cameras from different angles at a given moment. In practical applications, the preset number of frames N for tracing back can be set manually according to actual needs or automatically adjusted based on user preferences. For example, when tracing back at a fixed frame rate, if the user manually drags the progress bar multiple times, the required historical tracing duration can be determined based on the user's multiple drags, and the average historical tracing duration during multiple viewings can be used to readjust the preset number of frames N for tracing back.

[0036] The server determines the backtracking frame sequence corresponding to the target event based on the video frame corresponding to the target event and multiple video frames before the target event occurred. At this point, the backtracking frame sequence consists of multiple video frames at time [tN, t]. In other words, the backtracking frame sequence F′={f t ,f t 1,…,f t N}, f t Let be the video frame at time t.

[0037] In some embodiments, determining the backtracking frame sequence corresponding to the target event based on the video frame corresponding to the target event and multiple video frames before the target event means concatenating the video frame corresponding to the target event and multiple video frames before the target event in chronological order from early to late to form the backtracking frame sequence corresponding to the target event.

[0038] In some embodiments, the server is configured with a pre-trained event classification model, which is used to determine the event type of events occurring in video frames. The video playback system automatically identifies whether a target event has occurred in a video frame based on the pre-trained event classification model. This event classification model is trained based on historical video frames and their corresponding event types, and can be updated in real time based on each acquired video frame and the event type of the event identified in that video frame.

[0039] In some embodiments, the video playback system can not only automatically detect target events based on a pre-trained event classification model, but also detect target events based on user behavior information. This user behavior information includes the user's historical preference tags or user indications regarding target objects on the playback interface.

[0040] Taking the detection of target events based on users' historical preference tags as an example, the video playback system pre-analyzes users' historical viewing data and extracts historical preference tags from it, such as a preference for forward players, a liking for singers in the center of the stage, or a liking for rewatching goal actions. When an event corresponding to this historical preference tag occurs in a video frame, such as the appearance of a forward player, a singer in the center of the stage, or a goal being scored, it can be determined that the target event has occurred in that video frame. In practical applications, historical viewing data includes, but is not limited to, the number of times a user clicks on a certain object and the duration a user stays on a certain screen. Furthermore, these historical preference tags can also be customized by the user, meaning that users can define the events they are interested in. These customized events can be events targeting a specific action, events targeting a specific person / object, or events where a specific person / object performs a specific action. The video frame where this event occurs is then considered the video frame where the target event occurred. For example, if the user's customized event is a goal scored by player A, then during the match, when player A scores, the video frame corresponding to the timestamp of player A's goal is determined to have experienced the target event. Taking the determination of a target event based on user instructions on a target object on the playback interface as an example, the user can give instructions on a certain action, person / other object, etc. that occurs on the playback interface. In practical applications, instructions include, but are not limited to, voice input of a name, air gestures, mouse clicks, touch screens, etc. For example, when the number seven player scores a goal on the playback interface, the user clicks on the number seven player or the ball on the playback interface. The terminal responds to the click operation and obtains the timestamp information and click location information corresponding to the click operation. The terminal sends the timestamp information and click location information corresponding to the click operation to the server. The server determines the corresponding video frame based on the timestamp information and determines the number seven player goal event in the video frame related to the click behavior based on the click location information.

[0041] In some embodiments, each camera corresponds to a two-dimensional coordinate. The distance between two viewpoints can be determined based on the two-dimensional coordinates (row, col) of the camera corresponding to a certain viewpoint in two adjacent video frames. This distance can be used to define the smoothness of viewpoint switching in the future. By judging the smoothness, abrupt switching between the viewpoints corresponding to adjacent video frames can be avoided.

[0042] S202, determine the target viewpoint related to the target event from multiple viewpoints corresponding to each video frame in the retrospective frame sequence.

[0043] In this embodiment, the backtracking frame sequence includes video frames corresponding to the target event and multiple video frames prior to the occurrence of the target event. Each video frame includes images captured by multiple cameras from different perspectives at the same time. The video player on the playback terminal device is a specially designed player that displays the image corresponding to one of the perspectives of a video frame. Therefore, it is necessary to determine the target perspective from the multiple perspectives of each video frame in the backtracking frame sequence, and then play the target image corresponding to the target perspective in the video player of the playback terminal device.

[0044] Furthermore, the target image corresponding to the target viewpoint is an event related to the target event. That is, the target event needs to occur within the target image corresponding to the target viewpoint. This ensures that when the video player plays target images corresponding to multiple target viewpoints, the target images are all images related to the target event. Images with low relevance to the target event will not be played. This method can improve the relevance of the played video to the target event, making the video playback smoother for viewing the target event.

[0045] In some embodiments, the target viewpoint is determined from multiple viewpoints by considering the correlation between the frame corresponding to the viewpoint and the target event. For each video frame in the backtracking frame sequence, the correlation between the frame corresponding to each viewpoint and the target event can be determined, and the target viewpoint related to the target event can be determined from the corresponding multiple viewpoints based on the correlation between the frame corresponding to each viewpoint and the target event. Further, the correlation between the frame corresponding to each viewpoint and the target event can be evaluated by whether there is an object of interest corresponding to the target event in the frame corresponding to each viewpoint, whether the object of interest in the corresponding frame is in a preset position, and whether the object of interest is obstructed by interference.

[0046] In some embodiments, the target viewpoint is determined from multiple viewpoints by combining the correlation between the image corresponding to the viewpoint and the target event, as well as the image quality. For each video frame in the backtracking frame sequence, the image quality score of the image corresponding to each viewpoint is determined based on the sharpness of the image itself and the correlation between the corresponding image and the target event. The target viewpoint related to the target event is then determined from the multiple corresponding viewpoints using the image quality score of the image corresponding to each viewpoint.

[0047] In some embodiments, in order to further improve the smoothness of video playback, it is not only necessary to determine the target viewpoint from multiple viewpoints by considering the correlation between the image and the target event corresponding to each viewpoint in a single video frame, but also to consider the continuity between the two viewpoints corresponding to adjacent video frames and the current video frame.

[0048] Specifically, for each video frame in the backtracking frame sequence, a picture quality score is determined for each viewpoint based on the correlation between the corresponding image and the target event. The smoothness of viewpoint switching between this viewpoint and another viewpoint in an adjacent video frame is also determined. Based on the picture quality score for each viewpoint and the smoothness of viewpoint switching between this viewpoint and another viewpoint in an adjacent video frame, a target viewpoint is determined from multiple viewpoints. When the target image corresponding to this target viewpoint is played on the video player, not only is the picture quality of the target image high, but the jarring feeling during viewpoint switching between frames is reduced, resulting in visual continuity and enhancing the viewing immersion.

[0049] The process of determining the target viewpoint from multiple perspectives includes: for each viewpoint, determining whether the image quality score corresponding to the viewpoint is greater than a preset first threshold, and whether the smoothness of the viewpoint switching between the viewpoint and another viewpoint in an adjacent video frame is greater than a preset second threshold. If both are greater, then the viewpoint is determined as the target viewpoint; or, based on the image quality score corresponding to the viewpoint and the smoothness of the viewpoint switching between the viewpoint and another viewpoint in an adjacent video frame, constructing a viewpoint scoring function, obtaining the scoring result for each visual based on the image quality score corresponding to each viewpoint, the smoothness of the viewpoint switching between the viewpoint and another viewpoint in an adjacent video frame, and the visual scoring function, and selecting the optimal target viewpoint from multiple viewpoints based on the scoring results.

[0050] In some embodiments, the target viewpoint is determined from multiple viewpoints by simultaneously considering the correlation between the viewpoint and the target event, the image quality of the corresponding viewpoint, and the continuity between the two viewpoints corresponding to adjacent video frames and the current video frame. The specific implementation is as described in the above embodiments and will not be repeated here.

[0051] In some embodiments, in order to reduce the time required to determine the target viewpoint related to the target event and improve the efficiency of target viewpoint determination, multiple candidate viewpoints can be determined from multiple viewpoints corresponding to each video frame in the backtracking frame sequence based on the image quality score corresponding to each viewpoint. Then, the target viewpoint related to the target event can be determined from the multiple candidate viewpoints. It should be explained that since the operation of determining multiple candidate viewpoints based on the image quality score and then determining the target viewpoint from the multiple candidate viewpoints is simpler than determining the target viewpoint from multiple viewpoints according to complex determination logic, it is not necessary to directly determine the target viewpoint from all viewpoints. By first filtering out some viewpoints with lower image quality scores and then determining the target viewpoint from the remaining multiple viewpoints, the efficiency is greatly improved.

[0052] In some embodiments, if there is no target viewpoint related to the target event in a certain video frame, a transitional frame can be automatically inserted or an interpolation method can be used to determine the target viewpoint of the video frame. For example, if cameras 1-36 are arranged clockwise around the target area, and the target viewpoint of the (t-1)th video frame is viewpoint 2, the target viewpoint of the (t+1)th video frame is viewpoint 4, and there is no target viewpoint related to the target event in the tth video frame, then the frame corresponding to viewpoint 3 between viewpoint 2 and viewpoint 4 can be selected as the transitional frame. As another example, if the target viewpoint of the (t-1)th video frame is viewpoint 2, the target viewpoint of the (t+1)th video frame is viewpoint 6, and there is no target viewpoint related to the target event in the tth video frame, then the frame corresponding to viewpoint 4 between viewpoint 2 and viewpoint 6 can be selected as the transitional frame.

[0053] S203, determine the viewpoint switching path corresponding to the target event based on the target viewpoint of each video frame.

[0054] In this embodiment, based on the target viewpoints of multiple video frames in the backtracking frame sequence, the target viewpoints are arranged in chronological order to obtain the viewpoint switching path corresponding to the target event, ensuring a natural and smooth transition of the screen.

[0055] In some embodiments, to improve the continuity of viewpoint switching paths, the selection of viewpoints between consecutive frames is treated as a holistic path problem. A path planning algorithm is employed to determine the target viewpoint related to the target event from multiple candidate viewpoints in each video frame, and all target viewpoints can be combined into a highly continuous viewpoint switching path. In practical applications, path planning algorithms include, but are not limited to, A*, Dijkstra's algorithm, or greedy strategies. Specifically, a greedy algorithm can be used to select the optimal viewpoint for each video frame frame by frame to generate a viewpoint switching path. Alternatively, a pre-defined graph model can be constructed, and A* (A-star Algorithm, a heuristic search algorithm) or Dijkstra's algorithm can be used to generate viewpoint switching paths. The generated viewpoint switching paths include the optimal viewpoint for each video frame in the backtracking frame sequence.

[0056] In some embodiments, the viewpoint switching path includes the target viewpoint corresponding to each video frame. To obtain a more coherent and natural viewpoint switching path, the viewpoint switching path can meet the following requirements: 1. The degree of change in perspective between two adjacent video frames in the perspective switching path shall not exceed the set threshold.

[0057] 2. Maximize the overall image quality index for all target viewpoints in the viewpoint switching path. The overall image quality index includes the sum of the image quality scores for each target viewpoint.

[0058] S204 controls the video player to sequentially play target images corresponding to multiple target perspectives in the perspective switching path.

[0059] In this embodiment, the video player supports partial image rendering and dynamic perspective switching. The perspective switching path includes multiple target perspectives arranged in chronological order. Therefore, without manual user operation, the server can control the video player to sequentially render target images corresponding to multiple target perspectives on the playback interface according to the perspective switching path. Furthermore, since the target perspective is related to the target event, when playing the target images corresponding to multiple target perspectives in the perspective switching path sequentially, all target images are related to the target event, and images with low relevance to the target event will not be played. At this time, the user sees a smooth perspective switching process related to the target event.

[0060] Furthermore, if the target viewpoint is determined by comprehensively considering the cost of inter-frame viewpoint jumps, overall visual coherence, and viewpoint image quality as in the above embodiments, and the video player sequentially plays the target images corresponding to multiple target viewpoints in the viewpoint switching path, the switching between target images is more coherent and natural, and the overall image quality is also higher.

[0061] In some embodiments, when the target event is related to the target object, the target object can always be in a preset position, such as the center of the screen, by determining the view switching path.

[0062] In some embodiments, after controlling the video player to sequentially play target images corresponding to multiple target perspectives in the perspective switching path, this application also supports further adjustment of the target perspective using gyroscope, gesture control, screen swiping, etc., to improve the user experience.

[0063] In some embodiments, after determining the perspective switching path corresponding to the target event, a message is pushed to the terminal asking the user whether they want to review the target event. When the user clicks the review control and confirms that they need to review the target event, the video player is controlled to play the target images corresponding to multiple target perspectives in the perspective switching path in sequence.

[0064] In some embodiments, this application also provides a graphical interface on the playback interface to display some basic information about the target event, such as the state information of objects related to the target event (e.g., speed, action, etc.) and the duration of the target event.

[0065] In the above scheme, the backtracking frame sequence corresponding to the target event is determined from the video stream of the target area. The video stream includes multiple video frames ordered by time, and each video frame includes images captured from different perspectives by multiple preset cameras. The multiple preset cameras are arranged around the target area and capture images of the target area. The backtracking frame sequence includes the video frame corresponding to the target event, as well as multiple video frames before the target event occurs. The target perspective related to the target event is determined from the multiple perspectives corresponding to each video frame in the backtracking frame sequence. Based on the target perspective of each video frame, the perspective switching path corresponding to the target event is determined. The video player is controlled to sequentially play the target images corresponding to multiple target perspectives in the perspective switching path. The entire video playback process does not require manual control of perspective changes by the user. When a target event occurs, the backtracking frame sequence corresponding to the target event is automatically determined, and the target perspective corresponding to the target event in each video frame is selected from the backtracking frame sequence. Then, based on the target perspective corresponding to each video frame, a perspective switching path with high coherence for the target event is determined. This solves the problems of multi-view video playback systems that rely on manual intervention in perspective selection and cannot achieve smooth playback, thus improving the smoothness of video playback.

[0066] In other embodiments of this application, determining the target viewpoint related to the target event from multiple viewpoints corresponding to each video frame in the retrospective frame sequence includes: For each video frame in the backtracking frame sequence, the quality of the images from different perspectives in the video frame is evaluated to obtain the image quality score corresponding to each perspective; based on the image quality score corresponding to each perspective, multiple candidate perspectives are determined from multiple perspectives; and the target perspective related to the target event is determined from the multiple candidate perspectives corresponding to each video frame.

[0067] In this embodiment, for each video frame in the backtracking frame sequence, multiple candidate viewpoints are determined from multiple viewpoints based on the image quality score corresponding to each viewpoint. Then, a target viewpoint related to the target event is determined from the multiple candidate viewpoints corresponding to the video frame. Specifically, for each video frame in the backtracking frame sequence, the image quality of different viewpoints in the video frame is evaluated to obtain an image quality score corresponding to each viewpoint. Based on the image quality score corresponding to each viewpoint, multiple candidate viewpoints are determined from multiple viewpoints. Then, a target viewpoint related to the target event is determined from the multiple candidate viewpoints corresponding to the video frame. In some embodiments, the image quality score of each viewpoint includes the result obtained after evaluating the image relevance of that viewpoint.

[0068] In some embodiments, quality assessment of images from different perspectives in a video frame includes image correlation assessment for each perspective.

[0069] The image relevance assessment includes one or more of the following: target existence assessment, target location rationality assessment, and target motion information assessment. Target Existence Assessment: Detects whether the target event's corresponding object of interest exists in the corresponding screen. For example, when the target event is a goal event, it detects whether the player and ball corresponding to the goal event exist in the corresponding screen; or when the target event is a player going onto the field, it detects whether the player going onto the field exists in the corresponding screen.

[0070] Target location rationality assessment: Determine whether the object of interest in the corresponding image is in the preset position, and whether the object of interest is obscured by interference.

[0071] Target motion information assessment: Focus on the motion trend of the object.

[0072] By performing one or more of the following assessments on the image from each viewpoint: target existence assessment, target location rationality assessment, and target motion information assessment, an image quality score is obtained for each viewpoint.

[0073] Furthermore, quality assessment of images from different perspectives within a video frame can also include the following assessment methods: Image sharpness assessment: Determine whether the corresponding image is blurry, jittery, overexposed, etc.

[0074] Image stability assessment: Between consecutive frames with the same viewpoint, whether the position, scale, and tilt angle of the subject, background, target, and viewpoint of the corresponding image remain consistent, and whether there is any unnecessary jitter, drift, shaking, or jump.

[0075] Image noise assessment: whether there are irrelevant pixel disturbances in the corresponding image.

[0076] Image distortion assessment: This assesses whether the image is distorted and the degree of distortion.

[0077] By evaluating one or more of the following criteria for each viewpoint: image stability, image noise, image distortion, image sharpness, target presence, and target location rationality, an image quality score is obtained for each viewpoint.

[0078] In some embodiments, determining multiple candidate views from multiple views based on the image quality score corresponding to each viewpoint means determining multiple candidate views whose image quality scores exceed a preset score threshold from multiple views; or selecting a preset number of candidate views from multiple views according to their image quality scores from high to low. Taking selecting a preset number of candidate views from multiple views according to their image quality scores from high to low as an example, the candidate view set composed of multiple candidate views is: f i ∈F′,C i =TopK(V(f) i ),S), where F′ is the backtracking frame sequence, C i ′ represents the candidate viewpoint, f i For the i-th video frame in the backtracking frame, V(f) i ) represents multiple viewpoints of the i-th video frame, K is the preset number of selected candidate viewpoints, and S is the image quality score.

[0079] In some embodiments, the target viewpoint is determined from multiple candidate viewpoints by considering the correlation between the image corresponding to the viewpoint and the target event. Specifically, for each video frame in the backtracking frame sequence, the target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to the video frame. This can be achieved by determining the correlation between the image corresponding to each candidate viewpoint and the target event. That is, the correlation corresponding to each candidate viewpoint is used as an image quality score, and the target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to the video frame based on the image quality score corresponding to each candidate viewpoint. Furthermore, the correlation between the image corresponding to each candidate viewpoint and the target event can be evaluated by methods such as whether the object of interest corresponding to the target event exists in the image corresponding to each candidate viewpoint, whether the object of interest is in a preset position in the corresponding image, and whether the object of interest is obstructed by interference.

[0080] In some embodiments, the target viewpoint is determined from multiple candidate viewpoints by considering the correlation between the viewpoint and the target event and the image sharpness. For each video frame in the backtracking frame sequence, the image quality score of the image corresponding to each candidate viewpoint is determined based on the image sharpness of the image itself and the correlation between the corresponding image and the target event. The target viewpoint related to the target event is then determined from multiple candidate viewpoints using the image quality score of the image corresponding to each candidate viewpoint.

[0081] In some embodiments, in order to further improve the smoothness of video playback, the determination of the target viewpoint corresponding to each video frame not only needs to consider the correlation between the image corresponding to each candidate viewpoint in a single video frame and the target event, but also needs to consider the continuity between the two viewpoints corresponding to adjacent video frames and the current video frame.

[0082] Specifically, for each video frame in the backtracking frame sequence, the image quality score corresponding to each candidate viewpoint is determined based on the correlation between the image corresponding to each candidate viewpoint and the target event, and the smoothness of viewpoint switching between each candidate viewpoint and another candidate viewpoint in the adjacent video frame is determined. Based on the image quality score corresponding to each candidate viewpoint and the smoothness of viewpoint switching between each candidate viewpoint and another candidate viewpoint in the adjacent video frame, the target viewpoint is determined from multiple viewpoints.

[0083] In some embodiments, the target viewpoint can be determined from multiple candidate viewpoints by simultaneously considering the correlation between the image corresponding to the viewpoint and the target event, the image clarity, and the continuity between the two viewpoints corresponding to adjacent video frames and the current video frame. Specific methods are as described in the above embodiments and will not be repeated here.

[0084] In the above embodiments, for each video frame in the backtracking frame sequence, some viewpoints are filtered based on the image quality score corresponding to each viewpoint. Then, the target viewpoint related to the target event is determined from the remaining multiple candidate viewpoints. Compared to directly determining the target viewpoint related to the target event from multiple viewpoints corresponding to each video frame, the operation complexity is reduced, the operation time is shortened, and the efficiency of determining the target viewpoint is improved. At the same time, the method of determining multiple candidate viewpoints from multiple viewpoints based on the image quality score corresponding to each viewpoint has higher accuracy, thereby improving the accuracy of determining the target viewpoint.

[0085] In other embodiments of this application, the quality of images from different perspectives in a video frame is evaluated to obtain an image quality score corresponding to each perspective, including: For each viewpoint corresponding to a video frame, determine the first, second, and third image quality scores for each viewpoint; based on the first, second, and third image quality scores, determine the image quality score corresponding to each viewpoint.

[0086] The first image quality score represents the clarity of the corresponding image; the second image quality score represents whether there is an object of interest corresponding to the target event in the corresponding image; and the third image quality score represents whether the object of interest in the corresponding image is in a preset position and whether the object of interest is obscured by interference.

[0087] In this embodiment, a quality scoring function is used to determine the image quality score for each viewpoint. This quality scoring function needs to include a relevance scoring function to the target event. In this embodiment, the quality scoring function includes a first image quality score, a second image quality score, and a third image quality score, wherein the second image quality score is the result of the relevance scoring function evaluation, and the third image quality score is the result of the third image quality score evaluation.

[0088] Specifically, for each perspective: The first image quality score is the result of evaluating the sharpness of the image. Specifically, based on the sharpness of the corresponding image, the target motion information is determined to evaluate the image quality score. Sharpness can be determined using methods such as Laplacian variance and Tenengrad gradient. Taking Laplacian variance as an example, the sharper the image, the more drastic the grayscale changes at the edges / details, the greater the fluctuation in the Laplacian operator response value, and the larger the corresponding variance. Conversely, the blurrier the image, the smoother the grayscale changes, the more uniform the Laplacian result, and the smaller the variance. In practical applications, the sharpness of the image is directly proportional to the first image quality score.

[0089] The second-image quality score is the result of evaluating the existence of the object of interest corresponding to the target event in the image. Specifically, the second-image quality score is determined based on whether the object of interest corresponding to the target event exists in the corresponding image. Detecting the existence of the object of interest can be achieved using object detection models, such as various models in the YOLO series or other models with specific object detection functions. In practical applications, the object of interest can appear partially or completely in the image. If the object of interest corresponding to the target event appears completely in the corresponding image, the second-image quality score for that image is full. If only a part of the object of interest corresponding to the target event appears in the corresponding image, the second-image quality score for that image is lower.

[0090] The first-view quality score is obtained by evaluating the reasonableness of the position of the object of interest corresponding to the target event in the image. Specifically, the third-view quality score is determined based on whether the object of interest is in a preset position and whether it is obscured by distractions. Determining whether the object of interest is in a preset position can be done by the normalized distance from the target's center to the preset position in the image. In practical applications, the preset position can be the center of the image, the upper left corner, etc., depending on the user's actual viewing needs. When the preset position is the center of the image, if the object of interest is in the center and is not obscured by distractions, the third-view quality score is higher.

[0091] In some embodiments, the first image quality score, the second image quality score, and the third image quality score are weighted to obtain the image quality score corresponding to each viewpoint. The weight coefficients of the first, second, and third image quality scores can be adjusted according to the needs of the scene.

[0092] In some embodiments, the quality scoring function S(v ij The expressions for ) include: S(v ij )=αQ(v ij )+βO(v ij )+γM(v ij ) Where: ij represents the j-th viewpoint of the i-th video frame; Q(v ij ) represents the first image quality score of the i-th video frame from the j-th viewpoint, and α is the weighting coefficient of the first image quality score; O(v ij ) represents the second image quality score of the i-th video frame from the j-th viewpoint, and β represents the weighting coefficient of the second image quality score; M(v ij ) represents the third-view quality score of the i-th video frame from the j-th perspective, and γ is the weight coefficient of the third-view quality score; α+β+γ=1, and the weight coefficient can be adjusted according to the user's actual needs. For example, when the user pays more attention to the clarity of the picture, α>β and α>γ are configured. Or, when the user pays more attention to whether there is an object of interest in the picture, β>α and β>γ are configured.

[0093] Specifically, when the user has no specific needs, the default settings α=β=γ=1 / 3 can be used. In this case, the image quality score for each viewpoint is determined based on the average of the first, second, and third image quality scores.

[0094] In the above embodiments, the quality scoring function integrates image sharpness indicators and image relevance indicators (semantic understanding indicators such as target presence, centering, and occlusion). It not only evaluates the image sharpness corresponding to each viewpoint, but also combines the presence of the target event corresponding to the target object in the corresponding image, whether the target object is in a preset position, and whether the target object is occluded by interference objects to make a comprehensive score, thereby improving the accuracy of image quality scoring and enabling the accurate identification of multiple candidate viewpoints from multiple viewpoints.

[0095] In other embodiments of this application, determining the target viewpoint related to the target event from multiple candidate viewpoints corresponding to each video frame includes: Each video frame is assigned multiple candidate viewpoints as nodes in a preset graph model. Two adjacent nodes and the distance between them are determined. If the distance satisfies a preset distance constraint, an edge is added between the two adjacent nodes. Based on the preset graph model with added edges, the target viewpoint related to the target event is determined from the multiple candidate viewpoints for each video frame.

[0096] In this context, two adjacent nodes represent two video frames that are ordered to be adjacent in the video stream.

[0097] Among them, the preset graph model refers to a probabilistic graph / dataflow graph model with a pre-fixed graph topology, node definitions, edge connection relationships, and parameter constraints.

[0098] In this embodiment of the application, in order to more accurately determine the target view related to the target event from multiple candidate views corresponding to each video frame, the selection of the target view is treated as a whole path problem, and a preset graph model is used as a basic model of the path planning algorithm of this application. Path planning is performed based on all nodes and all edges in the preset graph model, and the target view related to the target event is determined from multiple candidate views corresponding to each video frame based on the path planning results.

[0099] Specifically, first, an initial preset graph model is obtained. The initial preset graph model does not contain nodes and edges. Multiple candidate viewpoints corresponding to each video frame are used as multiple nodes in the preset graph model. Then, based on all the added nodes, the edges between nodes in the preset graph model are determined.

[0100] It should be noted that not every two nodes can have an edge added. Since each node corresponds to a viewpoint, the edge between nodes represents a jump from one viewpoint to another. The edge between nodes must be nodes corresponding to the viewpoints of two adjacent video frames in the video stream. In other words, it is necessary to determine the nodes corresponding to the viewpoints of two adjacent video frames in the video stream, identify the nodes corresponding to the viewpoints of two adjacent video frames as two adjacent nodes in the preset graph model, and add an edge between the two adjacent nodes.

[0101] Based on the preset graph model with added edges, the target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame.

[0102] Furthermore, to improve the smoothness of perspective switching, in addition to determining two adjacent nodes, the distance between two adjacent nodes can also be determined. It is then determined whether the distance between two adjacent nodes meets the preset distance constraint. If the distance meets the preset distance constraint, it means that the switching cost between the perspectives corresponding to the two adjacent nodes is small, and a perspective switching action can be performed between these two perspectives. An edge is added between the two adjacent nodes. If the distance does not meet the preset distance constraint, it means that the switching cost between the perspectives corresponding to the two adjacent nodes is large, and there is no need to add an edge between the two adjacent nodes. Finally, based on the preset graph model after adding the edge, the target perspective related to the target event is determined from multiple candidate perspectives corresponding to each video frame.

[0103] In this system, each camera number corresponds to a two-dimensional coordinate (r, c), and each camera's viewpoint corresponds to a node. The distance between two adjacent nodes can be defined using Manhattan distance, Euclidean distance, etc. Taking Manhattan distance as an example, the distance D(c) between the j-th viewpoint and the k-th viewpoint is... j ,c k The expression for ) is as follows: D(c j ,c k )=∣r j r k |+|c j c k | In some embodiments, based on the preset graph model after adding edges, a path planning algorithm is used to find the target viewpoint corresponding to each video frame in the preset graph model, and all target viewpoints need to be able to form a viewpoint switching path in chronological order.

[0104] Specifically, based on the preset graph model after adding edges, multiple candidate paths are first determined in the preset graph model after adding edges. Then, the perspective switching path related to the target event is selected from the multiple candidate paths. Each node of the perspective switching path corresponds to a screen whose relevance to the target event exceeds a preset threshold.

[0105] In some embodiments, nodes in the preset graph model correspond to candidate viewpoints (i, c). j ), (i,c j ) represents the candidate viewpoint c in the i-th frame. j An edge represents a viewpoint switch between two adjacent video frames. An edge is formed if the distance between two adjacent nodes satisfies a distance constraint. For example, to switch from the j-th viewpoint of the i-th video frame to the k-th viewpoint of the (i+1)-th video frame, the edge weight is: w((i, c...). j ),(i+1,c k ))=λD(c j ,c k ), where D(c j ,c k ) represents the distance between the j-th viewpoint and the k-th viewpoint, and λ is the weight of the smoothness.

[0106] In some embodiments, to further improve the coherence of viewpoint switching, an optimal viewpoint switching path can be found from the preset graph model after adding edges using a path planning algorithm, such that the viewpoint switching path satisfies the following conditions: the switching of adjacent target viewpoints is as smooth as possible; the images corresponding to each target viewpoint in the viewpoint switching path are images with high image quality scores; and the total image quality score corresponding to all target viewpoints in the viewpoint switching path is maximized.

[0107] In the above embodiments, the selection of the target viewpoint is optimized into a path optimization problem of a graph model. The process of determining the target viewpoint by executing the path planning algorithm through the preset graph model is more accurate. Furthermore, when the distance between two adjacent nodes meets the preset distance constraint, an edge is added between the two adjacent nodes, which can effectively measure the jumpiness of the viewpoint switching and make the viewpoint switching process smooth.

[0108] In other embodiments of this application, based on a preset graph model with added edges, a target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame, including: Based on multiple nodes and edges in the pre-defined graph model after adding edges, multiple candidate paths are determined in the pre-defined graph model after adding edges. For each candidate path, the image quality score corresponding to each node in the candidate path and the distance between all two adjacent nodes in the candidate path are determined. Based on the image quality scores corresponding to each node in the candidate path, the image quality index corresponding to the candidate path is determined, and based on the distance between all two adjacent nodes in the candidate path, the viewpoint switching smoothness corresponding to the candidate path is determined. Based on the image quality index and viewpoint switching smoothness corresponding to each candidate path, the target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame.

[0109] In this embodiment, multiple candidate paths are determined from the pre-defined graph model after adding edges, based on multiple nodes and edges. Then, a path planning algorithm is used to find the optimal target viewpoint for each frame in the pre-defined graph model, and all target views need to be able to form an optimal viewpoint switching path in chronological order.

[0110] The goal of the path planning algorithm is to maximize the following expression:

[0111] in, This refers to the image quality metrics corresponding to the candidate paths. Rate the image quality of each of the N nodes. Adjust the smoothness of viewpoint switching for candidate paths. Let n be the distance between the (n-1)th node and the nth node. This is a smoothness weighting coefficient, used by users to balance image quality with the smoothness of perspective switching.

[0112] To maximize the above expression, an optimal view switching path P=(p0, p1, ..., p...) can be directly output. N The perspective switching path includes the target perspective related to the target event for each video frame.

[0113] In other words, the image quality index of the candidate path is obtained by summing the image quality scores of all nodes in the candidate path, and the viewpoint switching smoothness of the candidate path is determined by summing the distances between all adjacent nodes in the candidate path. Then, the path score of each candidate path is obtained by weighting the image quality index of each candidate path and the viewpoint switching smoothness. Based on the path score of each candidate path, the optimal path is determined, and the viewpoints of multiple nodes in the optimal path are determined as the target viewpoints of each video frame.

[0114] In some embodiments, to ensure natural perspective transitions and a sense of camera movement, the following constraints are set: 1. Candidate viewpoint limitation: For video frame f t n It is necessary to select the target viewpoint p from multiple TopK candidate views that meet the image quality score. n , where p n ∈TopK(V(f t n ),S), where S is the image quality score corresponding to the candidate viewpoint.

[0115] 2. Spatial Continuity Constraint: The cost of switching between the viewpoints of two adjacent video frames cannot exceed a threshold, which is the viewpoint p corresponding to the (n-1)th node in the path. n-1 The view p corresponding to the nth node n The distance between D(p) n 1,p n (Not exceeding the preset distance threshold d) max : D(p n 1,p n )≤d max In the above embodiments, by considering the picture quality index and the smoothness of the viewpoint switching corresponding to the candidate path, the target viewpoint related to the target event is determined from two dimensions: picture quality and viewpoint switching smoothness. This makes the determination process of the target viewpoint more accurate. Consequently, when the video player is controlled to play the target images corresponding to multiple target viewpoints in the viewpoint switching path in sequence, the video playback process is more natural and coherent, and the picture quality is high.

[0116] In other embodiments of this application, the target event is an event identified using a pre-trained event classification model; wherein, the pre-trained event classification model is used to determine the event type of an event occurring in a video frame; The backtracking determination frame sequence from the video stream of the target region to identify the target event includes: For each video frame in the video stream of the target region, feature extraction is performed on the video frame to obtain video frame features; based on the video frame features, the type of event occurring in the video frame is determined using a pre-trained event classification model; if the event type is the target event, the corresponding backtracking frame sequence is determined based on the video frame and the preset number of backtracking frames.

[0117] In this embodiment of the application, the video playback system is configured with a pre-trained event classification model. The video playback system automatically identifies the event type occurring in the video frame based on the pre-trained event classification model, and determines whether a target event has occurred in the video frame based on the event type. If the event type occurring in the video frame is a target event, the backtracking frame sequence corresponding to the target event is determined based on the video frame and a preset number of backtracking frames.

[0118] The pre-trained event classification model includes a feature extraction module and an event classification head. For each video frame in the video stream of the target region, each video frame is input into the feature extraction module, which extracts features from the video frame to obtain video frame features. The video frame features are then input into the event classification head, which classifies the video frame features to determine the type of event occurring in the video frame.

[0119] In some embodiments, the feature extraction module is generally configured as a lightweight deep neural network model, such as the YOLOv8 model, the SlowFast model, or the TimeSformer model. These models each have their own focus. When the target event is the appearance of a specified object in a video frame, the YOLOv8 model can be used for target detection. When the target event is the occurrence of a specified action in a video frame, the SlowFast model or the TimeSformer model can be used for action recognition.

[0120] Furthermore, YOLOv8 can detect key targets such as people and balls in real time and accurately determine whether a specified object appears in a video frame; the SlowFast network captures short-term appearance features and long-term motion dynamics through a dual-path structure, achieving efficient and accurate recognition of complex actions (such as dunking, shooting, jumping, etc.).

[0121] In some embodiments, the extracted video frame features are input into an event classification head. The softmax or sigmoid function in the event classification head determines whether a target event, such as a goal, score, or actor's appearance, has occurred in the video frame. The softmax or sigmoid function converts the model output into the probability of an event of a certain category. When the model confidence score output by the event classification head exceeds a preset threshold, such as when the confidence score corresponding to the target event output by the event classification head exceeds 0.85, it is determined that the target event has occurred in the video frame. The event type, the timestamp of the video frame corresponding to the target event, and the main objects involved (such as player representation, role name, etc.) are output for subsequent viewpoint filtering.

[0122] Meanwhile, the event classification head can also combine sliding window or temporal action localization mechanism (such as BMN (Boundary-Matching Network) or ActionFormer) to accurately locate the start and end frames of the target event. For example, it can locate the time period between the time the target event is about to occur and the time of occurrence, and determine the backtracking frame sequence of the target event from the video stream of the target area based on this time period.

[0123] In some embodiments, before determining the backtracking frame sequence corresponding to the target event from the video stream of the target area, the video stream needs to be preprocessed, that is, the video frames from 36 cameras are aligned by timestamps and frames are extracted at a fixed frame rate (e.g., 30fps) to obtain the preprocessed video stream; wherein, frame extraction refers to selecting a portion of the images from a continuous video stream according to rules and discarding the rest, which is essentially reducing the video frame rate.

[0124] In some embodiments, to meet the real-time requirements of event detection (end-to-end latency ≤200ms), the pre-trained event classification model is compressed using quantization (INT8), pruning, and knowledge distillation techniques, and deployed on edge computing devices or servers with integrated AI acceleration chips, utilizing GPU / TPU to process multiple video streams in parallel.

[0125] In the above embodiments, the pre-trained event classification model automatically identifies events in video frames, accurately determining whether a target event has occurred in the video frame. This eliminates the need for manual user operation, enabling the identification of target events and automatic video playback, thus enhancing the user's immersive experience.

[0126] In some embodiments, the target event is an event determined based on user behavior information; the user behavior information includes the user's historical preference tags or the user's indication information for the target object on the playback interface; The target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame, including: For each video frame, the target viewpoint related to the target event is determined from multiple candidate viewpoints based on user behavior information.

[0127] In the embodiments of this application, the target event is not limited to the specified event identified by the pre-trained event classification model, but can also be an event determined based on user behavior information.

[0128] Here, target events determined based on user behavior information are further divided into two categories: one is target events determined based on the user's historical preference tags, and the other is target events determined based on the user's instructions to the target object on the playback interface.

[0129] When the aforementioned target event is determined to have occurred from the video stream of the target area, the corresponding backtracking frame sequence is determined from the video stream of the target area. For each target video frame in the backtracking frame sequence, the quality of the images from different perspectives in the target video frame is evaluated to obtain the image quality score corresponding to each perspective. Based on the image quality score corresponding to each perspective, multiple candidate perspectives are determined from multiple perspectives. At this time, the candidate perspectives are perspectives filtered by the image quality scores. The image quality scores include a first image quality score, a second image quality score, and a third image quality score. The first image quality score represents the sharpness of the corresponding image. The second image quality score represents whether the object of interest corresponding to the target event exists in the corresponding image. The third image quality score represents whether the object of interest in the corresponding image is in a preset position and whether the object of interest is obscured by interference.

[0130] Furthermore, for each video frame, based on the user behavior information corresponding to the target event, a target viewpoint related to the user behavior information is determined from multiple candidate viewpoints. Specifically, based on the user behavior information corresponding to the target event, the target viewpoint with the highest matching degree with the user behavior information is determined from multiple candidate viewpoints.

[0131] In other words, when user behavior information represents the user's historical preference tags, the target view with the highest match to the user's historical preference tags is determined from multiple candidate viewpoints. Furthermore, the user's historical preference tags include, but are not limited to, one or more of the user's preference tags for a specific action and a specific person / object. For example, if the user's historical preference tag is a preference for scoring a goal, the target viewpoint containing the goal-scoring action is selected from multiple candidate viewpoints. If multiple candidate viewpoints contain the goal-scoring action, the viewpoint where the ball is located in a preset position within the corresponding viewpoint's frame, or the viewpoint where the ball's position is closest to the preset position, is determined as the target viewpoint. If none of the multiple candidate viewpoints contain the goal-scoring action, the viewpoint closest to the viewpoint containing the goal-scoring action is determined as the target viewpoint.

[0132] When user behavior information corresponds to a user's instruction on a target object on the playback interface, the target view with the highest match degree is determined from multiple candidate views. Furthermore, the user's instruction on the target object on the playback interface includes, but is not limited to, one or more of the following: instructions for a specific action, instructions for a person / other object. For example, when the user's instruction is generated by clicking on player number seven on the playback interface, the view containing that player is selected from multiple candidate views. If multiple candidate views contain the player, the player is determined to be in a preset position within the corresponding view's frame, or the view whose position is closest to the preset position is determined as the target view. If none of the multiple candidate views contain the goal action, the view containing the player is determined to be the closest to the preset position as the target view. In practical applications, the instruction information is generated based on the user's instruction action, which includes, but is not limited to, clicking on a target object on the playback interface, voice input of an object to be locked, and gestures indicating a target object on the playback interface.

[0133] In some embodiments, a user's historical preference tags can be determined through the user's historical behavior information. For example, the user's historical preference tags can be analyzed based on behavioral information such as the number of times the user clicked on a certain object, the frequency of clicking on a certain object, the duration of time the user stayed on a certain screen, and historical review information. For instance, if the number of times the user clicked on player number 7 in the past exceeds a preset threshold, then the user's historical preference tag is determined to be a preference for player number 7; if the number of times the user stayed on a screen containing a goal-scoring action exceeds a preset threshold, then the user's historical preference tag is determined to be a preference for watching goal-scoring actions.

[0134] In the above embodiments, by combining user behavior information, a personalized perspective can be achieved. Users can specify the target object they want to focus on during the viewing process, such as a singer or player. The video playback system will prioritize displaying the perspective of the target object, which meets user needs, improves the accuracy of perspective selection, and ensures a natural transition between perspectives.

[0135] In other embodiments of this application, a full-screen player is also provided, comprising a real-time display module, a video analysis module, and a frame assembly module, including, for example: Figure 4 As shown, the video stream can be a 3D full-motion video stream. The server uses a full-motion player to rewind multiple frames corresponding to the target event and generate exciting moments.

[0136] The real-time display module is used to display a specific frame of the latest acquired video on the playback interface of the playback terminal device.

[0137] The video analysis module uses a pre-trained event classification model to perform video analysis on the latest acquired video frames to determine whether a target event has occurred in the video frames. If a target event has occurred in the latest acquired video frames, the module determines the backtracking frame sequence corresponding to the target event from the video stream of the target region, and determines the target viewpoint related to the target event from multiple viewpoints corresponding to each video frame in the backtracking frame sequence. Based on the target viewpoint of each video frame, the module determines the viewpoint switching path corresponding to the target event.

[0138] The framing module is used to combine the target images from multiple target perspectives corresponding to the viewpoint switching path, so as to control the video player of the playback terminal device to play the target images corresponding to multiple target perspectives in the viewpoint switching path in sequence, that is, to control the video player to play the exciting moments of the target time.

[0139] This application addresses the problems in related technologies, such as the reliance on manual intervention in viewpoint selection, lack of inter-frame continuity optimization, inability to achieve natural camera movement effects in multi-view video playback systems, and the limitation of AI highlight recognition systems to single-view analysis and difficulty in dynamically recommending the optimal viewing path. It provides a video playback method by introducing a multi-dimensional picture scoring mechanism and an inter-frame viewpoint path planning algorithm, which automatically recommends the best viewpoint switching path for the target event with visual coherence without requiring manual operation by the user. This achieves the effect of improving the viewing immersion, interactive experience, and system intelligence.

[0140] Taking a user watching a basketball game as an example, when the AI ​​model identifies the target event, the hardware structure and software components utilized in its video playback method are as follows: I. Hardware Structure: 36 cameras: Numbered 0 to 35 clockwise, evenly distributed around the basketball court, each covering a 10° field of view, forming a complete 360° field of view.

[0141] Edge computing devices / servers: responsible for receiving, splicing, and processing video frames.

[0142] Playback terminal devices: mobile phones, tablets, or VR glasses that support multi-view video rendering.

[0143] AI acceleration chip (optional): Used to accelerate the AI ​​model inference process.

[0144] II. Software Implementation: This method aims to identify key moments in videos (such as goals, dunks, and actors' entrances) using AI models, and, combined with an image quality assessment mechanism, dynamically select the best combination of visual perspectives within 10-20 frames before the event. Simultaneously, a path planning algorithm ensures natural and smooth camera movement during perspective transitions between adjacent frames, thereby enhancing user immersion and interactive experience.

[0145] (1) Using AI models to identify exciting events: Preprocessing: From 36 cameras C={c0, c1, ..., c 35 The multiple video streams are aligned to multiple video frames frame_t according to the timestamp, and frames are extracted at a fixed frame rate of 30fps to obtain the preprocessed video stream video_frames. The format of video_frames is [frame_0, frame_1, ..., frame_t].

[0146] AI Model Processing: The AI ​​model includes a lightweight deep neural network model and an event classification head. A lightweight deep neural network model, such as YOLOv8 or SlowFast, is used to extract features from each video frame or video segment. The extracted features are then input into the event classification head, which uses a softmax or sigmoid function to determine whether the current video frame or video segment contains a predefined category of exciting events (such as goals, scores, actor appearances, etc.). This is combined with a sliding window or time-series motion localization mechanism to accurately locate the start and end frames of the exciting events.

[0147] Inference acceleration and edge deployment: To meet real-time requirements (end-to-end latency ≤200ms), the model is compressed using quantization (INT8), pruning and knowledge distillation techniques, and deployed on edge computing devices or servers with integrated AI acceleration chips, utilizing GPU / TPU to process multiple video streams in parallel.

[0148] Output results: When the confidence level of the AI ​​model output exceeds the set threshold (e.g., 0.85), it is determined that a wonderful event has been detected, and the event type, occurrence timestamp (frame number t) and main targets involved (e.g., player ID, role name) of the wonderful event are output for subsequent perspective determination.

[0149] When exciting events such as goals, scores, or actors appearing are detected, the timestamp t of the current video frame is recorded.

[0150] (2) Determine the backtracking frame sequence corresponding to the target event: Starting from the video frame corresponding to timestamp t, trace back N frames, and extract the traceback frame sequence from tN to t; each video frame in the traceback frame sequence contains 36 viewpoints, where the i-th video frame V(f i )={v i0 v i1 , ...,v i35}, v i0 The footage captured by the first camera, v i1 The footage captured by the second camera, ..., v i35This is footage taken by the 36th camera.

[0151] (3) Use the image quality scoring function score_function to evaluate the image quality score of 36 viewpoints in each video frame; for the j-th viewpoint in the i-th video frame, the dimensions of the image quality scoring function score_function include the following three: Image sharpness assessment: Determine if the image is blurry, jittery, overexposed, etc. Target Presence Assessment: Detects whether there are any people or objects of interest in the scene; Target location rationality assessment: Determine whether the person or object to be focused on is centered, whether the person or object to be focused on is obscured, etc. Furthermore, the image quality scoring function `score_function` is: S(v ij )=αQ(v ij )+βO(v ij )+γM(v ij ) Where: ij represents the j-th viewpoint of the i-th video frame; Q(v ij O(v) represents the first image quality score for evaluating the image sharpness of the i-th video frame from the j-th viewpoint, where α is the weighting coefficient of the first image quality score. ij M(v) represents the second image quality score for the target existence assessment at the j-th viewpoint in the i-th video frame, and β is the weighting coefficient of the second image quality score; ij ) represents the third-view quality score for evaluating the reasonableness of the target position in the j-th viewpoint of the i-th video frame, and γ is the weighting coefficient of the third-view quality score; α+β+γ=1.

[0152] The above three evaluation results are weighted to obtain the image quality score S(v) for this viewpoint. ij )∈[0,1], based on the picture quality score of each viewpoint, select multiple candidate viewpoints with the top 5 picture quality scores from the 36 viewpoints of each video frame.

[0153] (4) Path planning: Node determination of the preset graph model: The top 5 candidate viewpoints of each video frame are used as graph nodes of the preset graph model; among the multiple nodes, the nodes corresponding to two adjacent video frames in the video stream are two adjacent nodes.

[0154] Edge determination of the preset graph model: Based on the two-dimensional coordinates (row, col) corresponding to each camera, determine the distance between two adjacent nodes. For example, the distance between the j-th viewpoint and the k-th viewpoint is D(c j ck If the distance between nodes (n-1) ∈ R+ satisfies a preset distance constraint, then an edge is added between any two adjacent nodes; where the preset distance constraint refers to the viewpoint p corresponding to the (n-1)th node in the path. n-1 The view p corresponding to the nth node n The distance between D(p) n 1,p n (Not exceeding the preset distance threshold d) max ;D(p n 1,p n )≤d max .

[0155] Use A* or Dijkstra's path planning algorithm to find an optimal viewpoint switching path P=(p0, p1, ..., p... N This ensures that: the switching between the viewpoints of two adjacent nodes in the path is as smooth as possible (the distance between the viewpoints of two adjacent nodes is no greater than a preset threshold); the viewpoints of all nodes in the path are high-quality images; and the image quality score of all nodes in the path is maximized (i.e., the sum of the viewpoint scores for each frame is the highest).

[0156] In summary, the goal of the path planning algorithm is to select a target viewpoint from the 36 viewpoints in each of the N frames preceding the event frame t, and to construct a continuous viewpoint switching path P from all target viewpoints, thereby optimizing the following two objectives: Objective 1: Maximize overall visual quality:

[0157] in, The target viewpoint of the nth video frame. Corresponding image quality rating; This refers to the image quality metrics for the perspective switching path.

[0158] Objective 2: Maximize the smoothness of perspective switching (i.e., minimize the cost):

[0159] in, From the perspective of the target arrive The distance between them, which is the switching cost; The visual smoothness of the viewpoint switching path.

[0160] These two objectives can be combined into a single optimization problem: Find P = (p0, p1, ..., p...) N )

[0161] in, It is a smoothness weighting coefficient used to balance image quality and transition smoothness. Rate the paths and select the path with the highest rating as the viewpoint switching path P.

[0162] (5) Video player rendering: Based on the best_path of the viewpoint switching path, the video player on the playback terminal device renders the selected viewpoint sequentially; what the user sees is a smooth, natural, and visually high-quality transition process with a sense of camera movement.

[0163] In addition, the video player also supports gyroscope and gesture control to further adjust the viewing angle.

[0164] Taking a user watching a basketball game as an example, when the target event is identified in a personalized manner based on user behavior information, the hardware structure and software components utilized in its video playback method are as follows: I. Hardware Structure: Same as AI model for identifying target events.

[0165] II. Software Implementation: This section aims to provide personalized perspective recommendations based on user behavior information. Users can specify objects of interest (such as a player or singer) during viewing, and the system will prioritize displaying the perspective of that object, using a path planning algorithm to ensure smooth and natural perspective transitions.

[0166] The implementation of this software is similar to that of AI models for identifying target events, but the differences are as follows: (1) User interest modeling (or (2) target of interest locking): Analyze users' historical viewing data to extract preference tags; such as: preference for forward players, liking singers who are center stage, etc.

[0167] (2) Target of interest locking (either of (1) user interest modeling): Users can select a person by clicking on them in the screen, entering their name by voice, or using gestures; this will initiate event recognition for that person.

[0168] (3) Personalized perspective recommendations: In the Top-5 candidate viewpoints of each video frame, the viewpoints corresponding to the object with the preference label in (1) or the locked object in (2) are selected first. If there is no corresponding viewpoint in the current frame, the viewpoint with the closest position to the object with the preference label in (1) or the locked object in (2) is recommended. Combined with the path planning algorithm, the viewpoint switching is ensured to be natural.

[0169] (4) Optimization of perspective switching: Dynamically adjust the perspective switching path to ensure that the subject of interest is always in the center of the frame; Provides a graphical interface to display character status information (such as speed, actions, etc.).

[0170] Based on the above analysis, this application has the following advantages: 1. The multi-dimensional AI scoring and optimization steps based on image clarity assessment, target existence assessment, and target location rationality assessment have been added to the multi-view video playback process. Combined with user interest tags, personalized view recommendations are realized, and a new interactive form of "automatic optimization + dynamic switching" is given, which significantly reduces the user's operation burden and improves the viewing experience.

[0171] 2. The viewpoint selection algorithm adds a continuity constraint in the time dimension, transforming the traditional independent frame viewpoint decision problem into a multi-frame viewpoint path overall optimization problem. It introduces path planning algorithms such as A*, Dijkstra, or greedy strategies, and comprehensively considers spatial adjacency and image quality score, thereby improving the smoothness, naturalness, and overall visual quality of viewpoint switching.

[0172] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0173] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0174] It should be noted that the technical solutions described in the embodiments of the present invention can be combined arbitrarily without conflict.

[0175] In addition, in the embodiments of the present invention, "first," "second," etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0176] The video playback method of this application embodiment can be implemented by a processor in an electronic device, such as a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU), or a field-programmable gate array (FPGA).

[0177] To implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device. Figure 5 This is a schematic diagram of the hardware composition structure of an electronic device provided in an embodiment of this application, such as... Figure 5 As shown, an electronic device is equipped with a video playback system. The electronic device includes: The communication interface 501 enables information exchange with other devices, such as electronic devices. The processor 502 is connected to the communication interface 501 to enable information exchange with other devices and to execute the methods provided by one or more of the above-mentioned technical solutions when running a computer program. The computer program is stored in the memory 503.

[0178] Of course, in practical applications, the various components in an electronic device are coupled together through a bus system 504. It can be understood that the bus system 504 is used to implement communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 5 The general designated all buses as Bus System 504.

[0179] The memory 503 in this embodiment is used to store various types of data to support the operation of the computer device. Examples of such data include any computer program used to operate on the electronic device.

[0180] It is understood that memory can be volatile or non-volatile, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0181] The methods disclosed in the embodiments of this application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory. The processor reads the program from the memory and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0182] Optionally, when the processor executes the program, it implements the corresponding processes implemented by the electronic device in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0183] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a first memory storing a computer program, which can be executed by a processor of a computer device to complete the aforementioned method steps. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0184] In the several embodiments provided in this application, it should be understood that the disclosed apparatus, computer devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0185] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0186] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0187] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0188] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, etc.) to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0189] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor 502 of an electronic device to complete the steps of the video playback method in this application embodiment.

[0190] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0191] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0192] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A video playback method, characterized in that, The method includes: The backtracking frame sequence corresponding to the target event is determined from the video stream of the target area; the video stream includes multiple video frames ordered by time, each video frame including images captured from different angles by multiple preset cameras; the multiple preset cameras are arranged around the target area and capture images of the target area; the backtracking frame sequence includes the video frame corresponding to the target event, and multiple video frames before the target event occurred; Determine the target viewpoint related to the target event from multiple viewpoints corresponding to each video frame in the backtracking frame sequence; Based on the target viewpoint of each video frame, determine the viewpoint switching path corresponding to the target event; The video player is controlled to sequentially play target images corresponding to multiple target perspectives in the perspective switching path.

2. The method according to claim 1, characterized in that, Determining the target viewpoint related to the target event from multiple viewpoints corresponding to each video frame in the backtracking frame sequence includes: For each video frame in the backtracking frame sequence, the quality of the images from different perspectives in the video frame is evaluated to obtain the image quality score corresponding to each perspective. Based on the image quality score corresponding to each of the aforementioned viewpoints, multiple candidate viewpoints are determined from the multiple viewpoints. The target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame.

3. The method according to claim 2, characterized in that, The process of evaluating the quality of images from different perspectives within the video frame to obtain a quality score for each perspective includes: For each viewpoint corresponding to the video frame, a first image quality score, a second image quality score, and a third image quality score are determined for the image corresponding to each viewpoint; wherein, the first image quality score represents the clarity of the corresponding image; the second image quality score represents whether the object of interest corresponding to the target event exists in the corresponding image; and the third image quality score represents whether the object of interest in the corresponding image is in a preset position and whether the object of interest is obscured by interference. Based on the first image quality score, the second image quality score, and the third image quality score, a corresponding image quality score is determined for each viewpoint.

4. The method according to claim 2, characterized in that, The step of determining the target viewpoint related to the target event from multiple candidate viewpoints corresponding to each video frame includes: Each of the multiple candidate viewpoints corresponding to each video frame is respectively determined as nodes of a preset graph model; Determine two adjacent nodes among the plurality of nodes, and the distance between the two adjacent nodes; the two adjacent nodes represent two video frames that are ordered adjacent in the video stream; If the distance satisfies the preset distance constraint, then an edge is added between the two adjacent nodes; Based on the preset graph model with added edges, the target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame.

5. The method according to claim 4, characterized in that, The step of determining the target viewpoint related to the target event from multiple candidate viewpoints corresponding to each video frame based on the preset graph model with added edges includes: Based on multiple nodes and multiple edges in the preset graph model after adding edges, determine multiple candidate paths in the preset graph model after adding edges; For each candidate path, determine the image quality score corresponding to each node in the candidate path and the distance between each pair of adjacent nodes in the candidate path; Based on the image quality scores corresponding to all nodes in the candidate path, the image quality index corresponding to the candidate path is determined, and based on the distance between all two adjacent nodes in the candidate path, the viewpoint switching smoothness corresponding to the candidate path is determined. Based on the image quality index and viewpoint switching smoothness corresponding to each candidate path, the target viewpoint related to the target event is determined from multiple candidate viewpoints corresponding to each video frame.

6. The method according to claim 1, characterized in that, The target event is an event identified using a pre-trained event classification model; wherein the pre-trained event classification model is used to determine the event type of the event occurring in the video frame; Determining the backtracking frame sequence of the target event from the video stream of the target region includes: For each video frame in the video stream of the target region, feature extraction is performed on the video frame to obtain video frame features; Based on the video frame features, the type of event occurring in the video frame is determined using the pre-trained event classification model; If the event type is a target event, then the backtracking frame sequence corresponding to the target event is determined based on the video frame and the preset backtracking frame number.

7. The method according to claim 2, characterized in that, The target event is an event determined based on user behavior information; the user behavior information includes the user's historical preference tags or the user's indication information for the target object on the playback interface; The step of determining the target viewpoint related to the target event from multiple candidate viewpoints corresponding to each video frame includes: For each video frame, a target viewpoint related to the target event is determined from multiple candidate viewpoints based on the user behavior information.

8. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method according to any one of claims 1 to 7.

9. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images