An interactive system for exploring a collection of videos using embedded visualizations

Abstract

The application discloses an interactive system for exploring video set by using embedded visualization, comprising a video view module, a query view module and a data view module; wherein the video view module displays videos and extracted data, provides interactive functions with the videos, and carries out interactive query by specifying attributes; the query view module carries out hierarchical division according to video interactive query, and displays the division results; and the data view module displays detailed information of video data selected from the query view module. Through the interactive system, a user can interactively manipulate video elements, specify query conditions, generate query results, select video data meeting the conditions from the data set, and finally quickly locate a target typical video through a series of embedded visualization interactions designed for different video elements.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data visualization, and particularly relates to an interactive system for exploring a video collection using embedded visualizations. BACKGROUND

[0002] In racket sports (such as badminton, table tennis), experts usually collect a large number of competition videos, and cut them into different levels of segments (sets, rounds and rackets) for competition analysis. Experts manually mark data in the video, conduct statistical analysis, and convey the insights obtained to the coaches and players.

[0003] Research on video analysis usually focuses on designing and implementing an interactive system with customized views to help users better analyze video content. Traditional video analysis is done by watching videos, which is a very time-consuming process. Video visualization technology is introduced into video analysis systems to help users effectively analyze video content without watching the entire video. Some research attempts to reduce the time spent watching videos by reorganizing visual content. With the increase in the number of videos, it becomes difficult to visualize the original visual content. For this reason, researchers treat the original video as a data source and use state-of-the-art computer vision techniques to extract multi-dimensional data from the video. Researchers have designed different visualization analysis systems to analyze the extracted data.

[0004] The gap between abstract data and video content prompts researchers to bridge the gap through interaction. For example, Sacha et al. designed a method to help athletes query football videos by drawing the athletes' movements. Closely related to this work, Saquib et al. designed the graphiti system, which allows users to draw sketches directly on videos to select and link elements to create an abstract graph structure. However, the graphiti system is designed specifically for graph analysis. Analyzing video content is not its main focus, and this visualization method of data extracted from a video collection for easy analysis of visual content is lagging behind.

[0005] Video augmentation focuses more on facilitating the process of creating data-augmented videos. Video augmentation is a technique that enhances videos through embedded visualizations. Augmented videos are more engaging, so video augmentation techniques have been widely used. Sports is also a popular scenario for using video augmentation. In video augmentation, visualization has been widely used to explain complex concepts and patterns of movement. Television companies such as ESPN embed visualization elements in sports competition videos to explain certain tactics to the audience. In videos, key players and movements are often highlighted in different visual forms to help the audience better understand the competition background.

[0006] In addition, researchers have proposed various methods for creating augmented videos. Fischer et al. conducted a survey and classified the methods for creating augmented videos into two parts: commercial solutions and academic solutions. Among the commercial solutions, the BBC's Augmented video player allows users to add extra layers to a football video. The Piero system and Viz Libero software are widely used, allowing users to create tactical visualizations (e.g., formations and covered areas) in videos based on semi-automatically extracted data. Court vision allows users to watch a basketball game from the perspective of a spectator and a coach, thereby improving the viewing experience.

[0007] In academic solutions, Stein et al. proposed a comprehensive workflow for extracting sports data from videos and integrating the extracted data with visualizations into videos. This can help users perceive visual content and abstract data visualizations. VisCommentator designed by Chen et al. provides a more efficient and flexible way to create augmented sports videos, allowing users to easily integrate different types of visualizations and video effects into videos through data-level operations. Chen et al. further extended VisCommentator through a natural language interface to simplify the process of creating augmented videos. iball and Omnioculars further applied data augmentation techniques to enhance the viewing experience of basketball from the perspective of a spectator.

[0008] The above methods can create attractive and intuitive augmented videos. However, when faced with a collection of videos, even if each video can be augmented one by one, the entire process is still time-consuming. SUMMARY

[0009] In view of the above, the purpose of the present application is to provide an interactive system for exploring a video collection using embedded visualizations. Through this interactive system, users can interactively manipulate video elements, specify query conditions, generate query results, select video data that meets the conditions from the data set, and finally quickly locate the target typical video through a series of embedded visualization interactions designed for different video elements.

[0010] To achieve the above-mentioned purpose of the application, the present application provides an interactive system for exploring a video collection using embedded visualizations, comprising: a video view module, a query view module, and a data view module.

[0011] The video view module displays videos and extracted data, provides interactive functions with the videos, and performs interactive queries by specifying attributes.

[0012] The query view module divides the video interactive query into layers and displays the results of the division;

[0013] The data view module displays detailed information of the video data selected from the query view module.

[0014] Preferably, the video view module has basic interactive functions, and the video control bar is represented by small rectangles of video frames, and the user can play, pause or control the video progress through the video control bar, and the video view module also has speed control buttons to allow the user to change the video playing speed.

[0015] Preferably, the video view module also provides a stroke information panel, specifically a panel list under the video control bar, to display the specific information of each stroke, and a bar of different lengths is used to encode the time range of each stroke, and each panel corresponding to each stroke displays multi-dimensional stroke information, and when the video is played, the stroke panel corresponding to the current frame will be expanded and highlighted, and the user can click the corresponding panel to go to the corresponding video frame.

[0016] Preferably, the video view module also has a video interactive function of viewing data, and the user selects statistical information from the interactive menu, right-clicks the video to display the menu, and the statistical information will be displayed in the interactive video.

[0017] Preferably, the video view module also provides a video interactive function of specifying queries, specifically including region interaction, trajectory interaction, speed interaction and technology interaction.

[0018] For region interaction, the user uses a lasso to select a specified region, and creates a closed polygonal region by specifying the region of the ball landing point or the region of the player's movement, and this interaction can be used to define the falling position of the ball and the position of the player, and the position of the region selected by the mouse will be mapped to the position of the billiard table using the previously extracted perspective matrix;

[0019] For trajectory interaction, the user draws a continuous curve by left-clicking and dragging to represent the trajectory of the ball or the trajectory of the player, and when the video is played, the trajectory will be automatically displayed on the video, and when the user selects a stroke to draw the trajectory of the ball, the video will automatically switch to the serving frame and use a flashing semi-transparent circle to represent the hitting position;

[0020] For speed interaction, the user left-clicks all the time to represent the continuously increasing height of the bar of speed-related values;

[0021] For technology interaction, a circular interface is provided for the user to select the hitting technology, and the technology is divided into different categories, and the user first selects the technology category from the large circular ring, and then selects the specific technology from the small circular ring.

[0022] Preferably, the video view module also has a record function of the interaction, after the user interacts with the video and specifies the query condition, the interaction will be recorded under the shot information panel below the video, the user selects the shots on the panel, and after specifying the query condition, clicks the query button, the query can be performed in the database, when the query result is obtained, the video will be automatically slowed down in the relevant time period meeting the query condition when the user views the query result video, and the user is also allowed to switch between the original video and the queried video.

[0023] Preferably, in the query view module, the video interaction query result is hierarchically divided in a tree structure, specifically: first, the query is sorted according to the selection order, and the video set is divided into two groups of matching or not matching Q1 according to whether the query condition of the first query Q1 is met. The two groups will be presented as the second layer in the tree structure, then each group is iteratively divided according to the next query, and finally the tree structure is obtained, wherein the top layer of the tree structure represents the original video set, and the relevance score of each node representing a group of video segments in the query tree is further calculated, the relevance = the number of matched shot attributes / the total number of shot attributes of the query, and the relevance is encoded with a gradient color, in each matching node, a dark color bar is set, and the winning rate is encoded with the height of the bar, the higher the bar, the higher the winning rate.

[0024] Preferably, in the query view module, clicking a node can view the data distribution of different attributes of the corresponding video set, and the detailed information of the corresponding group of videos will also be displayed in the data view, the user scrolls to browse the attribute distribution panel, and in addition, the data of other panels is filtered by clicking the attributes in the attribute distribution panel.

[0025] Preferably, in the data view module, when the user clicks a node in the query tree, the basic information of each round corresponding to the node is displayed at the top of the round information, and the specific shot details are presented in the form of a table, wherein the basic information includes the name of the match, the position in the match, the server and the winner.

[0026] A filter selection box is arranged at the upper right corner of the view, and the user can select the filter condition and attribute value to filter the query result.

[0027] Compared with the prior art, the application has at least the following beneficial effects:

[0028] Different from common query systems, the application utilizes embedded visual interaction to help users define query conditions which are difficult to express in language conveniently and quickly, supports multi-fragment joint query, presents video query results through query view and data view, and enables users to quickly locate target videos. The application can effectively help sports experts quickly find target videos which can explain data analysis results to coaches and athletes. Through customization adjustment, the system can also be popularized to other video fields with similar data structures. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0030] Figure 1 is a structural schematic diagram of an interactive system for exploring video sets using embedded visualization provided by the embodiments;

[0031] Figure 2 is an interface diagram of the interactive system for exploring video sets using embedded visualization provided by the embodiments;

[0032] Figure 3 is a video interactive interface diagram for specifying queries provided by the embodiments;

[0033] Figure 4 is a tree structure schematic diagram in a query view module provided by the embodiments;

[0034] Figure 5 is a specific implementation scenario 1 schematic diagram provided by the embodiments;

[0035] Figure 6 is a specific implementation scenario 2 schematic diagram provided by the embodiments. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the protection scope of the present application.

[0037] This embodiment provides an interactive system for exploring a collection of videos using embedded visualization. The system's visual interface is implemented using Vue.js and D3.js, allowing users to interactively manipulate the videos. Users can use different interactions to manipulate the attributes of different elements, such as the player's technique, the ball's trajectory, and the area where the ball lands, to retrieve historical videos that match query criteria and see how other athletes handled the situation. Users can further compare the differences between the videos across multiple extracted data dimensions.

[0038] like Figure 1 and Figure 2 As shown, the embodiment provides an interactive system for exploring a collection of videos using embedded visualization, including a video view module ( Figure 2 (A) ), Query View Module ( Figure 2 (B) and Data View Module Figure 2 (C)), where video view ( Figure 1 The middle (A) view displays the video and its extracted data. Users can interact with the video, specifying values ​​for certain attributes (e.g., hitting technique or the ball's landing area) to perform interactive queries. The video collection is hierarchically segmented based on video interactions, and the results are displayed in the query view. Figure 2 (B) Data View Figure 2 The extracted video data (selected from the query view) is further displayed in a table format in the middle (C). Users can use the slider in the upper right corner of the system ( Figure 2 The strictness of video queries is controlled by (D). Each model is explained in detail below.

[0039] In this embodiment, the video view module is a major component of the system, including basic interactive functions, a photo information panel function, a video interactive function for viewing data, a video interactive function for specifying queries, and an interactive recording function.

[0040] For basic interactive functions, the video control bar ( Figure 2 The video frames (A2) are represented by small rectangles. Users can play, pause, and control the video progress using the video control bar. Because athletes in sports videos move very quickly, a speed control button is included in the system. Figure 2 The A2 section allows users to adjust the video playback speed for a better viewing experience.

[0041] For the shot information panel function, a panel list is set up below the control bar to display the specific information for each shot. Specifically, it uses bars of varying lengths (…). Figure 2The time range of each shot is encoded in the (A3) section. Each panel displays multi-dimensional shot information, such as shot technique and hitting position. When playing video, the pen panel corresponding to the current frame will be expanded and highlighted. Figure 2 (A3). Users can also click on the corresponding panel to go to the corresponding video frame.

[0042] For the video interaction function to view data, users can select statistical information from the interaction menu. Figure 2 In the middle (A5), right-click the video to display this menu, and information will be displayed in the interactive video ( Figure 2 (A4)). By selecting tactical interactions, users can further click on the highlighted athletes in sequence, creating connections between the selected athletes. Figure 3 (A9)). User-created connections can move with the athlete while the video is playing.

[0043] For video interaction functionality involving specified queries, a series of interactions based on common sketching operations were designed to manipulate motion data. After importing instance segmentation information, users can interact with instances in the view. For example, when the mouse hovers over an athlete, a mask is placed over the video to highlight the instance object. Four types of interactions—region interaction, trajectory interaction, speed interaction, and technical interaction—are defined to support flexible user interaction.

[0044] Specifically, for area interaction, users can use a lasso to flexibly specify an area, creating a closed polygonal area by clicking on the table (specifying the area where the ball will land) or the court (specifying the area where the player moves). This interaction can be used to define the ball's landing position (…). Figure 3 (A1) and player position ( Figure 3 (A2)). The area selected by the mouse will be mapped to the position of the pool table using a pre-extracted perspective matrix.

[0045] Specifically, for trajectory interaction, users can left-click and drag to draw a continuous curve to represent the ball's trajectory. Figure 3 (A3) or athletes Figure 3 (A4)). When the video plays, the trajectory will be automatically displayed on the video. When the user selects "Ball-Trajectory" in the interactive menu and chooses a shot to draw the ball's trajectory, the video will automatically switch to the serve frame, and the hitting position will be indicated by a flashing semi-transparent circle. Figure 3 (A5)

[0046] For speed-related interactions, users can continuously click the left mouse button to increase the height of a bar representing speed-related values ​​(such as velocity and acceleration) until the mouse button is released. Figure 3(A6)

[0047] Regarding technical interaction, referring to EventAnchor, the embodiment provides a circular interface ( Figure 2 The middle (A7) section allows users to select their hitting technique. Because the numerous techniques in table tennis cannot be displayed simultaneously, these techniques are categorized to simplify selection. Users can first select the technique category from the larger rings, and then choose the specific technique from the smaller rings that appear.

[0048] The user's current interaction will be displayed in the information bar at the top right corner of the video. Figure 2 (A6)). Users can choose to redraw, confirm, or delete the interaction.

[0049] Regarding the interaction recording function, after a user interacts with the video and specifies search criteria, the interaction will be recorded below the video's capture information panel. Users can select multiple captures on the panel. After specifying search criteria, clicking the search button... Figure 4 If you navigate to the right side of A2, you can perform a query in the database. When viewing the video of the query results, the video will automatically slow down for the relevant time period that matches the query criteria. Users can switch between the original video and the video being queried.

[0050] In this embodiment, the query view module mainly divides the video interactive query into layers and displays the division results. After clicking the query button, the query results will be displayed in the query view in the form of a tree structure.

[0051] like Figure 4 As shown, the system searches the video collection to find videos that match all queries. The video collection is hierarchically partitioned based on the queries to simplify the bottom-up search. Specifically, the queries are first sorted according to the order in which they were viewed. For example... Figure 4 In step (B), the video set is divided into two groups based on whether it matches the query condition Q1: videos that match or do not match Q1. These two groups will be represented as the second level in the tree structure. Then, each group is iteratively divided according to the next query, resulting in a four-level tree structure containing three query results, where the top level represents the original video set. Furthermore, a relevance score is calculated for each node (a group of video clips) in the query tree: relevance = (number of matched shot attributes) / (total number of stroke attributes in the query). Relevance is represented by a gradient color (...). Figure 2 The code uses the bottom (B) of the chart for encoding. In each matching node, a dark-colored bar is set, and the height of the bar encodes the winning percentage. The higher the bar, the higher the winning percentage.

[0052] Clicking on a node will display the data distribution of different attributes for that video set. Detailed information about that video set will also be displayed in the data view. Users can scroll through the attribute distribution panel. Additionally, they can filter data in other panels by clicking on attributes within the distribution panel.

[0053] In this embodiment, the data view module displays detailed information about the video data selected from the query view module. When the user clicks a node in the query tree, the detailed information is displayed in the data view (…). Figure 2 In the middle (C)). Basic information for each round (including the name of the match, position in the match, server and winner) is displayed at the top of the round information ( Figure 2 (C1) , and the specific details of the shots are presented in a table format. To help users quickly locate historical data of interest, a filter selection box is also set in the upper right corner of the view. Figure 5 (C2)). Users can select filter conditions and attribute values ​​to filter query results.

[0054] In sports videos, the positional information of some entities may be lost or inaccurately located due to camera movement and zoom. Compared to sports like football and basketball, each set of table tennis videos typically uses a fixed camera angle, and there is almost no zooming of the camera lens in the videos. Therefore, it was decided to use table tennis videos to implement a proof-of-concept system. The following details the interactive system for exploring video sets using embedded visualization, presented above, in two use cases. One scenario uses a video of a round between athlete 1 and athlete 2 (athlete 2 wins this round) as the target for exploration. The experts' goal is to help athlete 1 find a way to handle this situation. These two scenarios demonstrate how two different experts (referred to as Expert A and Expert B) use the system to explore possible improvements for athlete 1. A video dataset containing 264 video rounds is provided as a database for both scenarios.

[0055] Application Scenario 1

[0056] Expert A explored whether Player 1 could make changes in the first three shots to reverse the outcome of this rally. He stated that the first three shots (serve, receive, and serve-and-attack) are crucial to a rally. Players who seize the initiative in the first three shots are often more likely to capitalize on their strengths and control the rhythm of the match later. Expert A found that in the rally, Player 2 chose to serve a mid-to-backspin ball with reverse spin (…). Figure 5 (A1)). Player 1 responds to the serve with a chop shot, returning a forehand half-length underspin ball. Figure 5In the middle (A2), the ball is placed on the left corner of the table. After this, player 2 starts to attack. Expert A considers whether player 1 can respond with a short ball, and the ball is placed near the net. Since the opponent player has height and arm length limitations, she can respond with a short ball as well, so player 1 has a higher probability to take the initiative. Therefore, expert A clicks the second bat (i.e., player 1’s receiving bat), right-clicks the action option, and changes the technical attribute to “short push” ( Figure 5 In the middle (B1); in addition, he right-clicks the ball position option and draws a quadrilateral near the net by continuous clicks of the mouse ( Figure 5 In the middle (B2). This indicates that player 1 will hit the table tennis ball to this area by short push. Expert A selects the first bat (i.e., player 2’s serving bat) and the second bat (player 1’s receiving bat after adding the movement attribute through the query interaction), and clicks the query button.

[0057] In the query view, expert A clicks to select the third row and the first block (i.e., the query result that completely matches the selected bat, as shown in Figure 5 In the middle (C2), a series of rounds appear in the lower round details view. To quickly view the successful cases, expert A uses the filtering function ( Figure 5 In the middle (C3). The filtered results show that only 2 video clips respond to the serve with the specified technique and win the game.

[0058] Expert A selects the 11th round of the 5th game of the 2019 Chengdu World Cup player 3 vs. player 4 match for viewing. From the video, expert A finds that player 4’s serve round has the same technique as the query condition, and the ball landing point is similar to player 2’s serve. Player 3 responds with a short push, and the ball is near the opponent’s net. Player 4 also responds with a short push, but this time the ball landing point is far from player 3’s net. This creates an opportunity for player 3 to attack. Player 3 hits an arc loop to start the attack, and player 4 cannot respond in time, hitting the ball into the net ( Figure 6 In the middle (C4), so player 3 wins this round.

[0059] However, expert A finds that the completely matched query result corresponds to a tree node with a low dark block height, about one-fifth of the tree node height, indicating that this tactic has a low scoring rate. After reviewing other video rounds that do not win using this tactic, expert E indicates that this tactic can only have a chance if the opponent player also responds with a short push. If the opponent player switches to another technique or quickly moves to defend after a short push, this strategy will be neutralized.

[0060] Application Scenario 2

[0061] From a professional athlete's perspective, mis-hitting shots are crucial for learning from mistakes. Because Athlete 1 chose the wrong landing point, the ping-pong ball hit the net and went out of bounds. Therefore, Expert B chose to investigate Athlete 1's mis-hitting shot, right-clicking and selecting the "Ball-Trajectory" option. Starting from the system-provided starting point (a flashing circular area), Expert B held down the left mouse button to draw a smooth arc, positioning the ping-pong ball's landing point at the right corner of the table. Figure 6 In the middle (A1), click to confirm the query. Then, expert B selects the last two shots and clicks the query button on the right side of the video view.

[0062] In the query results ( Figure 6 In (B1), expert B discovered that the first result tree node in the modified view had only one child node. Figure 6 (B4) This indicates that the database retrieved Athlete 2's defensive paddle, but not Athlete 1's error paddle after adding interactive query conditions. This suggests that few athletes choose this offensive approach to counter the defensive tactics in the video. Expert B observed that the dark-colored blocks in the nodes matching the defensive paddle were not high, indicating that the defensive paddle player was unlikely to win the round, while the error paddle player had a high probability of winning the round after adjustment. Figure 6 (B4) He clicked on the first tree node in the third row (i.e., the query result that matched athlete 2's defensive shot but did not match athlete 1's mistake shot), and saw a row of data distribution cards in the query results view ( Figure 6 (B2) Expert B observed that in the "Position" card, the bar corresponding to "Attack" was the second longest, and its length was close to the longest "Rally" bar. This indicated that many athletes, like Athlete 1, chose to attack when facing Athlete 2's defensive shot tactic in the video. Therefore, he clicked on the bar corresponding to "Attack," and the data in other cards was switched to the statistics for the "Attack" shot. Expert B found that in the "Position" section, the selection of "Forehand Position" (the position for a failed shot) was half the number of the other two selections ("Side Position" and "Backhand Position"), indicating that "Forehand Position" was the choice of a minority of athletes. Therefore, he clicked on the option containing "Side Position" to filter, and observed in the "Landing Point" card that about 2 / 3 of the athletes chose "Long ball down the middle," indicating that a long ball down the middle with a side-position is a safe choice.

[0063] After some thought, Expert B adjusted his previous query. He deleted the original trajectory and, while still holding down the left mouse button, drew a smooth curve from the indicated starting point to the center of the table. Figure 6 In the middle (A2), select the last two frames, and slide the slider in the upper right corner to the left to relax the matching strictness. Figure 6 (A3) Then search again.

[0064] The right side search result view generates a more balanced binary tree at this time Figure 6 (C1), and the third row first tree node (i.e., the query result matching both search conditions, such as Figure 6 (C6) is clicked, and a series of search results are displayed in the lower round details view. Expert B selects the "body position" option from the filter selection box and selects the previously observed "side position" in the right sub-option box Figure 6 (C2), and further filters the results. He finds a round segment near the end of the search results in the relevant shot, which comes from the 2019 Chengdu World Cup match between player 3 and player 4. Player 4's defense shot completely matches player 2's defense shot. Although the drop point and the selected interactive video drop point are slightly different, both are forehand long balls Figure 6 (C3), and are near the center line. Player 3 (corresponding to the added interactive query condition of player 1's error shot) chooses a long ball side position arc loop shot attack, rather than using the error shot (in the original video segment) forehand long ball arc loop shot Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure 6 Figure (C4). Player 3 attacks the opponent's loophole when playing the arc loop shot with the opponent, and finally wins the round.

[0065] The specific embodiments described above have detailed the technical solutions and beneficial effects of the present application. It should be understood that the above description is only the most preferred embodiment of the present application and is not intended to limit the present application. Any modifications, supplements, and equivalent replacements made within the principle range of the present application should be included in the protection scope of the present application.

Claims

1. An interactive system for exploring a collection of videos using embedded visualization, characterized in that, include: Video view module, query view module, data view module; The video view module displays video and extracts data, provides interactive functions with the video, and allows interactive queries through specified attributes; The query view module divides the video interactive query into layers and displays the division results; The data view module displays detailed information about video data selected from the query view module; The video view module also provides video interaction functions for specified queries, specifically including area interaction, trajectory interaction, speed interaction and technology interaction; For area interaction, users use the lasso to select a specified area, creating a closed polygonal area by specifying the area where the ball lands or the area where the player moves. This interaction can be used to define the ball's landing position and the player's position. The area selected by the mouse will be mapped to the position of the table using a pre-extracted perspective matrix. For trajectory interaction, users can draw a continuous curve by left-clicking and dragging to represent the trajectory of the ball or the athlete. When the video is playing, the trajectory will be automatically displayed on the video. When the user selects a shot to draw the trajectory of the ball, the video will automatically switch to the serve frame and use a flashing semi-transparent circle to indicate the hitting position. For speed-related interactions, if the user keeps clicking the left mouse button, the height of the bar representing the speed-related value will continuously increase; For technical interaction, a circular interface is provided for users to select the hitting technique, and the techniques are divided into different categories. Users first select the technique category from the large circle, and then select the specific technique from the small circles that appear.

2. The interactive system for exploring a video collection using embedded visualization according to claim 1, characterized in that, The video view module has basic interactive functions. The video control bar represents video frames using small rectangles. Users can play, pause, or control the video process through the video control bar. It also has a speed control button to allow users to change the video playback speed.

3. The interactive system for exploring a video collection using embedded visualization according to claim 1, characterized in that, The video view module also provides a shot information panel, specifically a panel list set under the video control bar, to display the specific information of each shot, and to encode the time range of each shot with bars of different lengths. Each panel corresponding to each shot displays multi-dimensional shot information. When playing the video, the pen screen panel corresponding to the current frame will be expanded and highlighted, and the user can jump to the corresponding video frame by clicking the corresponding panel.

4. The interactive system for exploring a video collection using embedded visualization according to claim 1, characterized in that, The video view module also has a video interactive function for viewing data. Users can select statistics from the interactive menu, right-click the video to display this menu, and the statistics will be displayed in the interactive video.

5. The interactive system for exploring a video collection using embedded visualization according to claim 1, characterized in that, The video view module also has an interactive recording function. After the user interacts with the video and specifies the query conditions, the interaction will be recorded below the video's shot information panel. The user can select multiple shots on the panel, specify the query conditions, and click the query button to perform a query in the database. When viewing the query result video, the video will be automatically slowed down within the relevant time period that meets the query conditions. The user is also allowed to switch between the original video and the queried video.

6. The interactive system for exploring a video collection using embedded visualization according to claim 1, characterized in that, In the query view module, a tree structure is used to hierarchically divide the video interactive query results. Specifically, the queries are first sorted according to the order of selection. Based on whether they meet the query conditions of the first query Q1, the video set is divided into two groups: matching or not matching Q1. These two groups will be presented as the second layer in the tree structure. Then, each group is iteratively divided according to the next query, finally resulting in a tree structure. The top layer represents the original video set, and the relevance score of each node representing a group of video segments in the query tree is further calculated. Relevance = number of matched shot attributes / total number of shot attributes in the query. This relevance is encoded using gradient colors. In each matching node, a dark-colored bar is set, and the height of the bar is used to encode the win rate. The higher the bar, the higher the win rate.

7. The interactive system for exploring a video collection using embedded visualization according to claim 6, characterized in that, In the query view module, clicking on a node allows users to view the data distribution of different attributes for the corresponding video set. Detailed information about the corresponding video group will also be displayed in the data view. Users can scroll through the attribute distribution panel and filter data in other panels by clicking on attributes in the attribute distribution panel.

8. The interactive system for exploring a video collection using embedded visualization according to claim 1, characterized in that, In the data view module, when a user clicks on a node in the query tree, the basic information of each round corresponding to the node is displayed at the top of the round information. The specific racket details are presented in a table format, where the basic information includes the name of the match, the position in the match, the server and the winner. A filter selection box is set in the upper right corner of the view, allowing users to select filter conditions and attribute values ​​to filter query results.

Images