Multi-window video playback method and system, and terminal device

Abstract

The present application discloses a multi-window video playback method, system, and terminal device applied in the field of video surveillance technology, which is used to control the slave window to follow and execute the user's video operation actions on the main window when the user requires less operation or no operation, simplifying the user operation and improving the user experience. The multi-window video playback method provided by the present application includes: when there is a main window and a slave window for playing the video on the user interface, monitoring the user's video operation actions on the main window; when monitoring the user's video operation actions on the main window, executing the video operation actions, and determining the capability tree of the slave window; according to the capability tree of the slave window and the user's video operation actions on the main window, determining the video operation action sequence to be executed on the slave window, and executing the video operation action sequence.

Description

Technical Field

[0001] The present application relates to the field of video surveillance technology, and in particular to a multi-window video playback method and system, and a terminal device. Background Art

[0002] In the field of video surveillance, a server is equipped with multiple channels for connecting multiple surveillance devices. Multiple windows can be set on the user interface of the terminal device to play the recordings of multiple channels. When the user selects a window as the main window and performs video operations on the main window, such as setting the video stream, starting playback, pausing, setting the playback speed, previous frame, next frame, jump, etc., the secondary window cannot automatically follow the user's video operation actions on the main window without the user's participation. Summary of the invention

[0003] The embodiments of the present application provide a multi-window video playback method and system, and a terminal device, which are used to control the slave window to follow the user's video operation actions on the main window with little or no operation by the user, thereby simplifying user operations and improving user experience.

[0004] The embodiment of the present application provides a multi-window video playback method, including:

[0005] When a main window and a sub-window for playing a video exist on the user interface, monitoring a user's video operation action on the main window;

[0006] When monitoring a user's video operation action on the main window, the video operation action is executed, and a capability tree of the slave window is determined, the capability tree includes a root node and at least one leaf node, wherein the root node is used to represent an action of establishing the slave window, and each leaf node is used to represent an operation action performed on the video in the slave window;

[0007] According to the capability tree of the slave window and the video operation action of the user on the master window, a video operation action sequence to be executed on the slave window is determined, and the video operation action sequence is executed.

[0008] Through this method, when there are a main window and a slave window for playing videos on the user interface, the user's video operation actions on the main window are monitored; when the user's video operation actions on the main window are monitored, the video operation actions are executed, and the capability tree of the slave window is determined, the capability tree includes a root node and at least one leaf node, wherein the root node is used to represent the action of establishing the slave window, and each leaf node is used to represent the operation action performed on the video in the slave window; according to the capability tree of the slave window and the user's video operation actions on the main window, the video operation action sequence to be performed on the slave window is determined, and the video operation action sequence is executed, so that the slave window is controlled to follow and execute the user's video operation actions on the main window with less or no operation by the user, thereby simplifying user operations and improving user experience.

[0009] In some embodiments, determining a video operation action sequence to be performed on the slave window according to the capability tree of the slave window and the video operation action of the user on the master window includes:

[0010] According to the video operation action of the user on the main window, searching, from the leaf nodes of the capability tree of the slave window, a leaf node representing the same action as the video operation action of the user on the main window;

[0011] When the search is successful, the found leaf node is used as the target node, and the predecessor node of the target node is determined, where the predecessor node is the node between the target node and the root node;

[0012] Based on the target node and the preceding node, a video operation action sequence to be performed on the slave window is determined.

[0013] In some embodiments, based on the target node and the preceding node, determining a video operation action sequence to be performed on the slave window includes:

[0014] Determine necessary nodes according to the execution status of the video operation actions corresponding to the target node and the preceding node;

[0015] The video operation actions corresponding to the necessary nodes are determined as a video operation action sequence that needs to be executed on the slave window.

[0016] Through this method, it is possible to determine the pre-video operation action that needs to be performed by the slave window before the slave window performs the user's video operation action on the main window.

[0017] In some embodiments, executing the video operation action sequence includes:

[0018] The video operation action sequence is executed in sequence from the root node to the target node.

[0019] In some embodiments, the method further comprises:

[0020] When searching for a leaf node representing the same action as the video operation action of the user on the main window fails, it is determined that the video operation action sequence to be executed on the slave window is empty.

[0021] In some embodiments, the method further comprises:

[0022] When it is monitored that the user sets the slave window to play video streams of different channels in different time periods, when switching channels of the slave window, the following is executed:

[0023] Determine the channel information to be switched from the window, and determine the target capability tree form corresponding to the channel information to be switched according to the preset correspondence between the channel information and the capability tree form;

[0024] When the target capability tree form is inconsistent with the current form of the capability tree of the slave window, the capability tree of the slave window is updated according to the target capability tree form.

[0025] Through this method, the capability tree of the slave window is automatically updated according to the channel information set by the user for the slave window.

[0026] In some embodiments, after executing the video operation action sequence, the method further includes:

[0027] According to the type of each video operation action in the video operation action sequence, when it is determined that the video operation action belongs to the first type, modifying the execution state of the node corresponding to the video operation action to executed;

[0028] When it is determined that the video operation action belongs to the second type, the execution state of the node corresponding to the video operation action is maintained as unexecuted.

[0029] Through this method, the execution status of the executed first type of nodes is updated.

[0030] The embodiment of the present application provides a multi-window video playback system, including:

[0031] A monitoring module, used for monitoring the video operation actions of the user on the main window when there are a main window and a sub window for playing the video on the user interface;

[0032] a determination module, configured to, when monitoring a user's video operation action on the main window, execute the video operation action, and determine a capability tree of the slave window, wherein the capability tree includes a root node and at least one leaf node, wherein the root node is used to represent an action of establishing the slave window, and each leaf node is used to represent an operation action performed on the video in the slave window;

[0033] The control module is used to determine a video operation action sequence to be executed on the slave window according to the capability tree of the slave window and the video operation action of the user on the master window, and execute the video operation action sequence.

[0034] Through this system, with little or no user operation, the slave window can be controlled to follow and execute the user's video operation actions on the main window, thereby simplifying user operations and improving user experience.

[0035] Another embodiment of the present application provides a terminal device, which includes a memory and a processor, wherein the memory is used to store program instructions, and the processor is used to call the program instructions stored in the memory, and execute any of the above methods according to the obtained program.

[0036] Furthermore, according to an embodiment, for example, a computer program product for a computer is provided, which includes software code portions, and when the product is run on the computer, these software code portions are used to perform the steps of the method defined above. The computer program product may include a computer-readable medium having the software code portions stored thereon. Furthermore, the computer program product may be directly loaded into the internal memory of the computer and / or sent via a network through at least one of an upload process, a download process, and a push process.

[0037] Another embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable the computer to execute any of the above methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A schematic diagram of the overall process of a multi-window video playback method provided in an embodiment of the present application;

[0040] Figure 2A capability tree form provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of a flow chart for determining a sequence of video operation actions to be performed on a slave window provided in an embodiment of the present application;

[0042] Figure 4 A flowchart of determining a sequence of video operation actions to be performed on a slave window based on a target node and a preceding node provided in an embodiment of the present application;

[0043] Figure 5 Another capability tree form provided in the embodiment of the present application;

[0044] Figure 6 A schematic diagram of the structure of a multi-window video playback system provided in an embodiment of the present application;

[0045] Figure 7 A schematic diagram of a specific process of a multi-window video playback method provided in an embodiment of the present application;

[0046] Figure 8 A schematic diagram of a specific process of another multi-window video playback method provided in an embodiment of the present application;

[0047] Fig. 9 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] The embodiments of the present application provide a multi-window video playback method and system, and a terminal device, which are used to control the slave window to follow the user's video operation actions on the main window with little or no operation by the user, thereby simplifying user operations and improving user experience.

[0050] Among them, the method and the device are based on the same application concept. Since the method and the device solve the problem in a similar principle, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0051] The terms "first", "second", etc. (if any) in the specification and claims of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in a sequence other than the content illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0052] The following examples and embodiments are to be understood as illustrative examples only. Although this specification may refer to "one", "an" or "some" examples or embodiments in several places, this does not mean that each such reference relates to the same example or embodiment, nor does it mean that the feature applies only to a single example or embodiment. Individual features of different embodiments may also be combined to provide other embodiments. In addition, terms such as "including" and "comprising" should be understood as not limiting the described embodiments to consist only of those features that have been mentioned; such examples and embodiments may also include features, structures, units, modules, etc. that are not specifically mentioned.

[0053] The following is a detailed description of each embodiment of the present application in conjunction with the accompanying drawings. It should be noted that the display order of the embodiments of the present application only represents the order of the embodiments, and does not represent the advantages and disadvantages of the technical solutions provided by the embodiments.

[0054] It should be noted that the technical solution provided in the embodiment of the present application is explained by opening multiple windows for playing videos on the user interface of a terminal device. When the user performs a certain video operation on the selected main window, the slave window follows and executes the corresponding video operation as an example, but it is not limited to this.

[0055] See also Figure 1 , a multi-window video playback method provided in an embodiment of the present application, for example, is executed by a video playback terminal device, including:

[0056] Step S10: When a main window and a sub-window for playing a video exist on the user interface, monitoring a user's video operation action on the main window;

[0057] The main window may be a window selected by the user through the user interface. The number of slave windows is at least 1. The main window and the slave window may only be in the open state without playing any video.

[0058] Video operation actions, such as zooming the window, setting the video bitstream, starting playback, pausing, setting playback speed, jumping, previous frame, next frame, fisheye correction, changing the correction area, etc.

[0059] Setting a video stream means binding a window to a channel (i.e., monitoring device) to play the video stream pulled from the channel.

[0060] Step S20: when a user's video operation action on the main window is monitored, the video operation action is executed, and a capability tree of the slave window is determined, the capability tree includes a root node and at least one leaf node, wherein the root node is used to represent an action of establishing the slave window, and each leaf node is used to represent an operation action performed on the video in the slave window;

[0061] For example Figure 2 In the capability tree shown, establishing a window corresponds to the root node. Except for establishing a window, each other video operation action corresponds to a leaf node. The subsequent nodes of the root node are setting the video bitstream and scaling the window. The subsequent nodes of setting the video bitstream are setting the playback speed and starting playback. The subsequent nodes of starting playback are jumping and pausing. The subsequent nodes of pausing are previous frame and next frame.

[0062] In this step, for example, when establishing the capability tree of the slave window, a correspondence can be established between the identifier of the slave window and the established capability tree. Later, the identifier of the slave window is used to search the correspondence table between the identifier and the capability tree to determine the capability tree of each slave window.

[0063] After the capability tree is created in the window, according to the preset initial capability tree form, for example Figure 2 As shown, a capability tree is established for a newly created slave window. After the capability tree is established, the capability tree of the slave window can be updated according to the channel information set by the user for the slave window, or the capability tree form selected by the user for the slave window. Of course, when the user establishes a slave window, the user can establish an initial capability tree for the slave window according to the capability tree pre-selected by the user, and this application does not impose any restrictions on this.

[0064] Step S30: Determine a video operation action sequence to be executed on the slave window according to the capability tree of the slave window and the video operation action of the user on the master window, and execute the video operation action sequence.

[0065] If the user performs an operation on the video in the main window, such as pausing, then in this step, according to the capability tree of the slave window, for example Figure 2The capability tree shown determines each node from establishing a window to pausing, including: establishing a window 0, setting a video code stream 1, starting playback 2, and pausing 3, forming a node sequence 0, 1, 2, and 3, and then according to the execution status of each node in this node sequence, determines the necessary nodes that need to be executed, and uses the video operation actions corresponding to these necessary nodes as the video operation action sequence that needs to be executed by the slave window, and then controls the slave window to execute the video operation action sequence, for example, if the execution status of establishing a window and setting the video code stream is executed, the corresponding nodes of starting playback and pausing can be used as necessary nodes to generate a video operation action sequence that needs to be executed by the slave window. The slave window only needs to execute starting playback and pausing, and the slave window can follow the main window to perform the corresponding video operation actions.

[0066] The multi-window video playback method provided in the embodiment of the present application monitors the user's video operation actions on the main window. When the user's video operation actions on the main window are monitored, the video operation action sequence to be performed on the slave window is determined according to the capability tree of the slave window and the user's video operation actions on the main window, and the video operation action sequence is executed. This achieves the situation where the user only needs to perform less or no operation, and controls the slave window to follow and execute the user's video operation actions on the main window, thereby simplifying user operations and improving user experience.

[0067] In the above step S30, according to the capability tree of the slave window and the video operation action of the user on the master window, the video operation action sequence to be performed on the slave window is determined, for example Figure 3 As shown, the specific steps include:

[0068] Step S301: According to the video operation action of the user on the main window, searching from the leaf nodes of the capability tree of the slave window for a leaf node representing the same action as the video operation action of the user on the main window;

[0069] Step S302: when the search is successful, the found leaf node is used as the target node, and the predecessor node of the target node is determined, where the predecessor node is a node between the target node and the root node;

[0070] Step S303: Determine a video operation action sequence to be performed on the slave window based on the target node and the preceding node.

[0071] The user operates the video of the main window, such as jumping, to determine the capability tree of the slave window, such as Figure 2The capability tree shown in the figure is provided with a leaf node corresponding to the jump, and the leaf node is used as the target node. The preceding node of the target node is determined to include setting the video code stream and starting the playback, and then according to the execution status of setting the video code stream, starting the playback and jumping, the video operation action sequence to be executed from the window is determined.

[0072] In the above step S303, based on the target node and the preceding node, a video operation action sequence to be performed on the slave window is determined, for example Figure 4 As shown, the specific steps include:

[0073] Step S3031, determining a necessary node according to the execution status of the video operation action corresponding to the target node and the preceding node;

[0074] Step S3032: Determine the video operation action corresponding to the necessary node as a video operation action sequence that needs to be executed for the slave window.

[0075] Necessary nodes, such as nodes whose execution status is Not Executed.

[0076] In some embodiments, executing the video operation action sequence includes:

[0077] The video operation action sequence is executed in sequence from the root node to the target node.

[0078] For example Figure 2 In the capability tree shown, if the target node is the previous frame, the order from the root node to the target node is to establish a window, set the video stream, start playing, pause, and go to the previous frame. If the video operation action sequence is to start playing, pause, and go to the previous frame, the control starts playing from the window first, then pauses, and finally goes to the previous frame.

[0079] After the above step S301, for example Figure 3 As shown, the following steps may also be included:

[0080] Step S304: when searching for a leaf node representing the same action as the video operation action of the user on the main window fails, determining that the video operation action sequence to be executed on the slave window is empty.

[0081] If there is no node corresponding to the video operation action of the user on the main window in the capability tree of the slave window, it means that the slave window does not need to follow the main window to perform the corresponding video operation action. For example, if the user's video operation action on the main window is to change the correction area, the capability tree of the slave window is as follows: Figure 2 In the form shown, there is no node corresponding to the video operation action on the capability tree of the slave window, so the slave window does not need to follow the user's video operation action on the main window.

[0082] A multi-window video playback method provided in an embodiment of the present application may further include the following steps:

[0083] When it is monitored that the user sets the slave window to play video streams of different channels in different time periods, when switching channels of the slave window, the following is executed:

[0084] Determine the channel information to be switched from the window, and determine the target capability tree form corresponding to the channel information to be switched according to the preset correspondence between the channel information and the capability tree form;

[0085] When the target capability tree form is inconsistent with the current form of the capability tree of the slave window, the capability tree of the slave window is updated according to the target capability tree form.

[0086] For example, the user sets the window to play the video stream of monitoring device 1 from 10:00 to 12:00, the video stream of monitoring device 2 from 12:00 to 14:00, and the video stream of monitoring device 3 from 14:00 to 16:00. Assuming that monitoring devices 1 and 2 are ordinary cameras and monitoring device 3 is a fisheye camera, the capability tree corresponding to ordinary cameras is as follows Figure 2 As shown, the capability tree corresponding to the fisheye camera is as follows: Figure 5 As shown, when the channel of the window is switched from monitoring device 2 to monitoring device 3, since the capability trees corresponding to the two channels are different in form, when the channel is switched to monitoring device 3, the capability tree of the window needs to be adjusted to Figure 5 The ability tree shown in the figure requires Figure 2 Two leaf nodes are added to the capability tree shown: fisheye correction and change correction area.

[0087] According to the channel information pre-set by the user for the slave window, when switching channels, the capability tree of the slave window can be determined whether the current form of the capability tree of the slave window needs to be updated according to the corresponding relationship between the channel information to be switched and the capability tree; the user can also select a form for the capability tree of the slave window according to the needs, and update the capability tree of the slave window according to the capability tree form selected by the user; the user can also set the capability tree form of the slave window to have different forms in different time periods, for example, the user sets the capability tree of the slave window from 10 o'clock to 12 o'clock as Figure 2 The form shown, from 12 o'clock to 14 o'clock the ability tree of the window is as follows Figure 5 The form shown, and then according to the ability tree form preset by the user, the ability tree of the slave window is updated within the corresponding time period.

[0088] It should be noted that the capability tree of the slave window is only valid while the slave window is open. When the slave window is closed, the capability tree of the slave window becomes invalid.

[0089] In the above step S30, after executing the video operation action sequence, the following steps may also be included:

[0090] According to the type of each video operation action in the video operation action sequence, when it is determined that the video operation action belongs to the first type, modifying the execution state of the node corresponding to the video operation action to executed;

[0091] When it is determined that the video operation action belongs to the second type, the execution state of the node corresponding to the video operation action is maintained as unexecuted.

[0092] The first type indicates a one-time action or instantaneous action that is not persistent, such as creating a window, setting a video stream, starting playback, or pausing. The execution status statu of the node corresponding to this type of video operation action needs to be changed to true (executed) after the video operation action is executed once.

[0093] It should be noted that after a slave window is set to play the video stream of a certain channel, the execution status of the node corresponding to the video stream is set to true. When the video stream of the channel is finished playing, the execution status of the node corresponding to the video stream needs to be set to false (not executed), indicating that the video stream needs to be reset for the slave window.

[0094] If the start play has been executed from the window, the execution status of the start play is executed. When the pause is executed from the window, the execution status of the start play needs to be set to not executed. Similarly, when the start play and pause have been executed from the window, when the start play is executed from the window again, the execution status of the pause needs to be changed from executed to not executed.

[0095] The second type indicates an action that can be executed repeatedly, such as zooming the window, setting the playback speed, jumping, previous frame, and next frame. The execution status statu of the node corresponding to this type of video operation action remains false.

[0096] See also Figure 6 , a multi-window video playback system provided by an embodiment of the present application includes:

[0097] The monitoring module 100 is used to monitor the video operation actions of the user on the main window when there are a main window and a sub window for playing the video on the user interface;

[0098] The determination module 200 is used to perform the video operation action on the main window when monitoring the user's video operation action, and determine the capability tree of the slave window, the capability tree including a root node and at least one leaf node, wherein the root node is used to represent the action of establishing the slave window, and each leaf node is used to represent the operation action performed on the video in the slave window;

[0099] The control module 300 is used to determine a video operation action sequence to be executed on the slave window according to the capability tree of the slave window and the video operation action of the user on the master window, and execute the video operation action sequence.

[0100] Several specific method flow examples are given below.

[0101] Embodiment 1:

[0102] See also Figure 7 , a multi-window video playback method provided by an embodiment of the present application, specifically comprising the following steps:

[0103] For example, a user creates three windows in a terminal device's user interface, including a main window and two slave windows. The capability trees of the two slave windows are Figure 2 In the form shown, the user operates the jump operation on the main window;

[0104] Step S701: The user creates multiple windows on the user interface, selects one window as the main window, and monitors the user's video operation actions on the main window;

[0105] Step S702: when it is detected that the user has performed a jump operation on the main window, the main window is controlled to perform the jump operation, and the capability tree of each slave window stored locally is obtained;

[0106] Step S703: According to the user's jump operation on the main window, determine whether there is a leaf node corresponding to the jump in the capability tree of the slave window. If so, proceed to step S704; if not, proceed to step S706;

[0107] Step S704: Taking the leaf node corresponding to the jump as the target node, traverse the capability tree of the slave window to determine the nodes between the root node and the target node;

[0108] The nodes between the root node and the target node refer to the intermediate nodes connecting the root node to the target node.

[0109] Step S705: Generate a node sequence corresponding to the slave window using the nodes between the root node and the target node and the target node, where the node sequence includes the node order of each node;

[0110] Node sequence, for example, set video stream -> start playback -> jump.

[0111] Step S706, generating a sequence of empty nodes corresponding to the slave window;

[0112] Step S707, determining necessary nodes according to the execution status of each node in the node sequence corresponding to the window;

[0113] For example, if the video code stream has been set from window 1, but not from window 2, the video operation actions corresponding to the necessary nodes from window 1 are start playback and jump, and the video operation actions corresponding to the necessary nodes from window 2 are set video code stream, start playback and jump.

[0114] Step S708, according to the node sequence of the necessary nodes corresponding to the slave window, controlling the slave window to sequentially execute the video operation actions corresponding to the necessary nodes;

[0115] The above steps start playing and jump in sequence from window 1, and set video code stream, start playing and jump in sequence from window 2.

[0116] It should be noted that if the user has configured the channel information for the slave window in advance, when setting the video stream, the corresponding video stream can be automatically set for the slave window according to the channel information configured by the user. If the user has not set it, it is necessary to remind the user through the user interface to set the video stream for the slave window. After the user completes the operation of setting the video stream, the playback can be started.

[0117] Step S709: according to the type of the video operation action corresponding to the executed necessary node, determine whether the execution status of the executed necessary node needs to be modified. If yes, proceed to step S710; if not, proceed to step S701;

[0118] For example, if the type of video operation action corresponding to the necessary node is the first type, then after the video operation action corresponding to the necessary node is executed, its execution status needs to be modified; if the type of video operation action corresponding to the necessary node is the second type, then after the video operation action corresponding to the necessary node is executed, its execution status does not need to be modified.

[0119] Step S710: modify the execution status of the executed necessary node to be executed.

[0120] If the user performs a certain video operation on the main window multiple times in a row, a node sequence can be generated for the slave window for each video operation. For example, if it is performed three times in a row, three node sequences can be generated, and then the video operation actions corresponding to the necessary nodes in each node sequence can be executed one by one according to the generation time sequence of the node sequences. It is also possible to generate only one node sequence and mark the number of times the user has performed the video operation on the main window in the node sequence.

[0121] For example, from the window's capability tree Figure 2 In the form shown, the user performs three consecutive jump operations on the main window, and then three node sequences of setting video code stream -> starting playback -> jumping can be generated for the slave window. The generation time of the three node sequences is different. According to the generation time, after executing the video operation actions corresponding to the necessary nodes in the first node sequence, there is no need to repeat the setting video code stream and starting playback in the second and third node sequences, and only the jump needs to be executed; it is also possible to generate a node sequence of setting video code stream -> starting playback -> jumping (3 times) for the slave window. After setting the video code stream and starting playback are executed in the slave window, the slave window is controlled to jump three times in a row.

[0122] It should be noted that a capability tree can also be established for each window created by the user. There is no need to select a main window. As long as the user performs a video operation on one of the windows, the video operation action sequence corresponding to the other windows can be determined based on the capability trees of the other windows, and the other windows can be controlled to execute the video operation action sequence. In this way, the user can perform video operation actions on any window, and other windows can follow suit, further simplifying the operation and improving the user experience.

[0123] Embodiment 2:

[0124] See also Figure 8 , a multi-window video playback method provided by an embodiment of the present application, specifically comprising the following steps:

[0125] For example, the user creates a window in each of the user interfaces of terminal devices 1, 2, and 3. The capability tree of the window of terminal device 1 is: Figure 2 In the form shown, the capability tree of the windows of terminal devices 2 and 3 is Figure 5 In the form shown, the user operates the window of the terminal device 3 to correct the correction area, and the terminal devices communicate with each other through wired or wireless communication.

[0126] Each terminal device is equipped with the above-mentioned multi-window video playback system, which can be used to execute the above-mentioned multi-window video playback method.

[0127] Step S801: A user creates a window on each of the user interfaces of multiple terminal devices, and each terminal device creates a capability tree for the corresponding window;

[0128] Step S802: The user operates to correct the correction area on terminal device 3, and terminal device 3 sends a message carrying the corrected correction area to terminal devices 1 and 2;

[0129] Step S803: After receiving the message, the terminal devices 1 and 2 respectively obtain the capability tree stored locally and determine whether the capability tree has a leaf node representing the same action as the correction area; if so, proceed to step S804; if not, proceed to step S806;

[0130] If terminal device 1 determines that there is no leaf node in the capability tree that represents the same action as the corrected correction area, terminal device 1 proceeds to step S806; if terminal device 2 determines that there is a leaf node in the capability tree that represents the same action as the corrected correction area, terminal device 2 proceeds to step S804.

[0131] Step S804: taking the leaf node corresponding to the corrected area as the target node, traversing the capability tree, and determining the node between the root node and the target node;

[0132] Step S805, generating a node sequence using the nodes between the root node and the target node and the target node;

[0133] Step S806: Generate an empty node sequence;

[0134] Step S807, determining necessary nodes according to the execution status of each node in the node sequence;

[0135] In this step, the necessary node determined by terminal device 1 is 0.

[0136] Step S808, according to the node sequence of the necessary nodes, the control window sequentially executes the video operation actions corresponding to the necessary nodes;

[0137] Since the node sequence generated by the terminal device 1 in step S806 is empty, there is no need to control the corresponding window to perform the video operation action corresponding to the necessary node in this step, that is, no video operation action is performed.

[0138] It should be noted that multiple windows can be established on the user interface of each terminal device. The user performs a video operation on one of the windows. This window is used as the main window. The terminal device of the main window sends the video operation to other terminal devices. Each terminal device uses the above method to enable other windows to follow and execute the user's video operation on the main window.

[0139] Step S809, the terminal device 2 determines whether the execution status of the executed necessary node needs to be modified according to the type of the video operation action corresponding to the executed necessary node, and if so, proceeds to step S810; if not, proceeds to step S801;

[0140] Step S810: The terminal device 2 modifies the execution status of the executed necessary node to executed.

[0141] The following is an introduction to the equipment or device provided in the embodiments of the present application, in which the explanations or examples of technical features that are the same or corresponding to those described in the above method will not be repeated later.

[0142] See also Fig. 9 , a terminal device provided in an embodiment of the present application includes:

[0143] The processor 910 is used to read the program in the memory 930 and execute the following process:

[0144] When a main window and a sub-window for playing a video exist on the user interface, monitoring a user's video operation action on the main window;

[0145] When monitoring a user's video operation action on the main window, the video operation action is executed, and a capability tree of the slave window is determined, the capability tree includes a root node and at least one leaf node, wherein the root node is used to represent an action of establishing the slave window, and each leaf node is used to represent an operation action performed on the video in the slave window;

[0146] According to the capability tree of the slave window and the video operation action of the user on the master window, a video operation action sequence to be executed on the slave window is determined, and the video operation action sequence is executed.

[0147] In some embodiments, determining a video operation action sequence to be performed on the slave window according to the capability tree of the slave window and the video operation action of the user on the master window includes:

[0148] According to the video operation action of the user on the main window, searching, from the leaf nodes of the capability tree of the slave window, a leaf node representing the same action as the video operation action of the user on the main window;

[0149] When the search is successful, the found leaf node is used as the target node, and the predecessor node of the target node is determined, where the predecessor node is the node between the target node and the root node;

[0150] Based on the target node and the preceding node, a video operation action sequence to be performed on the slave window is determined.

[0151] In some embodiments, based on the target node and the preceding node, determining a video operation action sequence to be performed on the slave window includes:

[0152] Determine necessary nodes according to the execution status of the video operation actions corresponding to the target node and the preceding node;

[0153] The video operation actions corresponding to the necessary nodes are determined as a video operation action sequence that needs to be executed on the slave window.

[0154] In some embodiments, executing the video operation action sequence includes:

[0155] The video operation action sequence is executed in sequence from the root node to the target node.

[0156] In some embodiments, the processor 910 is further configured to execute:

[0157] When searching for a leaf node representing the same action as the video operation action of the user on the main window fails, it is determined that the video operation action sequence to be executed on the slave window is empty.

[0158] In some embodiments, the processor 910 is further configured to execute:

[0159] When it is monitored that the user sets the slave window to play video streams of different channels in different time periods, when switching channels of the slave window, the following is executed:

[0160] Determine the channel information to be switched from the window, and determine the target capability tree form corresponding to the channel information to be switched according to the preset correspondence between the channel information and the capability tree form;

[0161] When the target capability tree form is inconsistent with the current form of the capability tree of the slave window, the capability tree of the slave window is updated according to the target capability tree form.

[0162] In some embodiments, after executing the video operation action sequence, the method further includes:

[0163] According to the type of each video operation action in the video operation action sequence, when it is determined that the video operation action belongs to the first type, modifying the execution state of the node corresponding to the video operation action to executed;

[0164] When it is determined that the video operation action belongs to the second type, the execution state of the node corresponding to the video operation action is maintained as unexecuted.

[0165] In some embodiments, the terminal device provided in the embodiments of the present application also includes a transceiver 920 for receiving and sending data to a monitoring device under the control of the processor 910 .

[0166] Among them, Fig. 9 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 910 and memory represented by memory 930. The bus architecture may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are not further described herein. The bus interface provides an interface. The transceiver 920 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices over a transmission medium.

[0167] The processor 910 is responsible for managing the bus architecture and general processing, and the memory 930 can store data used by the processor 910 when performing operations.

[0168] In some embodiments, the processor 910 may be a CPU (central processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a CPLD (Complex Programmable Logic Device).

[0169] The embodiment of the present application provides a computing device, which may be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), etc. The computing device may include a central processing unit (CPU), a memory, an input / output device, etc. The input device may include a keyboard, a mouse, a touch screen, etc. The output device may include a display device, such as a liquid crystal display (LCD), a cathode ray tube (CRT), etc.

[0170] The memory may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor with program instructions and data stored in the memory. In the embodiment of the present application, the memory may be used to store the program of any of the methods provided in the embodiment of the present application.

[0171] The processor calls the program instructions stored in the memory, and the processor is used to execute any of the methods provided in the embodiments of the present application according to the obtained program instructions.

[0172] The present application embodiment also provides a computer program product or computer program, which includes a computer instruction, which is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction so that the computer device performs any of the methods described in the above embodiments. The program product can use any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0173] The embodiment of the present application provides a computer-readable storage medium for storing computer program instructions used by the apparatus provided in the embodiment of the present application, which includes a program for executing any of the methods provided in the embodiment of the present application. The computer-readable storage medium may be a non-transitory computer-readable medium.

[0174] The computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0175] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0176] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0177] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0179] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A multi-window video playback method, characterized in that: include: When a main window and a sub-window for playing a video exist on the user interface, monitoring a user's video operation action on the main window; When monitoring a user's video operation action on the main window, the video operation action is executed, and a capability tree of the slave window is determined, the capability tree includes a root node and at least one leaf node, wherein the root node is used to represent an action of establishing the slave window, and each leaf node is used to represent an operation action performed on the video in the slave window; According to the user's video operation action on the main window, the leaf node representing the same action as the user's video operation action on the main window is searched from the leaf nodes of the capability tree of the slave window. When the search is successful, the found leaf node is used as the target node, and the predecessor node of the target node is determined. According to the execution status of the video operation actions corresponding to the target node and the predecessor node, the necessary node is determined, and the video operation action corresponding to the necessary node is determined as a video operation action sequence that needs to be executed for the slave window, and the video operation action sequence is executed, wherein the predecessor node is a node between the target node and the root node.

2. The method according to claim 1, characterized in that: Executing the video operation action sequence includes: The video operation action sequence is executed in sequence from the root node to the target node.

3. The method according to claim 1, characterized in that: The method further comprises: When searching for a leaf node representing the same action as the video operation action of the user on the main window fails, it is determined that the video operation action sequence to be executed on the slave window is empty.

4. The method according to claim 1, characterized in that: The method further comprises: When it is monitored that the user sets the slave window to play video streams of different channels in different time periods, when switching channels of the slave window, the following is executed: Determine the channel information to be switched from the window, and determine the target capability tree form corresponding to the channel information to be switched according to the preset correspondence between the channel information and the capability tree form; When the target capability tree form is inconsistent with the current form of the capability tree of the slave window, the capability tree of the slave window is updated according to the target capability tree form.

5. The method according to claim 1, characterized in that After executing the video operation action sequence, the method further includes: According to the type of each video operation action in the video operation action sequence, when it is determined that the video operation action belongs to the first type, modifying the execution state of the node corresponding to the video operation action to executed; When it is determined that the video operation action belongs to the second type, the execution state of the node corresponding to the video operation action is maintained as unexecuted.

6. A multi-window video playback system, characterized in that: include: A monitoring module, used for monitoring the video operation actions of the user on the main window when there are a main window and a sub window for playing the video on the user interface; a determination module, configured to, when monitoring a user's video operation action on the main window, execute the video operation action, and determine a capability tree of the slave window, wherein the capability tree includes a root node and at least one leaf node, wherein the root node is used to represent an action of establishing the slave window, and each leaf node is used to represent an operation action performed on the video in the slave window; A control module is used to search, from leaf nodes of the capability tree of the slave window, for a leaf node representing the same action as the video operation action of the user on the main window, when the search is successful, using the found leaf node as the target node, determining the predecessor node of the target node, determining the necessary node according to the execution status of the video operation actions corresponding to the target node and the predecessor node, determining the video operation action corresponding to the necessary node as a video operation action sequence that needs to be executed for the slave window, and executing the video operation action sequence, wherein the predecessor node is a node between the target node and the root node.

7. A terminal device, characterized in that: include: A memory for storing program instructions; A processor, configured to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 5 according to the obtained program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable the computer to execute the method according to any one of claims 1 to 5.

Images