Video playing method, device, system, equipment and medium

Abstract

The embodiment of the invention provides a video playing method, device, system and equipment and a medium, and belongs to the technical field of video playing. A video processing device can obtain at least two videos to be played; fusing the at least two to-be-played videos based on the interested information included in the at least two to-be-played videos to obtain a fused video corresponding to the fused channel; and playing the fusion video in a playing window corresponding to the fusion channel. When watching the to-be-played video, the user pays attention to interested information in the to-be-played video. For example, the information of interest may be a person or a vehicle. The fused video can reflect the interested information included in the at least two videos to be played, so that a user can master the interested information in the at least two videos to be played only by watching the fused video without watching the at least two videos to be played, the user can be prevented from missing the information, and meanwhile, the visual fatigue of the user is reduced.

Description

Technical Field

[0001] The present application relates to the field of video playback technology, and in particular to a video playback method, device, system, equipment and medium. Background Art

[0002] In some scenarios, users need to view multiple videos at the same time. To display each video, the following two methods are usually used: The first method is split-screen playback, which requires dividing a screen into a corresponding number of playback windows according to the number of videos to be viewed, and displaying one video in each playback window. The second method is polling playback, which requires switching and displaying each video in a playback window at a certain time interval.

[0003] In the first method, since there are many channels, each playback window is relatively small, making it difficult for users to see the details of each video and prone to visual fatigue. In the second method, since only one video is displayed in the same time period, it is easy for users to miss key information of other videos. In addition, the video screens are constantly switched, which is also easy to cause visual fatigue to users. In summary, the current video playback method will cause users to miss information and cause visual fatigue. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a video playback method, device, system, equipment and medium to prevent users from missing information and reduce visual fatigue of users. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a video playback method, the method comprising:

[0006] Acquire at least two videos to be played, wherein the videos to be played include recorded videos and / or real-time videos;

[0007] Based on the information of interest included in at least two videos to be played, the at least two videos to be played are fused to obtain a fused video corresponding to a fusion channel;

[0008] The fused video is played in the playback window corresponding to the fused channel.

[0009] Optionally, the step of fusing the at least two videos to be played based on the information of interest included in the at least two videos to be played to obtain a fused video corresponding to a fused channel includes:

[0010] For the at least two videos to be played, extracting information of interest in each of the videos to be played, wherein the information of interest belongs to foreground information of the videos to be played;

[0011] The extracted information of interest is superimposed on the preset background to obtain a fused video.

[0012] Optionally, after the step of superimposing and displaying the extracted information of interest on a preset background, the method further comprises:

[0013] Within a preset range near each information of interest, an identifier corresponding to the information of interest is displayed, wherein the identifier corresponding to the information of interest includes a channel identifier of a channel for transmitting a video frame to which the information of interest belongs, time information of the video frame to which the information of interest belongs, and at least one of category information of the information of interest.

[0014] Optionally, after the step of playing the fused video in the playback window corresponding to the fused channel, the method further includes:

[0015] In response to a first switching operation issued by a user in the playback window corresponding to the fused channel, determining a target channel to be viewed by the user according to a triggering position of the first switching operation;

[0016] Play the video image of the target channel.

[0017] Optionally, the step of playing the video image of the target channel includes:

[0018] In the fused video, removing the information of interest corresponding to other channels except the target channel in the at least two channels;

[0019] The preset background is replaced by the video background of the to-be-played video corresponding to the target channel;

[0020] In the replaced video background, other foreground information except the information of interest in the foreground information corresponding to the target channel is superimposed and displayed.

[0021] Optionally, the step of determining the target channel that the user wants to view according to the trigger position of the first switching operation includes:

[0022] In the to-be-triggered area included in the fused video, determining a target to-be-triggered area including a trigger position of the first switching operation, wherein the to-be-triggered area is an area where the information of interest is located and / or an area where the channel identifier is located;

[0023] The channel corresponding to the to-be-played video corresponding to the target to-be-triggered area is used as the target channel.

[0024] Optionally, after the step of superimposing and displaying other foreground information except the information of interest in the foreground information corresponding to the target channel in the replaced video background, the method further includes:

[0025] In response to a second switching operation issued by the user in the playback window corresponding to the fusion channel, replacing the video background of the to-be-played video corresponding to the target channel with the preset background;

[0026] In the to-be-played video corresponding to the target channel, removing the other foreground information;

[0027] In the preset background, the information of interest corresponding to the other channels of the at least two channels except the target channel is superimposed and displayed.

[0028] Optionally, the step of superimposing and displaying the extracted information of interest on a preset background includes:

[0029] In a preset background, the information of interest included in the video frames of the to-be-played videos corresponding to at least two channels is superimposed and displayed, wherein each channel is used to transmit the to-be-played video captured by a video capture device; or,

[0030] In the preset background, the interesting information included in the video frames collected at different times in the video to be played corresponding to one channel is superimposed and displayed.

[0031] Optionally, the step of superimposing and displaying the information of interest included in the video frames of the to-be-played videos corresponding to at least two channels includes:

[0032] The information of interest included in each video frame acquired at the same time in the videos to be played corresponding to at least two channels is superimposed and displayed.

[0033] Optionally, the step of superimposing and displaying the information of interest included in the video frames of the to-be-played videos corresponding to at least two channels in the preset background includes:

[0034] In the at least two videos to be played, determining the position of the information of interest included in each video frame captured at the same time in the video frame to which the information of interest belongs;

[0035] Determine, according to the position and ratio of the information of interest in the video frame to which the information of interest belongs, a display position corresponding to the position of the information of interest in the video frame to which the information of interest belongs in the playback window, wherein the ratio includes a scaling ratio and / or a rotation ratio, and the scaling ratio is a scaling ratio between the size of the video frame to which the information of interest belongs and the size of the playback window corresponding to the fusion channel;

[0036] According to the preset resolution corresponding to the playback window, the information of interest is superimposed and displayed at a display position corresponding to the information of interest.

[0037] Optionally, the step of playing the fused video in the playback window corresponding to the fused channel includes:

[0038] Play the fused video in a main play window of a play interface, wherein the play interface includes the main play window and at least one auxiliary play window;

[0039] In the at least one auxiliary playback window, each video to be played is played.

[0040] Optionally, when the videos to be played are the videos to be played corresponding to the channels, the arrangement order of the auxiliary play windows for playing the videos to be played in the play interface is determined based on at least one of the following methods:

[0041] Determined based on a preset priority order of each channel;

[0042] Determining based on the number of times the to-be-played video corresponding to each channel is played in the main playback window;

[0043] Determining based on the last playback time of the to-be-played video corresponding to each channel in the main playback window;

[0044] In the case where the videos to be played are videos to be played in different time periods corresponding to a channel, the arrangement order of the auxiliary playback windows used to play the videos to be played in the playback interface is determined based on the arrangement order of the time periods corresponding to the videos to be played played by each auxiliary playback window.

[0045] Optionally, the display areas of the plurality of pieces of interesting information in the play window have overlapping areas;

[0046] The step of playing the fused video in the playback window corresponding to the fused channel comprises:

[0047] Displaying each piece of information of interest in the playback window corresponding to the fusion channel according to the preset priority order of the channel corresponding to each piece of information of interest, wherein the corresponding information of interest with a high priority is located above the corresponding information of interest with a low priority; or,

[0048] According to a preset transparency setting method, the transparency corresponding to each piece of information of interest is determined, and according to the determined transparency, each piece of information of interest is displayed.

[0049] Optionally, the video to be played is a playback video, and the playback interface includes a playback timeline, and the playback timeline corresponds to a statistical graph of events of interest;

[0050] The method further comprises:

[0051] In response to a selection operation issued by a user on the playback timeline or the focus event statistics chart, jumping to a time point corresponding to the selection operation;

[0052] Starting from the time point, the fused video is played.

[0053] Optionally, the event statistics graph is at least one of a bar graph, a wave graph, a line graph, a column graph, a peak graph, and an area graph, and the height or color depth of the quantity identifier corresponding to each time point of the event statistics graph is determined based on one of the following:

[0054] The sum of the number of events of interest occurring at each time point in the to-be-played video corresponding to the multiple channels of the fused video is obtained by fusion;

[0055] The sum of the number of attention events corresponding to the confidence levels that meet preset confidence requirements among the attention events occurring at various time points in the to-be-played video corresponding to the multiple channels of the fused video is obtained by fusion.

[0056] Optionally, the method further includes:

[0057] In the attention event statistics graph, the quantity identifiers corresponding to the to-be-played videos corresponding to different channels are represented by different colors.

[0058] Optionally, the priority is determined based on the importance of a scene captured by a video acquisition device corresponding to the channel, or the priority is determined based on the number of events of interest included in the video to be played transmitted by the channel.

[0059] Optionally, the display device for playing the video to be played includes multiple display screens;

[0060] Before the step of obtaining at least two videos to be played, the method further includes:

[0061] Obtaining a split-screen display position set by a user, wherein the split-screen display position represents a first display screen for playing the fused video and a second display screen for playing each video to be played;

[0062] The step of playing the fused video in the playback window corresponding to the fused channel comprises:

[0063] Play the fused video in the play window of the first display screen;

[0064] In the multiple play windows of the second display screen, each video to be played is played respectively.

[0065] In a second aspect, an embodiment of the present application provides a video playback device, the device comprising:

[0066] A video acquisition module, used to acquire at least two videos to be played, wherein the videos to be played include recorded videos and / or real-time videos;

[0067] A video fusion module, configured to fuse the at least two videos to be played based on the information of interest included in the at least two videos to be played, to obtain a fused video corresponding to a fusion channel;

[0068] The first video playing module is used to play the fused video in the playing window corresponding to the fused channel.

[0069] Optionally, the video fusion module includes:

[0070] An information of interest determination submodule, used for extracting information of interest from each of the at least two videos to be played, wherein the information of interest belongs to foreground information of the video to be played;

[0071] The first interesting information display submodule is used to overlay and display the extracted interesting information on a preset background to obtain a fused video.

[0072] Optionally, the device further comprises:

[0073] An identification display module is used to display an identification corresponding to each piece of information of interest within a preset range near the information of interest, wherein the identification corresponding to the information of interest includes at least one of a channel identification of a channel for transmitting a video frame to which the information of interest belongs, time information of the video frame to which the information of interest belongs, and category information of the information of interest.

[0074] Optionally, the device further comprises:

[0075] a target channel determination module, configured to respond to a first switching operation issued by a user in a playback window corresponding to the fused channel and determine a target channel to be viewed by the user according to a triggering position of the first switching operation;

[0076] The target channel playing module is used to play the video screen of the target channel.

[0077] Optionally, the target channel playing module includes:

[0078] An information of interest removal module, used to remove information of interest corresponding to other channels except the target channel in the at least two channels in the fused video;

[0079] A first background replacement module, used to replace the preset background with the video background of the to-be-played video corresponding to the target channel;

[0080] The foreground information display module is used to overlay and display other foreground information except the information of interest in the foreground information corresponding to the target channel in the replaced video background.

[0081] Optionally, the target channel determination module includes:

[0082] A target to-be-triggered area determination submodule, used to determine a target to-be-triggered area including a trigger position of the first switching operation in the to-be-triggered area included in the fused video, wherein the to-be-triggered area is an area where the information of interest is located and / or an area where the channel identifier is located;

[0083] The target channel determination submodule is used to use the channel corresponding to the to-be-played video corresponding to the target to-be-triggered area as the target channel.

[0084] Optionally, the device further comprises:

[0085] A second background replacement module, configured to replace the video background of the to-be-played video corresponding to the target channel with the preset background in response to a second switching operation issued by the user in the playback window corresponding to the fusion channel;

[0086] A foreground information removal module, used to remove the other foreground information in the to-be-played video corresponding to the target channel;

[0087] The interesting information display module is used to superimpose and display the interesting information corresponding to the other channels of the at least two channels except the target channel in the preset background.

[0088] Optionally, the first interesting information display submodule includes:

[0089] A first information-of-interest display unit is used to overlay and display information of interest included in video frames of to-be-played videos corresponding to at least two channels in a preset background, wherein each channel is used to transmit a to-be-played video captured by a video capture device;

[0090] The second interesting information display unit is used to overlay and display the interesting information included in the video frames collected at different times in the video to be played corresponding to one channel in a preset background.

[0091] Optionally, the first interesting information display unit includes:

[0092] A first position determination subunit is used to determine, in the at least two videos to be played, the position of the information of interest included in each video frame acquired at the same time in the video frame to which the information of interest belongs;

[0093] A second position determination subunit is used to determine the display position of the information of interest in the video frame to which the information of interest belongs, corresponding to the position of the information of interest in the playback window, according to the position and ratio of the information of interest in the video frame to which the information of interest belongs, wherein the ratio includes a scaling ratio and / or a rotation ratio, and the scaling ratio is a scaling ratio between the size of the video frame to which the information of interest belongs and the size of the playback window corresponding to the fusion channel;

[0094] The second information-of-interest display subunit is used to overlay and display the information of interest at a display position corresponding to the information of interest according to a preset resolution corresponding to the playback window.

[0095] Optionally, the first video playback module includes:

[0096] A first fused video playback submodule, configured to play the fused video in a main playback window of a playback interface, wherein the playback interface includes the main playback window and at least one auxiliary playback window;

[0097] The first to-be-played video playing submodule is configured to play each to-be-played video in the at least one auxiliary playing window.

[0098] Optionally, when the videos to be played are the videos to be played corresponding to the channels, the arrangement order of the auxiliary play windows for playing the videos to be played in the play interface is determined based on at least one of the following methods:

[0099] Determined based on a preset priority order of each channel;

[0100] Determining based on the number of times the to-be-played video corresponding to each channel is played in the main playback window;

[0101] Determining based on the last playback time of the to-be-played video corresponding to each channel in the main playback window;

[0102] In the case where the videos to be played are videos to be played in different time periods corresponding to a channel, the arrangement order of the auxiliary playback windows used to play the videos to be played in the playback interface is determined based on the arrangement order of the time periods corresponding to the videos to be played played by each auxiliary playback window.

[0103] Optionally, the display areas of the plurality of pieces of interesting information in the play window have overlapping areas;

[0104] The first video playback module includes:

[0105] The second interest information display submodule is used to display each interest information in the playback window corresponding to the fusion channel according to the preset priority order of the channel corresponding to each interest information, wherein the corresponding interest information with a high priority is located on the upper layer of the corresponding interest information with a low priority;

[0106] The third interest information display submodule is used to determine the transparency corresponding to each interest information according to a preset transparency setting method, and display each interest information according to the determined transparency.

[0107] Optionally, the video to be played is a playback video, and the playback interface includes a playback timeline, and the playback timeline corresponds to a statistical graph of events of interest.

[0108] Optionally, the device further comprises:

[0109] A time point jump module, for responding to a selection operation issued by a user on the playback timeline or the focus event statistical graph, and jumping to a time point corresponding to the selection operation;

[0110] The second video playing module is used to play the fused video starting from the time point, wherein the fused video is obtained by fusing the playback videos corresponding to a preset number of channels with the highest priority.

[0111] Optionally, the event statistics graph is at least one of a bar graph, a wave graph, a line graph, a column graph, a peak graph, and an area graph, and the height or color depth of the quantity identifier corresponding to each time point of the event statistics graph is determined based on one of the following:

[0112] The sum of the number of events of interest occurring at each time point in the to-be-played video corresponding to the multiple channels of the fused video is obtained by fusion;

[0113] The sum of the number of attention events corresponding to the confidence levels that meet preset confidence requirements among the attention events occurring at various time points in the to-be-played video corresponding to the multiple channels of the fused video is obtained by fusion.

[0114] Optionally, in the attention event statistics graph, quantity identifiers corresponding to the to-be-played videos corresponding to different channels are represented by different colors.

[0115] Optionally, the priority is determined based on the importance of a scene captured by a video acquisition device corresponding to the channel, or the priority is determined based on the number of events of interest included in the video to be played transmitted by the channel.

[0116] Optionally, the display device for playing the video to be played includes multiple display screens;

[0117] The device also includes:

[0118] A split-screen display position acquisition module, used to acquire a split-screen display position set by a user, wherein the split-screen display position represents a first display screen for playing the fused video and a second display screen for playing each video to be played;

[0119] The first video playback module includes:

[0120] A second fused video playing submodule, used for playing the fused video in the playing window of the first display screen;

[0121] The second to-be-played video playing submodule is used to play each to-be-played video in a plurality of playing windows of the second display screen respectively.

[0122] In a third aspect, an embodiment of the present application provides a video playback system, the video playback system comprising a video processing device and a display device, wherein:

[0123] The video processing device is used to obtain at least two videos to be played; based on the information of interest included in the at least two videos to be played, the at least two videos to be played are merged to obtain a fused video corresponding to a fused channel; and the fused video is played in a playback window corresponding to the fused channel in the display device, wherein the videos to be played include recorded videos and / or real-time videos.

[0124] In a fourth aspect, an embodiment of the present application provides a video processing device, including:

[0125] Memory, used to store computer programs;

[0126] The processor is used to implement any method described in the first aspect when executing a program stored in the memory.

[0127] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any method described in the first aspect is implemented.

[0128] Beneficial effects of the embodiments of the present application:

[0129] In the solution provided by the embodiment of the present application, the video processing device can obtain at least two videos to be played, wherein the videos to be played include recorded videos and / or real-time videos; based on the information of interest included in the at least two videos to be played, the at least two videos to be played are fused to obtain a fused video corresponding to the fused channel; and the fused video is played in the playback window corresponding to the fused channel. When watching the video to be played, the user pays attention to the information of interest in the video to be played. For example, the information of interest can be a person or a vehicle. Since the fused video can reflect the information of interest included in the at least two videos to be played, and the information of interest is the content that the user pays attention to in the video to be played, the user only needs to watch the fused video to grasp the information of interest in the at least two videos to be played, without watching at least two videos to be played. Therefore, it is possible to reduce the user's visual fatigue while avoiding the user from missing information. Of course, it is not necessary to achieve all the advantages described above at the same time for any product or method of implementing the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0130] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0131] Figure 1 A flowchart of a video playback method provided in an embodiment of the present application;

[0132] Figure 2 Based on Figure 1 A schematic diagram of a playback system of the illustrated embodiment;

[0133] Figure 3 (a) is based on Figure 1 A schematic diagram of a video 1 to be played in the illustrated embodiment;

[0134] Figure 3(b) is based on Figure 1 A schematic diagram of a video 2 to be played in the illustrated embodiment;

[0135] Figure 3 (c) is based on Figure 1 A schematic diagram of a video 3 to be played in the illustrated embodiment;

[0136] Figure 3(d) is based on Figure 1 A schematic diagram of a video 4 to be played in the illustrated embodiment;

[0137] Figure 3(e) is based on Figure 1 A schematic diagram of a video 5 to be played in the illustrated embodiment;

[0138] Figure 3(f) is based on Figure 1 A schematic diagram of a video 6 to be played in the illustrated embodiment;

[0139] Figure 3(g) is based on Figure 1 A first schematic diagram of the fused video of the illustrated embodiment;

[0140] Figure 4 for Figure 1 A specific flow chart of step S102 in the illustrated embodiment;

[0141] Figure 5 Based on Figure 1 A second schematic diagram of the fused video of the illustrated embodiment;

[0142] Figure 6 Based on Figure 1 A flow chart showing a video to be played corresponding to a target channel in the embodiment shown;

[0143] Figure 7 for Figure 6 A specific flow chart of step S602 in the illustrated embodiment;

[0144] Figure 8 (a) is based on Figure 1 A first schematic diagram of the refreshing process of the play window in the illustrated embodiment;

[0145] Figure 8(b) is based on Figure 1 A second schematic diagram of the refreshing process of the playback window in the illustrated embodiment;

[0146] Figure 8 (c) is based on Figure 1 A third schematic diagram of the refreshing process of the playback window in the illustrated embodiment;

[0147] Figure 8(d) is based on Figure 1 A fourth schematic diagram of the refreshing process of the playback window of the illustrated embodiment;

[0148] Fig. 9 for Figure 6 A specific flow chart of step S601 in the illustrated embodiment;

[0149] Fig.10 Based on Figure 1 A flow chart of displaying fused video in the illustrated embodiment;

[0150] Figure 11(a) is based on Figure 1 A fifth schematic diagram of the refreshing process of the play window in the illustrated embodiment;

[0151] Figure 11(b) is based on Figure 1 A sixth schematic diagram of the refreshing process of the playback window of the illustrated embodiment;

[0152] Figure 11(c) is based on Figure 1 A seventh schematic diagram of the refreshing process of the playback window in the illustrated embodiment;

[0153] Fig.12 Based on Figure 1 A flow chart of superimposed display of information of interest in the illustrated embodiment;

[0154] Fig.13 for Figure 1 A specific flow chart of step S103 in the illustrated embodiment;

[0155] Fig.14 Based on Figure 1 A first schematic diagram of the playback interface of the illustrated embodiment;

[0156] Figure 15(a) is based on Figure 1 A second schematic diagram of the playback interface of the illustrated embodiment;

[0157] Figure 15(b) is based on Figure 1 A third schematic diagram of the playback interface of the illustrated embodiment;

[0158] Fig.16 Based on Figure 1 A fourth schematic diagram of the playback interface of the illustrated embodiment;

[0159] Figure 17 (a) is based on Figure 1 A fifth schematic diagram of the playback interface of the illustrated embodiment;

[0160] Figure 17(b) is based on Figure 1 A sixth schematic diagram of the playback interface of the illustrated embodiment;

[0161] Fig.18 Based on Figure 1 A flow chart of playing fused video in the illustrated embodiment;

[0162] Fig.19 Based on Figure 1 The embodiment shown includes a playback interface of a statistical graph of events of interest;

[0163] Fig. 20 Based on Figure 1 A flow chart of a video playback method of the illustrated embodiment;

[0164] Fig.21 for Fig. 20 A specific flow chart of step S2001 in the illustrated embodiment;

[0165] Fig. 22 for Fig. 20 A specific flow chart of step S2002 in the illustrated embodiment;

[0166] Fig.23 for Fig. 22 A specific flow chart of step S2202 in the illustrated embodiment;

[0167] Fig.24 for Fig. 20 A specific flow chart of step S2004 in the illustrated embodiment;

[0168] Fig.25 for Fig. 20 Another specific flow chart of step S2004 in the illustrated embodiment;

[0169] Fig.26 A schematic diagram of the structure of a video playback device provided in an embodiment of the present application;

[0170] Fig. 27 A schematic diagram of the structure of a video playback system provided in an embodiment of the present application;

[0171] Fig.28 A schematic diagram of the structure of a video processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0172] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0173] In order to prevent users from missing information and reduce visual fatigue, the embodiments of the present application provide a video playback method, device, system, video processing equipment, computer-readable storage medium and computer program product. The following first introduces a video playback method provided by the embodiments of the present application.

[0174] like Figure 1 As shown, a video playing method, the method comprising:

[0175] S101, obtaining at least two videos to be played;

[0176] The video to be played includes recorded video and / or real-time video.

[0177] S102, based on the information of interest included in the at least two videos to be played, fusing the at least two videos to be played to obtain a fused video corresponding to a fusion channel;

[0178] S103: Play the fused video in the playback window corresponding to the fused channel.

[0179] It can be seen that in an embodiment of the present application, the video processing device can obtain at least two videos to be played, wherein the videos to be played include recorded videos and / or real-time videos; based on the information of interest included in the at least two videos to be played, the at least two videos to be played are fused to obtain a fused video corresponding to the fused channel; and the fused video is played in the playback window corresponding to the fused channel. When watching the videos to be played, the user pays attention to the information of interest in the videos to be played. For example, the information of interest may be a person or a vehicle. Since the fused video can reflect the information of interest included in the at least two videos to be played, and the information of interest is the content that the user pays attention to in the videos to be played, the user only needs to watch the fused video to grasp the information of interest in the at least two videos to be played, without having to watch at least two videos to be played. Therefore, it is possible to reduce the user's visual fatigue while avoiding the user from missing information.

[0180] The at least two videos to be played may be videos to be played corresponding to multiple channels, or may be videos to be played in different time periods corresponding to one channel. The following will introduce the two situations respectively:

[0181] In the first case, the at least two videos to be played may be videos to be played corresponding to multiple channels. For example, assuming that a scene is deployed with video acquisition device 1-video acquisition device 3, video acquisition device 1-video acquisition device 3 transmits video 1-video 3 to be played through channel 1-channel 3 respectively.

[0182] The videos to be played corresponding to the multiple channels are acquired through the multi-channel video playback system, which may include a video processing device, multiple video acquisition devices, and a display device. The video processing device may be a network video recorder (NVR), the video acquisition device may be an IP camera (IP camera), and the display device may be a local display screen of the video processing device or a display screen of a remote client, such as a mobile phone display screen, a tablet computer display screen, and a laptop computer display screen, etc.

[0183] For example, a schematic diagram of a playback system may be as follows: Figure 2 As shown, video acquisition devices 1-N can shoot scenes and transmit the captured videos to be played to video processing devices through corresponding channels. Video acquisition devices 1-N can be connected to video processing devices through network cables. Video processing devices can process the videos to be played to obtain fused videos.

[0184] Next, the video processing device can play the fused video on the local display screen, which can be connected to the video processing device through an HDMI (High Definition Multimedia Interface) cable. The video processing device can also play the fused video on the display screen of a remote client, which can be connected to the video processing device through a network cable.

[0185] In the second case, the at least two videos to be played may be videos to be played in different time periods corresponding to one channel. Different time periods refer to that the start time and end time of each time period are not exactly the same. For example, assuming that a video acquisition device 4 is deployed in a certain scene, and the video acquisition device 4 transmits a video through channel 4, and the time period corresponding to the video is 6:00-12:00, then the video can be divided to obtain video 1 to be played from 6:00-8:00, video 2 to be played from 8:00-10:00, and video 3 to be played from 10:00-12:00.

[0186] In step S101, the video processing device may obtain at least two videos to be played. The videos to be played may include recorded videos and / or real-time videos. The real-time videos may be videos captured in real time, and the recorded videos may be replayed videos. The following will be divided into three situations to introduce the videos to be played:

[0187] In the first case, the video to be played may be a recorded video, that is, the video to be played may be a video that was previously captured by the video capture device. In the second case, the video to be played may be a real-time video, that is, the video to be played may be a video that is currently being captured by the video capture device. In the third case, the video to be played may be a recorded video and a real-time video, that is, the video to be played may include the video that was previously captured by the video capture device and the video that is currently being captured.

[0188] When a user watches a video to be played, he actually pays attention to the information of interest in the video to be played. The information of interest is the information that the user pays attention to in the video to be played. For example, if the user pays attention to the characters in the video to be played, the information of interest can be the characters. If the user pays attention to the vehicles in the video to be played, the information of interest can be the vehicles.

[0189] Therefore, the video processing device can fuse the at least two videos to be played based on the information of interest included in the at least two videos to be played to obtain a fused video corresponding to the fusion channel, that is, execute step S102. The at least two videos to be played can be determined from all videos to be played according to the actual viewing needs of the user.

[0190] For example, the schematic diagrams of the to-be-played videos 1 to 6 corresponding to channels 1 to 6 can be shown in Figures 3 (a) to 3 (f), respectively. The information of interest included in the to-be-played videos 1 to 5 are the first person 301 to the fifth person 305, respectively, and the to-be-played video 6 does not include the information of interest. In the actual application process, the number of to-be-played videos that do not include the information of interest may be large, and Figure 3 (f) is used as an example here. Since only the to-be-played videos including the information of interest need to be fused, the video processing device can fuse the to-be-played videos 1 to 5 based on the information of interest included in the to-be-played videos 1 to 5, and then obtain the fused video shown in Figure 3 (g).

[0191] For another example, assuming that the video processing device obtains videos 1 to 5 to be played, and the user needs to watch videos 2 and 4 to be played, then based on the information of interest included in videos 2 and 4, videos 2 and 4 can be fused to obtain a fused video.

[0192] For another example, assuming that the total amount of interesting information appearing in all the videos to be played is small, then all the videos to be played can be fused based on the interesting information included in all the videos to be played to obtain a fused video. In this way, the user can watch the interesting information included in all the videos to be played through the fused video.

[0193] Next, the video processing device can play the fused video in the playback window corresponding to the fused channel, that is, execute step S103. Specifically, the playback can be performed in one of the following two implementations: In one implementation, the video processing device can transmit the fused video to a local display screen via an HDMI cable, and play the fused video in the playback window corresponding to the fused channel on the local display screen.

[0194] In another embodiment, the video processing device can transmit the fused video via a network cable, and play the fused video in a playback window corresponding to the fused channel on a display screen of a remote client. If each channel is used to transmit a video to be played, then multiple channels can be converted into one fused channel. During the transmission process, only the fused video corresponding to the fused channel needs to be transmitted, and the videos to be played corresponding to the multiple channels do not need to be transmitted. On the basis of the above-mentioned beneficial effects, bandwidth consumption can also be reduced when the fused video is transmitted via a network.

[0195] It can be seen that in the embodiment of the present application, when the video to be played is a video collected in real time, video preview can be realized, and when the video to be played is a playback video, viewing of the playback video can be realized, so it can be applicable to a variety of video playback scenarios. Since the fused video can reflect the information of interest included in at least two videos to be played, and the information of interest is the content that the user pays attention to in the video to be played, the user only needs to watch the fused video to grasp the information of interest in at least two videos to be played, and does not need to watch at least two videos to be played. In the case of a large number of videos to be played, multiple videos to be played can be fused based on the information of interest included in multiple videos to be played to obtain a fused video. If a video to be played has information of interest, the information of interest will be intuitively reflected in the fused video, and the user can easily know it by watching the fused video. Therefore, it can avoid users from missing information, reduce users' visual fatigue, and reduce users' viewing burden.

[0196] As an implementation method of the present application, Figure 4 As shown, the above step of fusing the at least two videos to be played based on the information of interest included in the at least two videos to be played to obtain a fused video corresponding to the fused channel may include:

[0197] S401, extracting interesting information from each of the at least two videos to be played;

[0198] The video to be played may include foreground information and background information. The background information is usually static, and the foreground information is usually dynamic. For example, the background information may be buildings, trees, road signs, etc., and the foreground information may be people, vehicles, animals, etc.

[0199] The interesting information that the user wants to watch is often the foreground information in the video to be played, so the video processing device can extract the interesting information in each of the at least two videos to be played, that is, perform the above step S401. Specifically, the video processing device can extract the foreground information in the video frames of the at least two videos to be played, and determine the preset category of interesting information from the foreground information.

[0200] The preset categories can be set according to the actual needs of the user, and can be one or more, which is not specifically limited here. For example, assuming that the categories of foreground information include people, vehicles, and animals, then if the user wants to view people, the preset category is people; if the user wants to view vehicles, the preset category is vehicles; if the user wants to view people and vehicles at the same time, the preset category is people and vehicles.

[0201] S402, superimposing and displaying the extracted information of interest on a preset background to obtain a fused video.

[0202] Since the user is concerned about the information of interest in the video to be played, rather than the background information in the video to be played, in order to weaken the background information and highlight the information of interest, the video processing device can superimpose and display the extracted information of interest on the preset background to obtain a fused video, that is, perform the above step S402. Specifically, the video processing device can superimpose and display the information of interest corresponding to each video frame on the preset background to obtain a fused video.

[0203] The preset background may be a pure color, for example, pure white, pure gray, pure black, etc. The preset background may also be composed of multiple color blocks, which is not specifically limited here. For example, in the fused video shown in FIG3 (g), the preset background is pure white.

[0204] In the case where at least two videos to be played are videos to be played corresponding to multiple channels, in order to ensure that multiple pieces of information of interest corresponding to the same frame in the fused video are collected at the same time, in one embodiment, the video processing device can superimpose and display the information of interest included in each video frame collected at the same time in a preset background.

[0205] For example, assuming that the video 1 to be played, the video 3 to be played, and the video 5 to be played are to be merged, the foreground information in the video 1 to be played, the video 3 to be played, and the video 5 to be played can be extracted. If the preset category of the foreground information that the user wants to view is a vehicle, the vehicle can be determined from the extracted foreground information. Next, the video processing device can overlay and display the vehicles included in the video frames collected at the same time in the video 1 to be played, the video 3 to be played, and the video 5 to be played on the preset gray background.

[0206] It can be seen that in the embodiment of the present application, the video processing device can extract the information of interest in each video to be played for at least two videos to be played; and superimpose and display the extracted information of interest in a preset background to obtain a fused video. Usually, the user pays attention to the information of interest corresponding to the video frame, but not the background information in the video frame. Therefore, the video processing device can superimpose and display the information of interest in a preset background to obtain a fused video. In this way, the information of interest can be made more eye-catching, which can further prevent the user from missing the information. In addition, the background of the fused video can be prevented from being too complicated, thereby further reducing the user's visual fatigue.

[0207] As an implementation of an embodiment of the present application, after the step of superimposing and displaying the extracted information of interest on a preset background, the method may further include:

[0208] In a preset range near each piece of information of interest, an identifier corresponding to the video frame to which the information of interest belongs is displayed.

[0209] The preset range may be composed of a position at a preset distance from a point on the outer contour of the information of interest, and the preset distance may be 10, 15, 20 pixels, etc. The point on the outer contour may be located at the top, side or bottom of the information of interest.

[0210] The identifier corresponding to the above-mentioned information of interest may include at least one of the channel identifier of the channel used to transmit the video frame to which the information of interest belongs, the time information of the video frame to which the information of interest belongs, and the category information of the information of interest. The above-mentioned situations will be described separately below:

[0211] As a first implementation, when at least two videos to be played are videos to be played corresponding to multiple channels, in order to intuitively reflect the channel corresponding to the video frame to which each information of interest belongs in the fused video, the video processing device can display the channel identification of the channel corresponding to the video frame to which the information of interest belongs within a preset range near each information of interest, thereby obtaining a fused video including the channel identification. For example, the channel identification can be located above, to the side, or below the information of interest. The above-mentioned channel identification can uniquely identify the channel, and can specifically be numbers, letters, a combination of numbers and letters, etc., which are all reasonable.

[0212] For example, the second schematic diagram of the fused video can be as follows Figure 5 As shown, Figure 5 The channel identification is displayed based on FIG3 (g). The video frames to which the first person 301 to the fifth person 305 belong correspond to channels 1 to 5, respectively. Assuming that the channel identifications corresponding to channels 1 to 5 are CH1 to CH5, respectively, then CH1 to CH5 can be displayed near the first person 301 to the fifth person 305, respectively.

[0213] When the fused video displays channel identification, the user can determine the channel corresponding to the video frame to which each information of interest belongs through the information of interest and the channel identification in the fused video, and then determine the scene to which each information of interest belongs based on the scenes captured by the known video acquisition devices corresponding to each channel, and then judge the safety situation of the scene.

[0214] For example, tourists are prohibited from entering the artificial lake in the park. The artificial lake is shot by the video acquisition device corresponding to channel A, and the fused video includes the information of interest in the video to be played corresponding to channel A, and the information of interest is a person. In this case, once a tourist appears in the fused video and channel A is displayed near the tourist, it means that the tourist has entered the artificial lake, and the user can immediately formulate a corresponding strategy to expel the tourist.

[0215] For another example, a factory includes workshop 1 to workshop 3, the video acquisition devices corresponding to channels 1 to channel 3 are used to shoot workshop 1 to workshop 3 respectively, the channel labels corresponding to channels 1 to channel 3 are workshop 1 to workshop 3 respectively, and the information of interest in the video to be played is people. If these workshops need to have operators at the same time, then the video processing device can fuse the videos to be played corresponding to channels 1 to channel 3 to obtain a fused video.

[0216] If at least three people appear in the fused video at the same time, and the channel logos of workshop 1-workshop 3 exist at the same time, it means that the operator complies with the operating specifications. If the number of people appearing in the fused video at the same time is less than 3, or the channel logos of workshop 1-workshop 3 do not appear at the same time, it means that the operator violates the operating specifications.

[0217] As a second implementation, when at least two videos to be played are videos to be played in different time periods corresponding to one channel, in order to intuitively reflect the time information of the video frame to which each information of interest belongs in the fused video, the video processing device can display the time information of the video frame to which the information of interest belongs within a preset range near each information of interest, thereby obtaining a fused video including the time information. The above time information can be the physical time under the same timing standard, that is, the time information of the video frame to which the information of interest belongs can be the physical time when the video frame is captured.

[0218] For example, assuming that a channel corresponds to video 1 to be played acquired between 8:00 and 10:00, video 2 to be played acquired between 10:00 and 12:00, and video 3 to be played acquired between 12:00 and 14:00, the fused video includes character A in video 1 to be played, character B in video 2 to be played, and character C in video 3 to be played, and the time information corresponding to the video frame to which character A belongs is 8:30, the time information corresponding to the video frame to which character B belongs is 10:30, and the time information corresponding to the video frame to which character C belongs is 12:30, then in the fused video, the time information 8:30, 10:30, and 12:30 can be displayed respectively within a preset range near character A, character B, and character C.

[0219] As a third implementation, the identifier corresponding to the information of interest may be the category information of the information of interest. For example, assuming that the fused video includes information of interest 1 to information of interest 3, and information of interest 1 to information of interest 3 are people, vehicles, and animals, respectively, then in the fused video, the category information "people", "vehicles", and "animals" may be displayed in a preset range near information of interest 1 to information of interest 3, respectively.

[0220] It can be seen that in the embodiment of the present application, the video display device can display the identification corresponding to the information of interest within a preset range near each information of interest. In the case where the displayed identification is a channel identification, the user can determine the channel corresponding to the video frame to which each information of interest belongs by integrating the information of interest and the channel identification in the video, and then determine the scene where each information of interest is located based on the scenes captured by the video acquisition devices corresponding to the known channels, which can facilitate the user to judge the safety of the scene. In the case where the displayed identification is time information, the user can intuitively determine the acquisition time of the video frame to which each information of interest belongs. In the case where the displayed identification is category information, the user can quickly determine the category of each information of interest through the category information. By combining the above-mentioned multiple implementation methods, the user's usage needs in different scenarios can be met.

[0221] As an implementation method of the present application, Figure 6 As shown in FIG. 1 , after the step of playing the fused video in the playback window corresponding to the fused channel, the method may further include:

[0222] S601, in response to a first switching operation issued by a user in a playback window corresponding to the fused channel, determining a target channel to be viewed by the user according to a triggering position of the first switching operation;

[0223] When the playback window corresponding to the fused channel is playing the fused video, if the user wants to view the to-be-played video corresponding to a certain channel, the first switching operation can be issued for the area where the information of interest or the channel identifier is located in the playback window. When the video processing device obtains the first switching operation, it can determine the target channel that the user wants to view according to the trigger position of the first switching operation. The first switching operation can be single-clicking the left mouse button, double-clicking the left mouse button, etc., which is not specifically limited here.

[0224] S602: Play the video image of the target channel.

[0225] After the video processing device determines the target channel that the user wants to view, it can play the video image of the target channel.

[0226] It can be seen that in the embodiment of the present application, the video processing device can respond to the first switching operation issued by the user in the playback window corresponding to the fusion channel, and determine the target channel that the user wants to view according to the trigger position of the first switching operation; and play the video screen of the target channel. In the case where the user wants to view a certain target channel, the user can issue a first switching operation in the playback window corresponding to the fusion channel, and the video processing device can play the video screen of the target channel. In this way, in the case that a certain information of interest in the fusion channel is abnormal, through the above implementation, the video screen can be quickly switched to the target channel.

[0227] As an implementation method of the present application, Figure 7 As shown, the step of playing the video picture of the target channel may include:

[0228] S701, in the fused video, removing information of interest corresponding to other channels in the at least two channels except the target channel;

[0229] In the case where the at least two videos to be played are videos to be played corresponding to at least two channels, after determining the target channel to be viewed by the user, the video processing device may remove the information of interest corresponding to the other channels except the target channel in the at least two channels in the fused video. That is, only the information of interest corresponding to the target channel is retained among the information of interest included in the fused video.

[0230] S702, replacing the preset background with the video background of the to-be-played video corresponding to the target channel;

[0231] Next, the video processing device may replace the preset background of the fused video with the video background of the to-be-played video corresponding to the target channel, wherein the video background of the to-be-played video may be obtained by extracting background information of the to-be-played video corresponding to the target channel.

[0232] S703, superimposing and displaying other foreground information except the information of interest in the foreground information corresponding to the target channel in the replaced video background.

[0233] Since the foreground information corresponding to the target channel may include other foreground information in addition to the information of interest, in order to restore the video to be played corresponding to the target channel, the video processing device can superimpose and display the other foreground information in the foreground information corresponding to the target channel, except the information of interest, on the replaced video background.

[0234] In one embodiment, the other foreground information may be stored in a buffer. In this case, the video processing device may extract the foreground information of the to-be-played video corresponding to the target channel in real time in the background, and store the other foreground information in the extracted foreground information in the buffer.

[0235] For example, it is assumed that the fused video is obtained by fusing the videos to be played corresponding to channels A, C and D based on the information of interest included in the videos to be played corresponding to channels A, C and D. If the trigger position of the first switching operation is the area where the channel identifier "channel C" displayed by the fused video is located, then it can be determined that the target channel to be viewed by the user is channel C.

[0236] Next, the video processing device can remove the information of interest corresponding to channels A and D in the fused video, and retain the information of interest corresponding to channel C. Assuming that the preset background of the fused video is a white background, the video processing device can replace the white background with the video background of the video to be played corresponding to channel C. Assuming that the foreground information corresponding to the target channel C includes vehicles and people, among which only the people are the information of interest, the video processing device can overlay and display the vehicles in the foreground information corresponding to the target channel C on the replaced video background.

[0237] For another example, after the user issues the first switching operation, the schematic diagram of the refreshing process of the playback window may be shown in FIG8 (a) to FIG8 (d). In FIG8 (a), the fused video displayed in the playback window is as follows: Figure 5 As shown, it includes the first person 301 to the fifth person 305, and channel identifiers CH1 to CH5. If the user wants to view the to-be-played video corresponding to channel 3, the user can click the area where the third person 303 is located in the fused video by the left button of the mouse, and the video processing device can determine that the target channel is channel 3.

[0238] Next, the video processing device can remove the information of interest corresponding to channel 1, channel 2, channel 4, and channel 5 in FIG8 (a), that is, remove the first person 301, the second person 302, the fourth person 304, and the fifth person 305, to obtain FIG8 (b). Then, the video processing device can use the video background of the to-be-played video corresponding to channel 3, that is, the two houses, to replace the white background in the fused video, to obtain FIG8 (c). Furthermore, the video processing device can superimpose and display the foreground information corresponding to channel 3, except for the third person 303, that is, the kitten 801, in the replaced video background, to obtain FIG8 (d).

[0239] It can be seen that in the embodiment of the present application, the video processing device can remove the information of interest corresponding to other channels except the target channel in at least two channels in the fused video; use the video background of the video to be played corresponding to the target channel to replace the preset background; in the replaced video background, superimpose and display the foreground information corresponding to the target channel, except for the information of interest. When switching, the video processing device can refresh according to the refresh order of each element, that is, first remove the information of interest corresponding to other channels, then replace the preset background of the fused video, and finally refresh the other foreground information corresponding to the target channel to the playback window for display. Since the elements in the playback window are switched one by one in a certain order, and not all switched at once, it can ensure that the picture is smooth and there will be no jumps, and seamless switching can be achieved, thereby improving the user's viewing experience.

[0240] As an implementation method of the present application, Fig. 9 As shown, the step of determining the target channel to be viewed by the user according to the trigger position of the first switching operation may include:

[0241] S901, determining a target to-be-triggered area including a trigger position of the first switching operation in the to-be-triggered area included in the fused video;

[0242] Since there is a corresponding relationship between the channel and the video to be played, and there is a corresponding relationship between the video to be played and the information of interest and the channel identifier, the channel can be determined by the information of interest and the channel identifier. In the fused video, the area where the information of interest is located and / or the area where the channel identifier is located can be used as the area to be triggered. Specifically, in the first embodiment, the area where the information of interest is located can be used as the area to be triggered. In the second embodiment, the area where the channel identifier is located can be used as the area to be triggered. In the third embodiment, the area where the information of interest is located and the area where the channel identifier is located can be used as the area to be triggered.

[0243] If the first switching operation issued by the user is obtained, the video processing device can determine the target to-be-triggered area including the trigger position of the first switching operation in the to-be-triggered area included in the fused video, that is, execute the above step S901.

[0244] If there is an overlapping area between the multiple areas to be triggered, in order to enable the video processing device to clearly determine the target channel that the user wants to view, the user should issue a first switching operation in other areas except the overlapping area.

[0245] S902: Use the channel corresponding to the to-be-played video corresponding to the target to-be-triggered area as the target channel.

[0246] After determining the target area to be triggered, the video processing device can determine the channel corresponding to the video to be played corresponding to the target area to be triggered as the target channel. Specifically:

[0247] When the target triggering area is the area where the information of interest is located, the video to be played corresponding to the video frame to which the information of interest belongs can be determined, and then the channel corresponding to the video to be played is used as the target channel. When the target triggering area is the area where the channel identifier is located, the channel corresponding to the channel identifier can be used as the target channel.

[0248] For example, assuming that the area where the information of interest is located includes the trigger position of the first switching operation, the area where the information of interest is located can be determined as the target area to be triggered. If the video frame to which the information of interest belongs corresponds to the video to be played corresponding to channel E, the target channel can be determined to be channel E.

[0249] It can be seen that in the embodiment of the present application, the video processing device can determine the target to-be-triggered area including the trigger position of the first switching operation in the to-be-triggered area included in the fused video, wherein the to-be-triggered area is the area where the information of interest is located and / or the area where the channel identifier is located; and the channel corresponding to the to-be-played video corresponding to the target to-be-triggered area is used as the target channel. In this way, when the user wants to view the details of the to-be-played video corresponding to a certain channel, the target channel can be determined quickly and accurately.

[0250] As an implementation method of the present application, Fig.10 As shown, after the step of superimposing and displaying other foreground information except the information of interest in the foreground information corresponding to the target channel in the replaced video background, the method may further include:

[0251] S1001, in response to a second switching operation issued by a user in the playback window corresponding to the fused channel, replacing the video background of the to-be-played video corresponding to the target channel with the preset background;

[0252] When the playback window corresponding to the fused channel is playing a video to be played corresponding to a certain channel, if the user wants to view the fused video, a second switching operation can be issued in the playback window. When the video processing device obtains the second switching operation, the video background of the video to be played corresponding to the target channel can be replaced with a preset background. The second switching operation can be a single click of the right mouse button, a double click of the right mouse button, etc., and the second switching operation is different from the above-mentioned first switching operation.

[0253] S1002, removing the other foreground information from the to-be-played video corresponding to the target channel;

[0254] Since the fused video does not include other foreground information in the to-be-played video corresponding to the target channel, the video processing device can remove other foreground information except the information of interest from the foreground information corresponding to the target channel in the to-be-played video corresponding to the target channel.

[0255] S1003, in the preset background, superimposing and displaying the information of interest corresponding to the other channels among the at least two channels except the target channel.

[0256] At this time, the information of interest in the playback window corresponding to the fused channel is only the information of interest corresponding to the target channel, and does not include the information of interest corresponding to other channels except the target channel. Therefore, the video processing device can superimpose and display the information of interest corresponding to other channels except the target channel in at least two channels in a preset background.

[0257] For example, assuming that the target channel is channel D, the foreground information corresponding to channel D includes people and vehicles, where the vehicle is the information of interest. When the video to be played corresponding to channel D is played in the playback window corresponding to the fused video, if the user issues a second switching operation by clicking the right button of the mouse, the video processing device will switch the background of the video to be played corresponding to channel D to a preset gray background, and then remove the person corresponding to channel D. If the fused video is obtained based on the fusion of channel A, channel C, and the video to be played corresponding to channel D, then next, the video processing device will superimpose and display the vehicles in the foreground information corresponding to channel A and channel C on the gray background, thereby switching from the video to be played corresponding to channel D to the fused video.

[0258] For another example, after the user issues the first switching operation, the schematic diagram of the refreshing process of the playback window may be shown in FIG. 11 (a) to FIG. 11 (c), and the video to be played currently displayed in the playback window may be shown in FIG. 8 (d). If the user issues the second switching operation by clicking the right button of the mouse in FIG. 8 (d), the video processing device will switch the video background of the video to be played corresponding to channel 3 to the preset white background, and obtain the playback interface shown in FIG. 11 (a).

[0259] Next, the video processing device can remove the foreground information other than the third person 303 in FIG11(a), that is, remove the kitten 801, and obtain FIG11(b). Then, the video processing device can overlay and display the information of interest corresponding to channel 1, channel 2, channel 4, and channel 5 and the channel identifier, that is, the first person 301, the second person 302, the fourth person 304, the fifth person 305, and CH1, CH2, CH4, and CH5 on the white background, and obtain FIG11(c).

[0260] It can be seen that in the embodiment of the present application, in response to the second switching operation issued by the user in the playback window corresponding to the fusion channel, the video processing device can replace the video background of the to-be-played video corresponding to the target channel with the preset background; in the to-be-played video corresponding to the target channel, remove other foreground information; in the preset background, superimpose and display the information of interest corresponding to other channels other than the target channel in at least two channels. Through the first switching operation and the second switching operation, the user can switch between the to-be-played video corresponding to the target channel and the fusion video according to actual viewing needs. In this way, a higher degree of freedom can be provided to the user, and the user can switch between the target channel and the fusion channel by mouse, which can meet the user's viewing needs in various situations.

[0261] As an implementation of the embodiment of the present application, the above step of superimposing and displaying the extracted information of interest in a preset background may include one of the following two implementations:

[0262] In a preset background, the information of interest included in the video frames of the videos to be played corresponding to at least two channels is superimposed and displayed; or, in a preset background, the information of interest included in the video frames collected at different times in the video to be played corresponding to one channel is superimposed and displayed.

[0263] In the first embodiment, when at least two videos to be played are videos to be played corresponding to at least two channels, each channel can be used to transmit a video to be played captured by a video capture device. For example, channel 1 to channel 5 can be used to transmit videos to be played captured by video capture devices 1 to 5, respectively.

[0264] Then, the video processing device can overlay and display the information of interest included in the video frames of the to-be-played videos corresponding to at least two channels in the preset background. For example, the video processing device can overlay and display the first person 301 to the fifth person 305 included in the video frames of the to-be-played videos 1 to 5 corresponding to FIG. 3 (a) to FIG. 3 (e) in the preset pure white background.

[0265] In a second embodiment, when at least two videos to be played are videos to be played in different time periods corresponding to a channel, the video processing device can superimpose and display the information of interest included in the video frames collected at different times in the video to be played corresponding to a channel in a preset background.

[0266] For example, assuming that a channel corresponds to video 1 to be played acquired at 12:00-14:00, video 2 to be played acquired at 14:00-16:00, and video 3 to be played acquired at 16:00-18:00, then the video processing device can superimpose and display the information of interest included in the video frame of video 1 to be played acquired at 12:30, the information of interest included in the video frame of video 2 to be played acquired at 14:30, and the information of interest included in the video frame of video 3 to be played acquired at 16:30 in a preset background.

[0267] It can be seen that in the embodiment of the present application, the video processing device can overlay and display the information of interest included in the video frames of the to-be-played videos corresponding to at least two channels in a preset background; or, in the preset background, overlay and display the information of interest included in the video frames collected at different times in the to-be-played videos corresponding to one channel. Through the above two implementations, the video processing device can overlay and display the information of interest included in the video frames of the to-be-played videos in different ways under different circumstances. In this way, the information of interest can be overlaid and displayed quickly and accurately.

[0268] As an implementation of the embodiment of the present application, the above-mentioned step of superimposing and displaying the information of interest included in the video frames of the to-be-played videos corresponding to at least two channels may include:

[0269] The information of interest included in each video frame acquired at the same time in the videos to be played corresponding to at least two channels is superimposed and displayed.

[0270] In the case where at least two videos to be played are videos to be played corresponding to at least two channels, if the information of interest included in the video frames captured at different times is superimposed and displayed, the same information of interest may appear multiple times at the same time in the obtained fused video, making it difficult for the user to determine in which scene the information of interest is actually located. To avoid the above situation, the video processing device can superimpose and display the information of interest included in each video frame captured at the same time in the videos to be played corresponding to at least two channels.

[0271] For example, assuming that the fused video is obtained based on the fusion of video 1 to be played corresponding to channel 1-channel 3, then the video processing device can determine the video frames captured at each moment from video 1 to be played 3, and superimpose and display the information of interest included in the three video frames corresponding to each moment.

[0272] It can be seen that in the embodiment of the present application, the video processing device can overlay and display the information of interest included in each video frame collected at the same time in the video to be played corresponding to at least two channels. In this way, the acquisition time of the video frame to which each information of interest belongs can be synchronized, thereby avoiding the situation where the same information of interest appears multiple times in the same frame of the fused video.

[0273] As an implementation method of the present application, Fig.12 As shown, the above step of superimposing and displaying the information of interest included in the video frames of the to-be-played videos corresponding to at least two channels in the preset background may include:

[0274] S1201, determining, in the at least two videos to be played, a position of information of interest included in each video frame acquired at the same time in the video frame to which the information of interest belongs;

[0275] In order to make the position of the information of interest in the fused video reflect the position of the information of interest in the video to be played, the video processing device can first determine the position of the information of interest included in each video frame acquired at the same time in the at least two videos to be played, in the video frame to which the information of interest belongs. The determined position can be the center position of the information of interest or the position of all pixels of the outer contour.

[0276] S1202, determining, according to the position and proportion of the information of interest in the video frame to which the information of interest belongs, a display position corresponding to the position of the information of interest in the video frame to which the information of interest belongs in the playback window;

[0277] Since there is usually a certain scaling ratio between the size of the video frame to which the information of interest belongs and the size of the playback window corresponding to the fusion channel, and there may be a certain rotation ratio between the angle of the video frame to which the information of interest belongs and the angle of the playback window corresponding to the fusion channel, the video processing device can then determine the display position of the information of interest in the video frame to which the information of interest belongs in the playback window corresponding to the position and ratio of the information of interest in the video frame.

[0278] Wherein, the display position is to map the position of the information of interest in the video frame to the corresponding position in the playback window. The above ratio may include a scaling ratio and / or a rotation ratio. In a first embodiment, the above ratio may include a scaling ratio. In a second embodiment, the above ratio may include a rotation ratio. In a third embodiment, the above ratio may include a scaling ratio and a rotation ratio.

[0279] S1203: Display the information of interest in a superimposed manner at a display position corresponding to the information of interest according to a preset resolution corresponding to the playback window.

[0280] The resolutions of the videos to be played captured by different video capture devices may be different. In order to unify the resolutions, users can pre-set the preset resolutions corresponding to the playback windows corresponding to the fusion channels. For example, the preset resolutions can be 1280×720, 1920×1080, 2560×1440, and so on.

[0281] Then, the video processing device can overlay and display the information of interest at the display position corresponding to the information of interest in the playback window according to the preset resolution. Specifically, the video processing device can adjust the resolution of the information of interest to the preset resolution, and then overlay and display the adjusted information of interest at the display position corresponding to the information of interest in the playback window.

[0282] For example, assuming that the position of a certain information of interest in the video frame is (100,50), and the scaling ratio between the size of the video frame and the size of the playback window corresponding to the fusion channel is 1:2, then it can be determined that the display position corresponding to the above position in the playback window is (200,100). Assuming that the resolution of the information of interest is 1280×720, and the preset resolution corresponding to the playback window is 1920×1080, the video processing device can adjust the resolution of the information of interest to 1920×1080, and then overlay the adjusted information of interest on the display position (200,100) in the playback window.

[0283] It can be seen that in the embodiment of the present application, the video processing device can determine the position of the information of interest included in each video frame captured at the same time in the video frame to which the information of interest belongs in at least two videos to be played; determine the display position corresponding to the position of the information of interest in the video frame to which the information of interest belongs in the playback window according to the position and ratio of the information of interest in the video frame to which the information of interest belongs, wherein the ratio includes a scaling ratio and / or a rotation ratio, and the scaling ratio is the scaling ratio between the size of the video frame to which the information of interest belongs and the size of the playback window corresponding to the fusion channel; and display the information of interest superimposed on the display position corresponding to the information of interest according to the preset resolution corresponding to the playback window. Since the ratio between the display position of the information of interest in the playback window and the position of the information of interest in the video frame is the scaling ratio between the size of the playback window and the size of the video frame, the user can quickly determine the position of the information of interest in the video to be played by the position of the information of interest in the fusion video.

[0284] As an implementation method of the present application, Fig.13As shown, the step of playing the fused video in the playback window corresponding to the fused channel may include:

[0285] S1301, playing the fused video in the main play window of the play interface;

[0286] The playback interface may include a main playback window and at least one auxiliary playback window. The main playback window may be used to play the fused video, and the auxiliary playback window may be used to play each video to be played, that is, the auxiliary playback window may play at least two videos to be played for generating the fused video, and may also play other videos to be played except for the at least two videos to be played for generating the fused video, which is not specifically limited here.

[0287] The number of auxiliary playback windows can be determined according to the number of videos to be played. In the case where at least two videos to be played are videos to be played corresponding to at least two channels, each channel can be used to transmit a video to be played captured by a video acquisition device, then the number of auxiliary playback windows can be determined according to the number of video acquisition devices. In one embodiment, the number of auxiliary playback windows can be the difference between the number of video acquisition devices and a preset subtrahend, and the preset subtrahend can specifically be 1, 2, 3, and so on. For example, assuming that the preset subtrahend is 2 and the number of video acquisition devices is 6, then the number of auxiliary playback windows can be 4. In another embodiment, the number of auxiliary playback windows can be equal to the number of video acquisition devices.

[0288] In the case where at least two videos to be played are videos to be played in different time periods corresponding to one channel, in one embodiment, the number of auxiliary play windows may be the difference between the number of videos to be played and the preset subtrahend. For example, assuming the preset subtrahend is 1 and the number of videos to be played is 5, the number of auxiliary play windows may be 4. In another embodiment, the number of auxiliary play windows may be equal to the number of videos to be played.

[0289] There are multiple arrangements between the main play window and the auxiliary play window. In one arrangement, all auxiliary play windows can be located on the same side of the main play window. For example, they can be located above, to the left, to the right, etc. of the main play window. In another arrangement, the auxiliary play window can be located on the opposite side of the main play window. For example, they can be located on the left and right, or above and below the main play window.

[0290] S1302: Play each video to be played in the at least one auxiliary playback window.

[0291] In order to allow the user to watch the fused video while taking into account the various videos to be played, the video processing device can play each video to be played in at least one auxiliary playback window. The specific playback method can be the same as the playback method in the current related technology, that is, a polling playback method or a split-screen playback method can be used, which is not specifically limited here.

[0292] In addition, when at least two videos to be played are videos to be played corresponding to at least two channels, the auxiliary playback window can display the channel identifier of the channel in its own corner when playing each video to be played. The specific display position can be: lower left corner, lower right corner, upper left corner, upper right corner, etc.

[0293] For example, assuming that the playback interface includes two auxiliary playback windows, namely auxiliary playback window 1 and auxiliary playback window 2, the playback system includes 6 channels, namely channel 1 to channel 6, and the fused video is obtained by fusion based on the to-be-played videos corresponding to channels 1 to channel 4, then the video processing device can use auxiliary playback window 1 and auxiliary playback window 2 to play the to-be-played videos corresponding to any two channels from channel 1 to channel 6. Assuming that auxiliary playback window 1 plays the to-be-played video corresponding to channel 5, and auxiliary playback window 2 plays the to-be-played video corresponding to channel 6, then the lower left corner of auxiliary playback window 1 can display the channel identifier "CH5" corresponding to channel 5, and the lower left corner of auxiliary playback window 2 can display the channel identifier "CH6" corresponding to channel 6.

[0294] In the case where there are multiple auxiliary play windows, and one auxiliary play window is used to play one video to be played, the multiple auxiliary play windows have a certain arrangement order, and the arrangement order can be determined based on the following implementation methods:

[0295] In the case where each to-be-played video is a to-be-played video corresponding to each channel, the arrangement order of the auxiliary play windows for playing each to-be-played video in the play interface may be determined based on at least one of the following three implementations:

[0296] In a first implementation, it can be determined based on the preset priority order of each channel. Specifically, as an implementation, the higher the priority of a channel, the higher the arrangement order of the auxiliary playback window used to play the video to be played corresponding to the channel. As another implementation, the higher the priority of a channel, the lower the arrangement order of the auxiliary playback window used to play the video to be played corresponding to the channel.

[0297] For example, the first schematic diagram of the playback interface can be as follows Fig.14As shown, assuming that the playback system includes channels 1 to 8, the main playback window 1401 is playing a fused video, which is obtained by fusion based on the to-be-played videos corresponding to channels 1 to 5. The first auxiliary playback window 1411 to the third auxiliary playback window 1413 are respectively used to play the to-be-played videos corresponding to channels 6 to 8, and CH6 to CH8 are respectively displayed in the upper left corner.

[0298] Assuming that the priority of channels 6-8 is in the order from high to low: channel 8, channel 7, channel 6, then it can be determined that the arrangement order of the auxiliary playback windows in the playback interface is: third auxiliary playback window 1413, second auxiliary playback window 1412, first auxiliary playback window 1411.

[0299] The priority mentioned in the above embodiment can be determined by one of the following two determination methods:

[0300] In the first determination mode, the priority can be determined based on the importance of the scene captured by the video acquisition device corresponding to the channel. Specifically, the higher the importance of the scene captured by the video acquisition device corresponding to the channel, the higher the priority corresponding to the channel.

[0301] For example, assuming that the video acquisition devices corresponding to channels A to D respectively shoot scenes A to D, and the order of importance of these scenes from high to low is: scene D, scene B, scene C, scene A, then it can be determined that the order of channel priority from high to low is: channel D, channel B, channel C, channel A.

[0302] In the second determination method, the priority can be determined based on the number of events of interest included in the video to be played transmitted by the channel. Specifically, the higher the number of events of interest included in the video to be played transmitted by the channel, the higher the priority corresponding to the channel. The event type of the event of interest is a pre-set event type that the user is interested in, which can be the appearance of a person, the appearance of a vehicle, the appearance of an animal, and so on.

[0303] For example, assuming that the event of interest is the appearance of a person, and the number of people appearing in the videos to be played corresponding to channels A-Channel D are 10, 8, 14, and 2 respectively, then the priority of the channels can be determined in descending order: channel C, channel A, channel B, channel D.

[0304] In a second implementation, the order of the auxiliary playback windows for playing the videos to be played corresponding to each channel in the main playback window can be determined based on the number of times the videos to be played corresponding to each channel are played in the main playback window. If the number of times the videos to be played in the main playback window is high, it means that the importance of the videos to be played is high, so the arrangement order of the auxiliary playback windows for playing the videos to be played in the playback interface can be determined based on the number of times the videos to be played corresponding to each channel are played in the main playback window.

[0305] The number of times the video to be played in the main play window may be the sum of the number of times the channel used to transmit the video to be played is used as the target channel and the number of times the auxiliary play window corresponding to the video to be played receives the trigger operation. The trigger operation may be used to play the video to be played corresponding to the auxiliary play window in the main play window.

[0306] Specifically, as an implementation, the higher the number of times a video to be played corresponding to a certain channel is played in the main playback window, the higher the arrangement order of the auxiliary playback window used to play the video to be played corresponding to the channel. As another implementation, the higher the number of times a video to be played corresponding to a certain channel is played in the main playback window, the higher the arrangement order of the auxiliary playback window used to play the video to be played corresponding to the channel.

[0307] For example, assuming that auxiliary playback window 1-auxiliary playback window 3 are used to play videos 1-video 3 to be played corresponding to channels 1-channel 3 respectively, and the number of times channels 1-channel 3 are used as target channels is 2 times, 0 times and 3 times respectively, and the number of times auxiliary playback window 1-auxiliary playback window 3 receive trigger operations is 3 times, 1 time and 4 times respectively, then the number of times videos 1-video 3 to be played in the main playback window is 5 times, 1 time and 7 times respectively.

[0308] If the number of times the video to be played corresponding to a certain channel is played in the main playback window, the auxiliary playback window used to play the video to be played corresponding to the channel will be arranged in an upper order, then the arrangement order of the auxiliary playback windows in the playback interface will be: auxiliary playback window 3, auxiliary playback window 1 and auxiliary playback window 2.

[0309] In a third implementation, the order of the auxiliary playback windows used to play the videos to be played can be determined based on the last playback time of the videos to be played corresponding to each channel in the main playback window. If the last playback time of the videos to be played in the main playback window is close to the current time, it means that the user has recently viewed the videos to be played in the main playback window, and the importance of the videos to be played is high. Therefore, the arrangement order of the auxiliary playback windows used to play the videos to be played in the playback interface can be determined based on the last playback time of the videos to be played corresponding to each channel in the main playback window.

[0310] Specifically, as an implementation method, the closer the last playback time of a to-be-played video corresponding to a certain channel in the main playback window is to the current time, the earlier the auxiliary playback window used to play the to-be-played video corresponding to the channel is arranged. As another implementation method, the closer the last playback time of a to-be-played video corresponding to a certain channel in the main playback window is to the current time, the later the auxiliary playback window used to play the to-be-played video corresponding to the channel is arranged.

[0311] For example, suppose auxiliary play window 1 to auxiliary play window 3 are used to play videos 1 to 3, respectively, and the last play time of videos 1 to 3 in the main play window is 11:30, 10:00, 8:20, and the current time is 12:00. If the last play time of the video to be played corresponding to a certain channel in the main play window is closer to the current time, the auxiliary play window used to play the video to be played corresponding to the channel is arranged in the front, then the auxiliary play window is arranged in the play interface in the following order: auxiliary play window 1, auxiliary play window 2, auxiliary play window 3.

[0312] In the case where each video to be played is a video to be played in different time periods corresponding to a channel, the arrangement order of the auxiliary playback windows used to play each video to be played in the playback interface is determined based on the arrangement order of the time periods corresponding to the videos to be played played by each auxiliary playback window. Specifically, the arrangement order of the time periods can be determined based on the start time of the time period, or based on the end time of the time period, which is not specifically limited here. For example, the earlier the start time of the time period, the earlier the arrangement order of the time period. For another example, the later the end time of the time period, the earlier the arrangement order of the time period.

[0313] For example, assuming that the time periods corresponding to the to-be-played videos 1 to 3 are 8:00-10:00, 9:00-11:00, and 12:00-13:00, respectively, and the earlier the start time of the time period, the earlier the arrangement order of the time period, and the earlier the arrangement order of the auxiliary play window to which the to-be-played window corresponding to the time period belongs. Then the arrangement order of the auxiliary play windows in the play interface is: auxiliary play window 1, auxiliary play window 2, auxiliary play window 3.

[0314] When the user is watching normally, the size of the main playback window is usually larger than the size of the auxiliary playback window. If the user finds that there is interesting information in the video to be played in a certain auxiliary playback window and wants to focus on the auxiliary playback window, the user can issue a trigger operation to the auxiliary playback window. After obtaining the trigger operation for the auxiliary playback window, the video processing device will swap the position of the auxiliary playback window and the main playback window in the playback interface to increase the size of the auxiliary playback window. Among them, the trigger operation can be a single click of the left mouse button, a double click of the left mouse button, and so on.

[0315] If the user believes that the video to be played in the auxiliary play window does not need special attention, and wants to reduce the size of the auxiliary play window and increase the size of the main play window, the user can issue a trigger operation to the main play window. After obtaining the trigger operation for the main play window, the video processing device will swap the positions of the auxiliary play window and the main play window in the play interface to restore the original layout of the play interface.

[0316] For example, suppose the user wants to zoom in to see Fig.14 15(a), the user can click the third auxiliary play window 1413 with the left mouse button. Next, the video processing device can swap the position of the third auxiliary play window 1413 with the main play window 1401 to obtain the play interface shown in FIG15(b).

[0317] Subsequently, assuming that the user clicks the main play window 1401 in FIG. 15( b) with the left button of the mouse, the video processing device will again swap the positions of the third auxiliary play window 1413 and the main play window 1401, and the play interface will return to the following state: Fig.14 shown.

[0318] The fourth schematic diagram of the playback interface can be Fig.16 As shown, Fig.16 The fused video played by the main play window 1601 is obtained by fusion of the videos to be played corresponding to channels 1 to 5 and 8 based on the information of interest in the videos to be played corresponding to channels 1 to 5 and 8. The first auxiliary play window 1611 to the third auxiliary play window 1613 are used to play the videos to be played corresponding to channels 6 to 8, and CH6 to CH8 are displayed in the upper left corner respectively. Assume that the user wants to zoom in to view Fig.16 17(a), the user can click the third auxiliary play window 1613 with the left mouse button. Next, the video processing device can swap the position of the third auxiliary play window 1613 with the main play window 1601 to obtain the play interface shown in FIG. 17(b).

[0319] It can be seen that in the embodiment of the present application, the video processing device can play the fused video in the main playback window of the playback interface, wherein the playback interface includes a main playback window and at least one auxiliary playback window; in at least one auxiliary playback window, each video to be played is played. Through the method provided in the embodiment of the present application, at least two videos to be played are fused instead of all videos to be played. It is also possible to fuse multiple videos to be played with fewer characters or lower importance. The user only needs to play the fused video in the main playback window to know the situation of these videos to be played, and for videos to be played with more characters or higher importance, they can be viewed separately through the auxiliary playback window. In this way, multiple videos to be played can be played flexibly according to the actual situation of the scene.

[0320] As an implementation of an embodiment of the present application, the display areas of multiple pieces of information of interest in the above-mentioned playback window may have overlapping areas. In this case, in the above-mentioned playback window corresponding to the fusion channel, the step of playing the fusion video may include one of the following two implementations:

[0321] According to the preset priority order of the channel corresponding to each information of interest, each information of interest is displayed in the playback window corresponding to the fusion channel; or, according to the preset transparency setting method, the transparency corresponding to each information of interest is determined, and each information of interest is displayed according to the determined transparency.

[0322] In the first implementation, if the priority of the channel corresponding to the information of interest is higher, it means that the information of interest is more important. Therefore, in the case of overlapping areas, the video processing device can display each information of interest in the playback window corresponding to the fused channel according to the preset priority order of the channel corresponding to each information of interest. Specifically, the corresponding information of interest with a higher priority can be located on the upper layer of the corresponding information of interest with a lower priority.

[0323] For example, as shown in FIG3 (g), since the priorities of channel 2 and channel 4 are both higher than the priority of channel 3, for the overlapping area between the display positions of the second character 302 and the third character 303, the second character 302 is located above the third character 303; for the overlapping area between the display positions of the fourth character 304 and the third character 303, the fourth character 304 is located above the third character 303.

[0324] In the second implementation, in order to enable the user to view each information of interest whose display position is located in the overlapping area, the video processing device may determine the transparency corresponding to each information of interest according to a preset transparency setting method, and then display each information of interest according to the determined transparency. Specifically, the preset transparency setting method may be one of the following setting methods:

[0325] As a first setting mode, the video processing device can set the transparency of each information of interest to the same preset transparency. For example, the preset transparency can be 80%, 70%, 50%, etc., which is not specifically limited here.

[0326] As a second setting method, the video processing device can set the transparency of each information of interest to the preset transparency corresponding to the channel corresponding to the information of interest, and at least two channels correspond to different preset transparency. For example, assuming that the preset transparency corresponding to channel 1-channel 3 is 50%, 65% and 80% respectively, then the transparency of information of interest 1-information of interest 3 corresponding to channel 1-channel 3 can be set to 50%, 65% and 80% respectively.

[0327] As a third setting method, the video processing device can determine the transparency of the information of interest corresponding to each channel according to the priority of each channel. For example, assuming that the transparency of the information of interest corresponding to the channels is 90%, 70% and 40% respectively in descending order of the channel priority, and the order of the channel priority from high to low is channel B, channel C and channel A, then the transparency of the information of interest A-C corresponding to channel A-channel C can be set to 40%, 90% and 70% respectively.

[0328] It can be seen that in the embodiment of the present application, the video processing device can display each information of interest in the playback window corresponding to the fused channel according to the preset priority order of the channel corresponding to each information of interest, wherein the corresponding information of interest with a high priority is located on the upper layer of the corresponding information of interest with a low priority; or, according to the preset transparency setting method, determine the transparency corresponding to each information of interest, and display each information of interest according to the determined transparency. Through the first implementation, the video processing device can determine the information of interest to be displayed on the upper layer in the overlapping area according to the importance of each channel. Through the second implementation, the user can view each information of interest in the overlapping area at the same time, which can avoid occlusion.

[0329] As an implementation of an embodiment of the present application, the video to be played may be a playback video, and the playback interface may include a playback timeline, and the playback timeline may correspond to a statistical graph of events of interest. In this case, Fig.18 As shown, the above method may also include:

[0330] S1801, in response to a selection operation issued by a user on the playback timeline or the focus event statistics chart, jumping to a time point corresponding to the selection operation;

[0331] When the video to be played is a playback video, the method of generating the fused video may be one of the following two implementation modes:

[0332] As an implementation method, the video processing device may fuse playback videos corresponding to a preset number of channels with the highest priority to obtain a fused video. The preset number may be determined based on the total number of video acquisition devices, and the preset number is not greater than the total number of video acquisition devices, and may be 2, 4, 6, etc.

[0333] For example, assuming that the playback system includes channels 1 to 4, the priority of each channel is in the order of channel 3, channel 2, channel 1, channel 4. If the preset number is 2, the playback videos corresponding to channel 3 and channel 2 can be fused to obtain a fused video.

[0334] As another implementation, the video processing device may fuse the playback videos corresponding to all channels to obtain a fused video.

[0335] In the case where the video to be played is a playback video, the video processing device can display the playback timeline and the event statistics corresponding to the playback timeline in the playback interface. The height or color depth of the quantity mark in the above-mentioned event statistics can reflect the number of events of interest that occur in the fused video at each moment. The number of events of interest can be determined by the video processing device performing event recognition on the playback video, or it can be sent to the video processing device after other processing devices perform event recognition on the playback video. It is not specifically limited here.

[0336] The event statistics chart can be at least one of a bar chart, a wave chart, a column chart, a line chart, a peak chart, and an area chart, and is not specifically limited here. The quantity mark can be a straight bar in a bar chart, a wavy line in a wave chart, a rectangle in a column chart, a broken line in a line chart, a semicircle in a peak chart, and a stacked area in an area chart.

[0337] The height or color depth of the quantity mark corresponding to each time point in the attention event statistics graph may be determined based on one of the following two implementations:

[0338] In a first implementation, the total number of events of interest occurring at each time point in the to-be-played video corresponding to the multiple channels used to fuse the fused video can be determined. Specifically, for each time point, the total number of events of interest occurring at that time point in the to-be-played video corresponding to the multiple channels used to fuse the fused video can be determined, and then the height or color depth of the quantity identifier corresponding to the time point can be determined based on the total number of events of interest.

[0339] As an implementation method, the more the total number of attention events corresponding to a certain time point, the higher the number mark corresponding to the time point, or the darker the color depth. For example, the fused video is obtained based on the fusion of the to-be-played videos corresponding to channels 1-3. Taking the time points 12:00 and 12:01 as examples, assuming that the total number of attention events occurring at 12:00 in the to-be-played videos corresponding to channels 1-3 is 6, and the total number of attention events occurring at 12:01 is 10, and the attention event statistics chart is a bar chart, then the number mark corresponding to 12:00 is light blue, the number mark corresponding to 12:01 is dark blue, and the number mark corresponding to 12:01 is higher than the number mark corresponding to 12:00.

[0340] In a second embodiment, the total number of attention events whose corresponding confidences meet the preset confidence requirements in the attention events occurring at various time points in the to-be-played video corresponding to the multiple channels used for fusion to obtain the fused video can be determined. Specifically, for each time point, the attention events occurring at the to-be-played video corresponding to the multiple channels used for fusion to obtain the fused video can be determined, and then the total number of attention events whose corresponding confidences meet the preset confidence requirements can be determined from the determined attention events, and the height or color depth of the quantity identifier corresponding to the time point can be determined based on the total number of attention events. The preset confidence requirement can be greater than a preset confidence threshold, and the preset confidence threshold can be 85%, 90%, 95%, and so on.

[0341] As an implementation method, the more the sum of the number of attention events that meet the preset confidence requirements corresponding to a certain time point, the higher the number mark corresponding to the time point, or the darker the color depth. For example, the fused video is obtained based on the fusion of the to-be-played videos corresponding to channels 4-channel 6. Taking the time points 13:30 and 13:40 as examples, assuming that the sum of the number of attention events that meet the preset confidence requirements in the attention events that occur at 13:30 in the to-be-played videos corresponding to channels 4-channel 6 is 5, and the sum of the number of attention events that meet the preset confidence requirements in the attention events that occur at 13:40 is 14. If the attention event statistics chart is a bar chart, then the number mark corresponding to 13:30 is light red, the number mark corresponding to 13:40 is dark red, and the number mark corresponding to 13:40 is higher than the number mark corresponding to 13:30.

[0342] In one embodiment, in addition to displaying the quantity identification determined based on the above-mentioned total number, the quantity identification corresponding to the video to be played corresponding to the channel at each time point can also be displayed for each channel used for fusion to obtain the fused video, based on the number of attention events occurring at each time point of the video to be played corresponding to the channel. In this case, in order to intuitively reflect the number of attention events occurring at each channel corresponding to the video to be played, different colors can be used to represent the quantity identification corresponding to the videos to be played corresponding to different channels.

[0343] For example, assuming that the fused video is obtained by fusion based on the videos to be played corresponding to channel 1 and channel 2, then in the attention event statistics chart, a gray quantity icon can be displayed based on the total number of attention events occurring at each time point in the videos to be played corresponding to channel 1 and channel 2, a green quantity icon can be displayed based on the number of attention events occurring at each time point in the videos to be played corresponding to channel 1, and a blue quantity icon can be displayed based on the number of attention events occurring at each time point in the videos to be played corresponding to channel 2.

[0344] The attention event statistics graph can be displayed within a preset range near the playback time axis. Specifically, the attention event statistics graph can be displayed above, below, to the side of the playback time axis, etc. For example, when the attention event statistics graph is a bar graph, the video processing device can display the bar graph above the playback time axis of the video to be played. The time point of the playback time axis at the height corresponding to the bar graph can reflect the number of attention events that occurred in the fused video at that time point.

[0345] For another example, when the attention event statistics chart is a line chart, the video processing device can display the line chart on the side of the playback timeline of the video to be played, and the horizontal axis of the line chart can represent the time point of the fused video, and the vertical axis can represent the number of attention events. The vertical axis value corresponding to the time point of the playback timeline in the bar chart is the number of attention events that occurred in the fused video at that time point.

[0346] Since the time point at which the event of interest occurs usually requires special attention from the user, and the event of interest statistics chart can reflect the number of events of interest occurring at each time point in the fused video, the user can issue a selection operation on the playback timeline or the event of interest statistics chart, that is, the user can issue a selection operation on the playback timeline or the event of interest statistics chart for the time point at which a large number of events of interest occur. After the video processing device obtains the selection operation issued by the user, the fused video can be jumped to the time point corresponding to the selection operation. The selection operation can be a single click of the left mouse button, a double click of the left mouse button, a right click of the mouse, and so on.

[0347] S1802, starting from the time point, playing the fused video.

[0348] Next, the video processing device can continue to play the fused video from the time point jumped to. For example, a schematic diagram of the playback interface including the focus event statistics chart can be as follows: Fig.19 As shown, the vertical grid displayed above the playback timeline 1901 is the attention event statistics chart, which can reflect the number of attention events occurring at each moment of the fused video. The higher the attention event statistics chart, the more attention events occurred at the corresponding moment. The user can issue a selection operation at the time point 1902 corresponding to the playback timeline 1901 at the maximum value of the attention event statistics chart based on the attention event statistics chart. Next, the video processing device will cause the fused video to jump to the time point 1902, and start playing the fused video from the time point 1902.

[0349] It can be seen that in the embodiment of the present application, in response to the user's selection operation on the attention event statistics chart or the attention event statistics chart, the video processing device can jump to the time point corresponding to the selection operation; starting from the time point, the fused video is played, wherein the fused video is obtained by fusing the playback videos corresponding to the preset number of channels with the highest priority. By displaying the attention event statistics chart in the playback interface, the number of attention events occurring at each moment can be reflected. In this way, it can assist the user to quickly locate the time point where a large number of attention events occur, and then view the playback video in a targeted manner.

[0350] As an implementation of an embodiment of the present application, a display device for playing the video to be played may include multiple display screens. In this case, before the step of obtaining at least two videos to be played, the method may further include:

[0351] Gets the split-screen display position set by the user.

[0352] In the case where the display device includes multiple display screens, the user can set the split screen display position before the video processing device obtains the videos to be played corresponding to the multiple channels. In this way, the video processing device can obtain the split screen display position set by the user. The split screen display position can be characterized as a display screen used to play the fused video as the first display screen, and a display screen used to play each video to be played as the second display screen.

[0353] Accordingly, the step of playing the fused video in the playback window corresponding to the fused channel may include:

[0354] The fused video is played in the play window of the first display screen; and each video to be played is played separately in multiple play windows of the second display screen.

[0355] Since the user has set the split-screen display position, the video processing device can play each video to be played according to the split-screen display position set by the user when playing the fused video. Specifically, the video processing device can play the fused video in the play window of the first display screen, and play each video to be played in multiple play windows of the second display screen. The play mode corresponding to each video to be played can be a split-screen play mode in the current related technology.

[0356] Among them, each video to be played can be other videos to be played except the at least two videos to be played used to generate the fused video among all the videos to be played, or can be at least two videos to be played used to generate the fused video, which is not specifically limited here. For example, assuming that the playback system includes channels 1-5, and the fused video is obtained based on the fusion of the videos to be played corresponding to channels 1-3, then the fused video can be played through the first display screen, and the videos to be played corresponding to channels 1-5 can be played through the second display screen.

[0357] It can be seen that in the embodiment of the present application, the video processing device can obtain the split-screen display position set by the user, wherein the split-screen display position represents a first display screen for playing the fused video, and a second display screen for playing each other video to be played; then, in the play window of the first display screen, the fused video is played; in multiple play windows of the second display screen, each video to be played is played separately. In the above manner, the user can view the fused video and each video to be played separately on multiple display screens. The role of playing the fused video is to give an alarm prompt for the global information, and the role of playing each video to be played is to display the detailed information of each video to be played. In this way, the user can take into account both the overview of the global information and the specific detailed information.

[0358] As an implementation method of the present application, a flow chart of a video playback method can be as follows: Fig. 20 As shown, the following steps may be specifically included:

[0359] S2001, obtaining user configuration information;

[0360] The video processing device can obtain at least two channels and split-screen display positions configured by the user, wherein the at least two channels are channels that need to be merged for the corresponding videos to be played.

[0361] S2002, generating a fused video;

[0362] After acquiring at least two channels configured by the user, the video processing device may fuse the videos to be played corresponding to the at least two channels based on the information of interest included in the videos to be played corresponding to the at least two channels to obtain a fused video corresponding to the fused channel.

[0363] S2003, determining whether a switching operation is obtained; if yes, executing step S2004; if no, returning to executing step S2003;

[0364] The video processing device may determine whether a switching operation is acquired, wherein the switching operation may include the first switching operation and the second switching operation.

[0365] S2004, switching the video played in the play window.

[0366] If the video processing device obtains the first switching operation, the fused video played in the play window may be switched to the to-be-played video corresponding to the target channel indicated by the first switching operation.

[0367] If the video processing device obtains the second switching operation, the to-be-played video corresponding to the target channel played in the play window may be switched to the fused video.

[0368] The following will describe the above steps S2001, S2002 and S2004 in detail in conjunction with the flow chart:

[0369] like Fig.21 As shown, the step of obtaining the user configuration information, that is, the step S2001, may specifically include:

[0370] S2101, obtaining a fusion start instruction issued by a user;

[0371] When receiving a fusion start instruction from the user, the video processing device can display an information configuration interface to obtain the corresponding channel for the video to be played that needs to be fused, the display screen for playing the fused video, and the playback position on the display screen configured by the user in the information configuration interface.

[0372] S2102, obtaining the channel that needs to be merged for the corresponding to-be-played video configured by the user;

[0373] The video processing device can obtain the channels that need to be merged for the corresponding to-be-played videos configured by the user, that is, the at least two channels mentioned above.

[0374] S2103, obtaining a display screen configured by the user for playing the fused video;

[0375] The video processing device can obtain a display screen configured by the user for playing the fused video. The configured display screen can be a local display screen of the video processing device or a remote display screen that needs to receive the fused video through the network.

[0376] S2104, obtaining the playback position of the fused video configured by the user on the display screen.

[0377] After determining the display screen for playing the fused video, the video processing device can also obtain the playback position of the fused video configured by the user on the display screen. For example, the display interface of the display screen may include a main playback window and an auxiliary playback window, and the above playback position may be the position of the main playback window on the playback interface.

[0378] like Fig. 22 As shown, the step of generating a fused video, i.e., the step S2002, may specifically include:

[0379] S2201, obtaining a preset background;

[0380] In order to weaken the background in the video to be played and highlight the information of interest in the video to be played, the video processing device can obtain a preset background.

[0381] S2202, extracting information of interest corresponding to the channel;

[0382] The video processing device can extract the information of interest corresponding to the channel that needs to be merged with the corresponding video to be played selected by the user.

[0383] S2203, superimposing and displaying the information of interest on the preset background;

[0384] The video processing device can overlay and display the information of interest on a preset background to obtain a fused video corresponding to the fused channel.

[0385] S2204: Play the fused video in the playback window corresponding to the fused channel.

[0386] Next, the video processing device can play the fused video in the playback window corresponding to the fusion channel pre-configured by the user.

[0387] like Fig.23 As shown, the step of extracting the information of interest corresponding to the channel, that is, the step S2202, may specifically include:

[0388] S2301, decoding the video to be played;

[0389] For the channel selected by the user and corresponding to the video to be played that needs to be merged, the video processing device can decode the video to be played corresponding to the channel.

[0390] S2302, determining the location of the information of interest in the video to be played;

[0391] The video processing device can identify the information of interest in the video to be played, extract the outline of the information of interest, and thus determine the location of the information of interest. If the video to be played is a real-time video, the algorithm used in the above steps of identifying the information of interest and extracting the outline of the information of interest needs to have high real-time performance.

[0392] S2303, extracting the determined position to obtain information of interest;

[0393] After determining the location of the information of interest, the video processing device may extract the determined location to obtain the information of interest.

[0394] S2304: convert the information of interest extracted from the multiple videos to be played into the same preset resolution, and obtain the information of interest after the resolution conversion and the channel identifier corresponding to the information of interest.

[0395] The video processing device can convert the information of interest extracted from the multiple videos to be played into the same preset resolution. Next, the video processing device can obtain the information of interest after the resolution conversion and the channel identification corresponding to each information of interest so as to display them in the play window.

[0396] like Fig.24 As shown, the step of switching the video played in the play window, that is, the step S2004, may specifically include:

[0397] S2401, the user issues a first switching operation;

[0398] If the user wants to view the to-be-played video corresponding to the target channel, the user may issue a first switching operation in the playback window.

[0399] S2402, obtaining a trigger position of the first switching operation, and determining a target channel selected by the user;

[0400] The video processing device may obtain a trigger position of the first switching operation, and determine a target channel selected by the user according to the trigger position and the area to be triggered included in the fused video.

[0401] S2403, in the fused video, removing information of interest corresponding to other channels except the target channel in at least two channels;

[0402] The video processing device may remove the information of interest corresponding to the other channels except the target channel in the at least two channels in the fused video, that is, only the information of interest corresponding to the target channel is retained.

[0403] S2404, using the background of the video to be played corresponding to the target channel to replace the preset background;

[0404] Next, the video processing device may extract the background of the video to be played corresponding to the target channel, and use the background of the video to be played to replace the preset background.

[0405] S2405 , in the replaced background, other foreground information except the information of interest in the foreground information corresponding to the target channel is superimposed and displayed.

[0406] In order to restore other foreground information of the target channel, the video processing device may overlay and display other foreground information of the foreground information corresponding to the target channel except the information of interest in the replaced background.

[0407] like Fig.25 As shown, the step of switching the video played in the play window, that is, the step S2004, may specifically include:

[0408] S2501, the user issues a second switching operation;

[0409] If the user wants to view the to-be-played video corresponding to the target channel, the user can issue a second switching operation in the playback window.

[0410] S2502, replacing the background of the video to be played corresponding to the target channel with a preset background;

[0411] After acquiring the second switching operation, the video processing device may replace the background of the to-be-played video corresponding to the target channel with a preset background.

[0412] S2503, extracting information of interest corresponding to other channels except the target channel from at least two channels;

[0413] Next, the video processing device may remove other foreground information from the to-be-played video corresponding to the target channel, and extract information of interest corresponding to other channels except the target channel from at least two channels.

[0414] S2504, superimposing and displaying the information of interest on the preset background;

[0415] The video processing device can overlay and display the information of interest corresponding to the other channels except the target channel in at least two channels on a preset background.

[0416] S2505: Play the fused video in the playback window corresponding to the fused channel.

[0417] Next, the video processing device can play the fused video in the playback window corresponding to the fusion channel pre-configured by the user.

[0418] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0419] Corresponding to the above-mentioned video playing method, the embodiment of the present application further provides a video playing device. The following is an introduction to a video playing device provided by the embodiment of the present application.

[0420] like Fig.26 As shown, a video playback device, the device comprising:

[0421] The video acquisition module 2601 is used to acquire at least two videos to be played, wherein the videos to be played include recorded videos and / or real-time videos;

[0422] The video fusion module 2602 is used to fuse the at least two videos to be played based on the information of interest included in the at least two videos to be played, so as to obtain a fused video corresponding to a fusion channel;

[0423] The first video playing module 2603 is used to play the fused video in the playing window corresponding to the fused channel.

[0424] It can be seen that in an embodiment of the present application, the video processing device can obtain at least two videos to be played, wherein the videos to be played include recorded videos and / or real-time videos; based on the information of interest included in the at least two videos to be played, the at least two videos to be played are fused to obtain a fused video corresponding to the fused channel; and the fused video is played in the playback window corresponding to the fused channel. When watching the videos to be played, the user pays attention to the information of interest in the videos to be played. For example, the information of interest may be a person or a vehicle. Since the fused video can reflect the information of interest included in the at least two videos to be played, and the information of interest is the content that the user pays attention to in the videos to be played, the user only needs to watch the fused video to grasp the information of interest in the at least two videos to be played, without having to watch at least two videos to be played. Therefore, it is possible to reduce the user's visual fatigue while avoiding the user from missing information.

[0425] As an implementation of the embodiment of the present application, the video fusion module 2602 may include:

[0426] An information of interest determination submodule, used for extracting information of interest from each of the at least two videos to be played, wherein the information of interest belongs to foreground information of the video to be played;

[0427] The first interesting information display submodule is used to overlay and display the extracted interesting information on a preset background to obtain a fused video.

[0428] As an implementation of the embodiment of the present application, the above device may further include:

[0429] An identification display module is used to display an identification corresponding to each piece of information of interest within a preset range near the information of interest, wherein the identification corresponding to the information of interest includes at least one of a channel identification of a channel for transmitting a video frame to which the information of interest belongs, time information of the video frame to which the information of interest belongs, and category information of the information of interest.

[0430] As an implementation of the embodiment of the present application, the above device may further include:

[0431] a target channel determination module, configured to respond to a first switching operation issued by a user in a playback window corresponding to the fused channel and determine a target channel to be viewed by the user according to a triggering position of the first switching operation;

[0432] The target channel playing module is used to play the video screen of the target channel.

[0433] As an implementation of the embodiment of the present application, the target channel playing module may include:

[0434] An information of interest removal module, used to remove information of interest corresponding to other channels except the target channel in the at least two channels in the fused video;

[0435] A first background replacement module, used to replace the preset background with the video background of the to-be-played video corresponding to the target channel;

[0436] The foreground information display module is used to overlay and display other foreground information except the information of interest in the foreground information corresponding to the target channel in the replaced video background.

[0437] As an implementation of the embodiment of the present application, the target channel determination module may include:

[0438] A target to-be-triggered area determination submodule, used to determine a target to-be-triggered area including a trigger position of the first switching operation in the to-be-triggered area included in the fused video, wherein the to-be-triggered area is an area where the information of interest is located and / or an area where the channel identifier is located;

[0439] The target channel determination submodule is used to use the channel corresponding to the to-be-played video corresponding to the target to-be-triggered area as the target channel.

[0440] As an implementation of the embodiment of the present application, the above device may further include:

[0441] A second background replacement module, configured to replace the video background of the to-be-played video corresponding to the target channel with the preset background in response to a second switching operation issued by the user in the playback window corresponding to the fusion channel;

[0442] A foreground information removal module, used to remove the other foreground information in the to-be-played video corresponding to the target channel;

[0443] The interesting information display module is used to superimpose and display the interesting information corresponding to the other channels of the at least two channels except the target channel in the preset background.

[0444] As an implementation of the embodiment of the present application, the first interesting information display submodule may include:

[0445] A first information-of-interest display unit is used to overlay and display information of interest included in video frames of to-be-played videos corresponding to at least two channels in a preset background, wherein each channel is used to transmit a to-be-played video captured by a video capture device;

[0446] The second interesting information display unit is used to overlay and display the interesting information included in the video frames collected at different times in the video to be played corresponding to one channel in a preset background.

[0447] As an implementation of the embodiment of the present application, the first interesting information display unit may include:

[0448] A first position determination subunit is used to determine, in the at least two videos to be played, the position of the information of interest included in each video frame acquired at the same time in the video frame to which the information of interest belongs;

[0449] A second position determination subunit is used to determine the display position of the information of interest in the video frame to which the information of interest belongs, corresponding to the position of the information of interest in the playback window, according to the position and ratio of the information of interest in the video frame to which the information of interest belongs, wherein the ratio includes a scaling ratio and / or a rotation ratio, and the scaling ratio is a scaling ratio between the size of the video frame to which the information of interest belongs and the size of the playback window corresponding to the fusion channel;

[0450] The second information-of-interest display subunit is used to overlay and display the information of interest at a display position corresponding to the information of interest according to a preset resolution corresponding to the playback window.

[0451] As an implementation of the embodiment of the present application, the first video playback module 2603 may include:

[0452] A first fused video playback submodule, configured to play the fused video in a main playback window of a playback interface, wherein the playback interface includes the main playback window and at least one auxiliary playback window;

[0453] The first to-be-played video playing submodule is configured to play each to-be-played video in the at least one auxiliary playing window.

[0454] As an implementation of an embodiment of the present application, when the above-mentioned videos to be played are videos to be played corresponding to each channel, the arrangement order of the auxiliary playback windows for playing the videos to be played in the playback interface can be determined based on at least one of the following methods:

[0455] Determined based on a preset priority order of each channel;

[0456] Determining based on the number of times the to-be-played video corresponding to each channel is played in the main playback window;

[0457] Determining based on the last playback time of the to-be-played video corresponding to each channel in the main playback window;

[0458] When each video to be played is a video to be played in different time periods corresponding to a channel, the arrangement order of the auxiliary playback windows used to play each video to be played in the playback interface can be determined based on the arrangement order of the time periods corresponding to the videos to be played played by each auxiliary playback window.

[0459] As an implementation of the embodiment of the present application, the display areas of multiple pieces of information of interest in the play window may have overlapping areas. In this case, the first video play module 2603 may include:

[0460] The second interest information display submodule is used to display each interest information in the playback window corresponding to the fusion channel according to the preset priority order of the channel corresponding to each interest information, wherein the corresponding interest information with a high priority is located on the upper layer of the corresponding interest information with a low priority;

[0461] The third interest information display submodule is used to determine the transparency corresponding to each interest information according to a preset transparency setting method, and display each interest information according to the determined transparency.

[0462] As an implementation of an embodiment of the present application, the video to be played may be a playback video, and the playback interface may include a playback timeline, and the playback timeline may correspond to a statistical graph of events of interest. In this case, the apparatus may further include:

[0463] A time point jump module, for responding to a selection operation issued by a user on the playback timeline or the focus event statistical graph, and jumping to a time point corresponding to the selection operation;

[0464] The second video playing module is used to play the fused video starting from the time point, wherein the fused video is obtained by fusing the playback videos corresponding to a preset number of channels with the highest priority.

[0465] As an implementation manner of an embodiment of the present application, the above-mentioned event statistics graph may be at least one of a bar graph, a wave graph, a line graph, a column graph, a peak graph, and an area graph, and the height or color depth of the quantity mark corresponding to each time point of the event statistics graph is determined based on one of the following:

[0466] The sum of the number of events of interest occurring at each time point in the to-be-played video corresponding to the multiple channels of the fused video is obtained by fusion;

[0467] The sum of the number of attention events corresponding to the confidence levels that meet preset confidence requirements among the attention events occurring at various time points in the to-be-played video corresponding to the multiple channels of the fused video is obtained by fusion.

[0468] As an implementation of an embodiment of the present application, in the above-mentioned attention event statistics chart, the quantity identifiers corresponding to the to-be-played videos corresponding to different channels can be represented by different colors.

[0469] As an implementation of an embodiment of the present application, the priority is determined based on the importance of the scene captured by the video acquisition device corresponding to the channel, or the priority is determined based on the number of events of interest included in the video to be played transmitted by the channel.

[0470] As an implementation of an embodiment of the present application, a display device for playing the video to be played may include multiple display screens. In this case, the apparatus may further include:

[0471] A split-screen display position acquisition module, used to acquire a split-screen display position set by a user, wherein the split-screen display position represents a first display screen for playing the fused video and a second display screen for playing each video to be played;

[0472] The first video playback module may include:

[0473] A second fused video playing submodule, used for playing the fused video in the playing window of the first display screen;

[0474] The second to-be-played video playing submodule is used to play each to-be-played video in a plurality of playing windows of the second display screen respectively.

[0475] Corresponding to the above-mentioned video playing method, the embodiment of the present application further provides a video playing system. The following is an introduction to the video playing system provided by the embodiment of the present application.

[0476] like Fig. 27 As shown, a video playback system includes a video processing device 2701 and a display device 2702, wherein:

[0477] The video processing device 2701 is used to obtain at least two videos to be played; based on the interest information included in the at least two videos to be played, the at least two videos to be played are merged to obtain a fused video corresponding to a fused channel; and the fused video is played in the playback window corresponding to the fused channel in the display device 2702, wherein the video to be played includes a recorded video and / or a real-time video.

[0478] It can be seen that in an embodiment of the present application, the video processing device can obtain at least two videos to be played, wherein the videos to be played include recorded videos and / or real-time videos; based on the information of interest included in the at least two videos to be played, the at least two videos to be played are fused to obtain a fused video corresponding to the fused channel; and the fused video is played in the playback window corresponding to the fused channel. When watching the videos to be played, the user pays attention to the information of interest in the videos to be played. For example, the information of interest may be a person or a vehicle. Since the fused video can reflect the information of interest included in the at least two videos to be played, and the information of interest is the content that the user pays attention to in the videos to be played, the user only needs to watch the fused video to grasp the information of interest in the at least two videos to be played, without having to watch at least two videos to be played. Therefore, it is possible to reduce the user's visual fatigue while avoiding the user from missing information.

[0479] The display device 2702 may be a local display screen of the video processing device or a display screen of a remote client, such as a display screen of a mobile phone, a display screen of a tablet computer, a display screen of a laptop computer, and the like.

[0480] The present application also provides a video processing device, such as Fig.28 As shown, including:

[0481] Memory 2801, used for storing computer programs;

[0482] The processor 2802 is used to implement the video playback method described in any of the above embodiments when executing the program stored in the memory 2801.

[0483] In addition, the video processing device may further include a communication bus and / or a communication interface, and the processor 2802, the communication interface, and the memory 2801 communicate with each other via the communication bus.

[0484] It can be seen that in an embodiment of the present application, the video processing device can obtain at least two videos to be played, wherein the videos to be played include recorded videos and / or real-time videos; based on the information of interest included in the at least two videos to be played, the at least two videos to be played are fused to obtain a fused video corresponding to the fused channel; and the fused video is played in the playback window corresponding to the fused channel. When watching the videos to be played, the user pays attention to the information of interest in the videos to be played. For example, the information of interest may be a person or a vehicle. Since the fused video can reflect the information of interest included in the at least two videos to be played, and the information of interest is the content that the user pays attention to in the videos to be played, the user only needs to watch the fused video to grasp the information of interest in the at least two videos to be played, without having to watch at least two videos to be played. Therefore, it is possible to reduce the user's visual fatigue while avoiding the user from missing information.

[0485] The communication bus mentioned in the above video processing device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0486] The communication interface is used for communication between the above video processing device and other devices.

[0487] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0488] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0489] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned video playback methods are implemented.

[0490] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the video playback methods in the above embodiments.

[0491] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0492] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0493] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, system, video processing device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0494] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A video playback method, characterized in that: The method comprises: Acquire at least two videos to be played, wherein the videos to be played include recorded videos and / or real-time videos; Based on the information of interest included in at least two videos to be played, the at least two videos to be played are fused to obtain a fused video corresponding to a fusion channel; The fused video is played in the playback window corresponding to the fused channel.

2. The method according to claim 1, characterized in that The step of fusing the at least two videos to be played based on the information of interest included in the at least two videos to be played to obtain a fused video corresponding to a fused channel includes: For the at least two videos to be played, extracting information of interest in each of the videos to be played, wherein the information of interest belongs to foreground information of the videos to be played; The extracted information of interest is superimposed on the preset background to obtain a fused video.

3. The method according to claim 2, characterized in that After the step of superimposing and displaying the extracted information of interest on the preset background, the method further comprises: Within a preset range near each information of interest, an identifier corresponding to the information of interest is displayed, wherein the identifier corresponding to the information of interest includes a channel identifier of a channel for transmitting a video frame to which the information of interest belongs, time information of the video frame to which the information of interest belongs, and at least one of category information of the information of interest.

4. The method according to claim 3, characterized in that After the step of playing the fused video in the playback window corresponding to the fused channel, the method further includes: In response to a first switching operation issued by a user in the playback window corresponding to the fused channel, determining a target channel to be viewed by the user according to a triggering position of the first switching operation; Play the video image of the target channel.

5. The method according to claim 4, characterized in that The step of playing the video screen of the target channel includes: In the fused video, removing the information of interest corresponding to other channels except the target channel in the at least two channels; The preset background is replaced by the video background of the to-be-played video corresponding to the target channel; In the replaced video background, other foreground information except the information of interest in the foreground information corresponding to the target channel is superimposed and displayed; and / or, The step of determining the target channel to be viewed by the user according to the trigger position of the first switching operation includes: In the to-be-triggered area included in the fused video, determining a target to-be-triggered area including a trigger position of the first switching operation, wherein the to-be-triggered area is an area where the information of interest is located and / or an area where the channel identifier is located; The channel corresponding to the to-be-played video corresponding to the target to-be-triggered area is used as the target channel; and / or, After the step of superimposing and displaying other foreground information except the information of interest in the foreground information corresponding to the target channel in the replaced video background, the method further includes: In response to a second switching operation issued by the user in the playback window corresponding to the fusion channel, replacing the video background of the to-be-played video corresponding to the target channel with the preset background; In the to-be-played video corresponding to the target channel, removing the other foreground information; In the preset background, the information of interest corresponding to the other channels of the at least two channels except the target channel is superimposed and displayed.

6. The method according to claim 2, characterized in that The step of superimposing and displaying the extracted information of interest on a preset background comprises: In a preset background, the information of interest included in the video frames of the to-be-played videos corresponding to at least two channels is superimposed and displayed, wherein each channel is used to transmit the to-be-played video captured by a video capture device; or, In the preset background, the interesting information included in the video frames collected at different times in the video to be played corresponding to one channel is superimposed and displayed.

7. The method according to claim 6, characterized in that The step of superimposing and displaying the information of interest included in the video frames of the to-be-played videos corresponding to at least two channels comprises: The information of interest included in each video frame acquired at the same time in the videos to be played corresponding to at least two channels is superimposed and displayed.

8. The method according to claim 6, characterized in that The step of superimposing and displaying the information of interest included in the video frames of the to-be-played videos corresponding to at least two channels in the preset background comprises: In the at least two videos to be played, determining the position of the information of interest included in each video frame captured at the same time in the video frame to which the information of interest belongs; Determine, according to the position and ratio of the information of interest in the video frame to which the information of interest belongs, a display position corresponding to the position of the information of interest in the video frame to which the information of interest belongs in the playback window, wherein the ratio includes a scaling ratio and / or a rotation ratio, and the scaling ratio is a scaling ratio between the size of the video frame to which the information of interest belongs and the size of the playback window corresponding to the fusion channel; According to the preset resolution corresponding to the playback window, the information of interest is superimposed and displayed at a display position corresponding to the information of interest.

9. The method according to any one of claims 1 to 8, characterized in that: The step of playing the fused video in the playback window corresponding to the fused channel comprises: Play the fused video in a main play window of a play interface, wherein the play interface includes the main play window and at least one auxiliary play window; In the at least one auxiliary playback window, each video to be played is played.

10. The method according to claim 9, characterized in that In the case where the videos to be played are the videos to be played corresponding to the channels, the arrangement order of the auxiliary play windows for playing the videos to be played in the play interface is determined based on at least one of the following methods: Determined based on a preset priority order of each channel; Determining based on the number of times the to-be-played video corresponding to each channel is played in the main playback window; Determining based on the last playback time of the to-be-played video corresponding to each channel in the main playback window; In the case where the videos to be played are videos to be played in different time periods corresponding to a channel, the arrangement order of the auxiliary playback windows used to play the videos to be played in the playback interface is determined based on the arrangement order of the time periods corresponding to the videos to be played played by each auxiliary playback window.

11. The method according to any one of claims 1 to 8, characterized in that: There is an overlapping area in the display area of ​​the plurality of pieces of interesting information in the play window; The step of playing the fused video in the playback window corresponding to the fused channel comprises: Displaying each piece of information of interest in the playback window corresponding to the fusion channel according to the preset priority order of the channel corresponding to each piece of information of interest, wherein the corresponding information of interest with a high priority is located above the corresponding information of interest with a low priority; or, According to a preset transparency setting method, the transparency corresponding to each piece of information of interest is determined, and according to the determined transparency, each piece of information of interest is displayed.

12. The method according to any one of claims 1 to 8, characterized in that: The video to be played is a playback video, and the playback interface includes a playback timeline, and the playback timeline corresponds to a statistical graph of events of interest; The method further comprises: In response to a selection operation issued by a user on the playback timeline or the focus event statistics chart, jumping to a time point corresponding to the selection operation; Starting from the time point, the fused video is played.

13. The method according to claim 12, characterized in that: The event statistics graph is at least one of a bar graph, a wave graph, a line graph, a column graph, a peak graph, and an area graph. The height or color depth of the quantity mark corresponding to each time point of the event statistics graph is determined based on one of the following: The sum of the number of events of interest occurring at each time point in the to-be-played video corresponding to the multiple channels of the fused video is obtained by fusion; The total number of attention events corresponding to the attention events occurring at various time points in the to-be-played video corresponding to the multiple channels of the fused video and having confidence levels that meet preset confidence requirements is obtained by fusing; and / or, The method further comprises: In the attention event statistics graph, the quantity identifiers corresponding to the to-be-played videos corresponding to different channels are represented by different colors.

14. The method according to claim 11, characterized in that The priority is determined based on the importance of the scene captured by the video acquisition device corresponding to the channel, or the priority is determined based on the number of events of interest included in the video to be played transmitted by the channel.

15. The method according to any one of claims 1 to 8, characterized in that: The display device for playing the video to be played includes a plurality of display screens; Before the step of obtaining at least two videos to be played, the method further includes: Obtaining a split-screen display position set by a user, wherein the split-screen display position represents a first display screen for playing the fused video and a second display screen for playing each video to be played; The step of playing the fused video in the playback window corresponding to the fused channel comprises: Play the fused video in the play window of the first display screen; In the multiple play windows of the second display screen, each video to be played is played respectively.

16. A video playback device, characterized in that: The device comprises: A video acquisition module, used to acquire at least two videos to be played, wherein the videos to be played include recorded videos and / or real-time videos; A video fusion module, configured to fuse the at least two videos to be played based on the information of interest included in the at least two videos to be played, to obtain a fused video corresponding to a fusion channel; The first video playing module is used to play the fused video in the playing window corresponding to the fused channel.

17. The device according to claim 16, characterized in that The video fusion module comprises: An information of interest determination submodule, used for extracting information of interest from each of the at least two videos to be played, wherein the information of interest belongs to foreground information of the video to be played; A first interesting information display submodule is used to superimpose and display the extracted interesting information on a preset background to obtain a fused video; The device also includes: an identification display module, used to display an identification corresponding to each piece of information of interest within a preset range near the information of interest, wherein the identification corresponding to the information of interest includes at least one of a channel identification of a channel for transmitting a video frame to which the information of interest belongs, time information of the video frame to which the information of interest belongs, and category information of the information of interest; The device also includes: a target channel determination module, configured to respond to a first switching operation issued by a user in a playback window corresponding to the fused channel and determine a target channel to be viewed by the user according to a triggering position of the first switching operation; A target channel playing module, used to play the video screen of the target channel; The target channel playing module includes: An information of interest removal module, used to remove information of interest corresponding to other channels except the target channel in the at least two channels in the fused video; A first background replacement module, used to replace the preset background with the video background of the to-be-played video corresponding to the target channel; A foreground information display module, used for superimposing and displaying other foreground information, except the information of interest, in the foreground information corresponding to the target channel in the replaced video background; The target channel determination module includes: A target to-be-triggered area determination submodule, used to determine a target to-be-triggered area including a trigger position of the first switching operation in the to-be-triggered area included in the fused video, wherein the to-be-triggered area is an area where the information of interest is located and / or an area where the channel identifier is located; A target channel determination submodule is used to use the channel corresponding to the to-be-played video corresponding to the target area to be triggered as the target channel; The device also includes: A second background replacement module, configured to replace the video background of the to-be-played video corresponding to the target channel with the preset background in response to a second switching operation issued by the user in the playback window corresponding to the fusion channel; A foreground information removal module, used to remove the other foreground information in the to-be-played video corresponding to the target channel; An interesting information display module, used for superimposing and displaying interesting information corresponding to other channels of the at least two channels except the target channel in the preset background; The first interesting information display submodule includes: A first information-of-interest display unit is used to overlay and display information of interest included in video frames of to-be-played videos corresponding to at least two channels in a preset background, wherein each channel is used to transmit a to-be-played video captured by a video capture device; A second information-of-interest display unit is used to overlay and display the information of interest included in the video frames collected at different times in the to-be-played video corresponding to one channel in a preset background; The first interesting information display unit includes: A first interesting information display subunit is used to overlay and display the interesting information included in each video frame acquired at the same time in the to-be-played videos corresponding to at least two channels; The first interesting information display unit includes: A first position determination subunit is used to determine, in the at least two videos to be played, the position of the information of interest included in each video frame acquired at the same time in the video frame to which the information of interest belongs; A second position determination subunit is used to determine the display position of the information of interest in the video frame to which the information of interest belongs, corresponding to the position of the information of interest in the playback window, according to the position and ratio of the information of interest in the video frame to which the information of interest belongs, wherein the ratio includes a scaling ratio and / or a rotation ratio, and the scaling ratio is a scaling ratio between the size of the video frame to which the information of interest belongs and the size of the playback window corresponding to the fusion channel; A second information-of-interest display subunit is used to overlay and display the information of interest at a display position corresponding to the information of interest according to a preset resolution corresponding to the playback window; The first video playback module includes: A first fused video playback submodule, configured to play the fused video in a main playback window of a playback interface, wherein the playback interface includes the main playback window and at least one auxiliary playback window; A first to-be-played video playing submodule, used for playing each to-be-played video in the at least one auxiliary playing window; In the case where the videos to be played are the videos to be played corresponding to the channels, the arrangement order of the auxiliary play windows for playing the videos to be played in the play interface is determined based on at least one of the following methods: Determined based on a preset priority order of each channel; Determining based on the number of times the to-be-played video corresponding to each channel is played in the main playback window; Determining based on the last playback time of the to-be-played video corresponding to each channel in the main playback window; In the case where the videos to be played are videos to be played in different time periods corresponding to one channel, the arrangement order of the auxiliary play windows used to play the videos to be played in the play interface is determined based on the arrangement order of the time periods corresponding to the videos to be played played by each auxiliary play window; There is an overlapping area in the display area of ​​the plurality of pieces of interesting information in the play window; The first video playback module includes: The second interest information display submodule is used to display each interest information in the playback window corresponding to the fusion channel according to the preset priority order of the channel corresponding to each interest information, wherein the corresponding interest information with a high priority is located on the upper layer of the corresponding interest information with a low priority; A third interest information display submodule, used to determine the transparency corresponding to each interest information according to a preset transparency setting method, and display each interest information according to the determined transparency; The video to be played is a playback video, and the playback interface includes a playback timeline, and the playback timeline corresponds to a statistical graph of events of interest; The device also includes: A time point jump module, for responding to a selection operation issued by a user on the playback timeline or the focus event statistical graph, and jumping to a time point corresponding to the selection operation; A second video playing module, used for playing the fused video starting from the time point; The event statistics graph is at least one of a bar graph, a wave graph, a line graph, a column graph, a peak graph, and an area graph. The height or color depth of the quantity mark corresponding to each time point of the event statistics graph is determined based on one of the following: The sum of the number of events of interest occurring at each time point in the to-be-played video corresponding to the multiple channels of the fused video is obtained by fusion; The total number of attention events corresponding to the attention events occurring at various time points in the to-be-played video corresponding to the multiple channels of the fused video and having confidence levels that meet preset confidence requirements is obtained by fusing; In the attention event statistics graph, the quantity identifiers corresponding to the to-be-played videos corresponding to different channels are represented by different colors; The priority is determined based on the importance of the scene captured by the video acquisition device corresponding to the channel, or the priority is determined based on the number of events of interest included in the video to be played transmitted by the channel; The display device for playing the video to be played includes multiple display screens; The device also includes: A split-screen display position acquisition module, used to acquire a split-screen display position set by a user, wherein the split-screen display position represents a first display screen for playing the fused video and a second display screen for playing each video to be played; The first video playback module includes: A second fused video playing submodule, used for playing the fused video in the playing window of the first display screen; The second to-be-played video playing submodule is used to play each to-be-played video in a plurality of playing windows of the second display screen respectively.

18. A video playback system, characterized in that: The video playback system includes a video processing device and a display device, wherein: The video processing device is used to obtain at least two videos to be played; based on the information of interest included in the at least two videos to be played, the at least two videos to be played are merged to obtain a fused video corresponding to a fused channel; and the fused video is played in a playback window corresponding to the fused channel in the display device, wherein the videos to be played include recorded videos and / or real-time videos.

19. A video processing device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-15 when executing a program stored in a memory.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 15 is implemented.

Images