Synthetic video decomposition method and device, electronic equipment and computer storage medium

Abstract

The invention relates to a synthetic video decomposition method and device, electronic equipment and a computer storage medium, and belongs to the technical field of video processing.The synthetic video decomposition method comprises the steps that split screen detection is conducted on a synthetic video image, and split screen areas of split screens in the synthetic video image are obtained; image features in the split-screen areas are extracted, similarity values of the image features in the split-screen areas in the synthesized video images are calculated, split screens to be recombined belonging to the same decomposed sub-video are determined based on the similarity values, and the similarity values of the image features of the split screens to be recombined are larger than a preset similarity threshold value; and recombining the split screens to be recombined based on the time frame sequence of the synthesized video images corresponding to the split screens to be recombined in the synthesized video to obtain decomposed sub-videos. The video can be accurately decomposed and synthesized.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing, and particularly to a method, apparatus, electronic device, and computer storage medium for decomposing a synthesized video. Background Art

[0002] A synthesized video refers to a video obtained by combining multiple sub-videos captured by multiple cameras into one video. In many cases, the synthesized video needs to be decomposed, such as in a remote dual-recording post-quality inspection and audit system, video conferencing, monitoring systems, multimedia content analysis, etc.

[0003] In the existing synthesized video decomposition technology, due to the possible variations in the resolution of each sub-video in the synthesized video, the position of each sub-video in the synthesized video, the number of sub-videos in each frame of the synthesized video image, the type of the main body in each sub-video, the number of main bodies, etc., it is difficult to decompose the synthesized video, resulting in inaccurate sub-videos obtained after decomposing the synthesized video, or even unable to decompose.

[0004] It can be seen that the existing technology cannot adapt to various variation factors of sub-videos in the synthesized video, resulting in difficulties in decomposing the synthesized video. Summary of the Invention

[0005] In view of this, it is necessary to provide a method, apparatus, electronic device, and computer storage medium for decomposing a synthesized video to solve the problem that the existing technology cannot adapt to various variation factors of sub-videos in the synthesized video, resulting in difficulties in decomposing the synthesized video.

[0006] To solve the above problems, in a first aspect, the present invention provides a method for decomposing a synthesized video, including: Performing split-screen detection on a synthesized video image to obtain the split-screen regions of each split screen in the synthesized video image; Extracting the image features within the split-screen regions, calculating the similarity values of the image features within each split-screen region in each synthesized video image, and determining the split screens to be recombined belonging to the same decomposed sub-video based on the similarity values, where the similarity value of the image features of the split screens to be recombined is greater than a preset similarity threshold; Recombining each split screen to be recombined based on the time frame order of the synthesized video images corresponding to each split screen to be recombined in the synthesized video to obtain a decomposed sub-video.

[0007] In a possible implementation manner, performing split-screen detection on a synthesized video image to obtain the split-screen regions of each split screen in the synthesized video image includes: Using a preset split-screen detection algorithm to perform split-screen detection on each frame of the synthesized video image of the synthesized video, and determining the positions of the split-screen regions in each frame of the synthesized video image, where the position of the split-screen region is the coordinate of the split-screen region in the synthesized video image.

[0008] In a possible implementation, extracting the image features within the split-screen area includes: Extracting the color histogram features within the split-screen area.

[0009] In a possible implementation, extracting the image features within the split-screen area includes: When the color histogram features do not meet the preset determination conditions for the split-screens to be recombined, extracting the Gabor texture features within the split-screen area.

[0010] In a possible implementation, determining the split-screens to be recombined that belong to the same decomposed sub-video based on the similarity value includes: Determining a second split-screen corresponding to the first split-screen based on the similarity value, where the similarity value between the image features within the second split-screen and the image features within the first split-screen is the largest, and the second split-screen and the first split-screen are split-screens in different synthesized video images; When the similarity value between the image features within the first split-screen and the image features within the second split-screen is greater than the preset similarity threshold, determining that the first split-screen and the second split-screen are the split-screens to be recombined that belong to the same decomposed sub-video.

[0011] In a possible implementation, when the similarity value between the image features within the first split-screen and the image features within the second split-screen is less than or equal to the preset similarity threshold, the synthesized video decomposition method includes: Creating a new decomposed sub-video based on the second split-screen and using the second split-screen as the split-screen to be recombined in the new decomposed sub-video.

[0012] In a possible implementation, recombining the split-screens to be recombined based on the time frame order of the synthesized video images corresponding to the split-screens to be recombined in the synthesized video to obtain a decomposed sub-video includes: Determining the time frame order of the split-screens to be recombined in the decomposed sub-video based on the time frame order of the synthesized video images corresponding to the split-screens to be recombined in the synthesized video; Recombining the split-screens to be recombined based on the time frame order of the split-screens to be recombined in the decomposed sub-video to obtain a decomposed sub-video.

[0013] In a second aspect, the present invention further provides a synthesized video decomposition device, including: A split-screen detection module, configured to perform split-screen detection on the synthesized video image to obtain the split-screen areas of each split-screen in the synthesized video image; A similarity calculation module, configured to extract the image features within the split-screen area, calculate the similarity values of the image features within the split-screen areas of each synthesized video image, and determine the split-screens to be recombined that belong to the same decomposed sub-video based on the similarity values, where the similarity value of the image features of the split-screens to be recombined is greater than the preset similarity threshold; A sub-video recombination module, configured to recombine each sub-screen to be recombined based on the time frame order of the synthesized video images corresponding to each sub-screen to be recombined in the synthesized video, so as to obtain a decomposed sub-video.

[0014] In a third aspect, the present invention further provides an electronic device, including a memory and a processor, wherein, The memory is used to store programs; The processor is coupled to the memory and is configured to execute the programs stored in the memory to implement the steps in the synthesized video decomposition method in any of the above embodiments; The display is used to display the decomposed sub-video after the synthesized video is decomposed.

[0015] In a fourth aspect, the present invention further provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the programs or instructions are executed by a processor, the steps in the synthesized video decomposition method in any of the above embodiments can be implemented.

[0016] The beneficial effects of the present invention are as follows: The synthesized video decomposition method provided by the present invention performs split-screen detection on the synthesized video images, obtains the split-screen areas of each split-screen in the synthesized video images, and effectively determines the number of split-screens in each frame of the synthesized video image; determines the sub-screens to be recombined belonging to the same decomposed sub-video through the similarity values of the image features in the split-screen areas, and can effectively cope with the changes in resolution, position, subject type, and the number of subjects in each sub-video, and accurately identify the split-screens belonging to the same sub-video in the synthesized video image; recombines each sub-screen to be recombined based on the time frame order of the synthesized video images corresponding to the sub-screens to be recombined in the synthesized video, which can ensure the order of each frame of sub-video images in the decomposed sub-video and determine the accuracy of the decomposed sub-video. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those skilled in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a schematic flowchart of a synthesized video decomposition method provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of split-screen detection of a synthesized video image provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of split-screen detection of multiple frames of synthesized video provided by an embodiment of the present invention; Figure 4Schematic flowchart of a method for determining a sub-screen to be recombined provided by an embodiment of the present invention; Figure 5 Schematic flowchart of a method for recombining decomposed sub-videos provided by an embodiment of the present invention; Figure 6 Schematic structural diagram of a composite video decomposition device provided by an embodiment of the present invention; Figure 7 Schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0019] The following will specifically describe the preferred embodiments of the present invention with reference to the accompanying drawings. The accompanying drawings form a part of this application and are used together with the embodiments of the present invention to explain the principles of the present invention, rather than to limit the scope of the present invention.

[0020] In the embodiments of the present invention, the descriptions such as "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Therefore, the technical features defined with "first" and "second" may explicitly or implicitly include at least one of such features.

[0021] Referring to "embodiment" herein means that the specific features, structures, or characteristics described in connection with the embodiment may be included in at least one embodiment of the present invention. The appearance of this phrase at various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments.

[0022] A specific embodiment of the present invention, as Figure 1 shown, discloses a method for decomposing a composite video, including: S101, perform split-screen detection on the composite video image to obtain the split-screen regions of each split screen in the composite video image.

[0023] In the embodiments of the present invention, the composite video image refers to a frame image in the composite video. The composite video is composed of multiple composite video images in the order of time frames. Each composite video image includes multiple sub-video images collected by multiple image acquisition devices. For a frame of composite video image, one sub-video image is displayed in the form of a split screen. As Figure 2 shown, it is a schematic diagram of a frame of composite video image, which includes three split screens, namely split-screen region 1, split-screen region 2, and split-screen region 3. These three split screens are video images collected by three image acquisition devices.

[0024] The split-screen detection of the synthesized video image refers to identifying the number of split-screens in the synthesized video image and the split-screen areas of the split-screens in the synthesized video image. For example, in combination with Figure 2 , this frame of the synthesized video image includes three split-screens, and the split-screen areas of the three split-screens in the synthesized video image are respectively the rectangular frame where sub-video 1 is located, the rectangular frame where sub-video 2 is located, and the rectangular frame where sub-video 3 is located. Specifically, the split-screen detection method for each frame of the synthesized video image will be described in detail later in the present invention.

[0025] It should be noted that for the split-screen detection of the synthesized video image, it can be carried out in the order of the time frames of the synthesized video image in the synthesized video, or simultaneously. The present invention does not limit this.

[0026] S102, extract the image features within the split-screen area, calculate the similarity values of the image features within each split-screen area of each synthesized video image, and determine the split-screens to be recombined belonging to the same decomposed sub-video based on the similarity values, where the similarity value of the image features of the split-screens to be recombined is greater than the preset similarity threshold.

[0027] In the embodiment of the present invention, after the split-screen detection of the synthesized video image to obtain each split-screen area, extract the image features within each split-screen area. This image feature should be able to characterize features such as the resolution, the number of subjects, and the type of subjects of the image within the split-screen area. This image feature can be used to determine whether two split-screens are videos captured by the same image acquisition device. The method for extracting the image features within the split-screen area will be described in detail later in the present invention.

[0028] In the embodiment of the present invention, calculate the similarity values of the image features within each split-screen area, and determine the split-screens to be recombined belonging to the same decomposed sub-video based on the similarity values, where the similarity value of the image features of the split-screens to be recombined is greater than the preset similarity threshold. Optionally, as Figure 3 shown, the figure shows two frames of synthesized video images. Each synthesized video image has three split-screens. For the split-screen areas 1, 2, and 3 in the synthesized video image 1, they are the split-screens to be recombined of sub-videos 1, 2, and 3 respectively. After feature extraction and similarity calculation of the image features, the split-screens to be recombined corresponding to sub-videos 1, 2, and 3 in the synthesized video image 2 are split-screen areas 4, 5, and 6 respectively. Then it can be determined that split-screen area 1 and split-screen area 4 belong to the same decomposed sub-video, split-screen area 2 and split-screen area 5 belong to the same decomposed sub-video, and split-screen area 3 and split-screen area 6 belong to the same decomposed sub-video. Among them, the calculation method of the similarity value of the image features can be the cosine similarity calculation method or the Euclidean distance similarity calculation method. The present invention does not limit this.

[0029] S103. Recombine each screen to be recombined according to the time frame order of the corresponding synthesized video images in the synthesized video to obtain a decomposed sub-video.

[0030] In the embodiment of the present invention, after determining the screens to be recombined that belong to the same decomposed sub-video, each screen to be recombined is recombined according to the time frame order of the corresponding synthesized video images in the synthesized video to obtain a decomposed sub-video. The specific method for synthesizing the decomposed sub-video will be described in detail later in the present invention.

[0031] The synthesized video decomposition method provided by the present invention performs screen detection on the synthesized video images to obtain the screen regions of each screen in the synthesized video images, and can effectively determine the number of screens in each frame of the synthesized video images; determines the screens to be recombined that belong to the same decomposed sub-video through the similarity values of the image features in the screen regions, and can effectively handle the changes in resolution, position, subject type, and the number of subjects in each sub-video, and accurately identify the screens in the synthesized video images that belong to the same sub-video; recombines each screen to be recombined according to the time frame order of the corresponding synthesized video images in the synthesized video, which can ensure the order of each frame of sub-video images in the decomposed sub-video and determine the accuracy of the decomposed sub-video.

[0032] In some possible embodiments of the present invention, performing screen detection on the synthesized video images to obtain the screen regions of each screen in the synthesized video images includes: Performing screen detection on each frame of the synthesized video images by using a preset screen detection algorithm to determine the positions of the screen regions in each frame of the synthesized video images, where the position of the screen region is the coordinate of the screen region in the synthesized video image.

[0033] In the embodiment of the present invention, when performing screen detection on each frame of the synthesized video images, a screen detection algorithm, such as an object detection algorithm, a segmentation algorithm, a line detection algorithm, etc., can be used to construct a screen detection model for screen detection, and a dataset of synthesized video images annotated with screen data is collected to train the screen detection model, and then the trained model is used to perform screen detection on each frame of the synthesized video images to obtain the positions of the screen regions in each frame of the synthesized video images, where the position of the screen region is the coordinate of the screen region in the synthesized video image. Specifically, a rectangular coordinate system can be constructed in the synthesized video image, and the coordinates of the four vertices of the screen rectangle are used to represent the screen region of the screen region in the synthesized video image. For example, assuming that there are k decomposed sub-videos in the synthesized video, the kth decomposed sub-video is represented as a list, and the elements in the list are all the frames of the kth decomposed sub-video, that is sub_video_k = [sub_k_frame_u,u=0,1,2,...,Mk-1] Among them, sub_video_k represents the k-th decomposed sub-video, Mk represents the total number of frames of the k-th decomposed sub-video, and sub_k_frame_u represents the u-th frame image of sub_video_k, where k = 0, 1,..., K - 1. The i-th frame merge_frame_i of the synthesized video (i.e., the i-th frame synthesized video image) is subjected to split-screen detection. The results of the split-screen detection are represented as merge_frame_i_rect_j and merge_frame_i_sub_j, where j = 0, 1, 2,..., Ni - 1, Ni is the number of split-screens of the i-th frame merge_frame_i of the synthesized video, merge_frame_i_rect_j is the rectangular coordinates of the j-th split-screen area of the i-th frame merge_frame_i of the synthesized video, and merge_frame_i_sub_j is the image of the j-th split-screen of the i-th frame merge_frame_i of the synthesized video.

[0034] In the embodiments of the present invention, the split-screen detection algorithm is used to perform split-screen detection on each frame of the synthesized video image, with high split-screen detection efficiency and more accurate split-screen detection results.

[0035] In some possible embodiments of the present invention, extracting the image features within the split-screen area includes: Extracting the color histogram features within the split-screen area.

[0036] In the embodiments of the present invention, for the extraction of the image features within the split-screen area, it is necessary to ensure that the extracted image features are robust to changes in the video resolution size. The color histogram features can be selected. Within a preset color space, such as the RGB color space, the proportion of each color within the split-screen area is extracted, and a color proportion histogram is drawn, which is not affected by the image size and can stably represent the image content. Further, in order to reduce the feature dimension, the R, G, and B components can be respectively mapped to 64 discrete intervals, constituting a 64 * 3 = 192-dimensional feature space. Among them, the image features of the j-th split-screen of the i-th frame merge_frame_i of the synthesized video can be represented as merge_frame_i_fea_j.

[0037] In the embodiments of the present invention, the extraction is performed through the color histogram features within the split-screen area, which is robust to changes in the sub-video resolution size, not affected by the image size, and can stably represent the image content.

[0038] In some possible embodiments of the present invention, extracting the image features within the split-screen area includes: When the color histogram features do not meet the preset conditions for determining the sub - screens to be recombined, extract the Gabor texture features within the sub - screen regions. It should be noted that it is also possible to use other texture features (such as LBP features).

[0039] In the embodiments of the present invention, for some video acquisition scenarios, the color histogram features may not meet the conditions for determining the sub - screens to be recombined. For example, in the same room with the same participants and the same dominant environmental color tone, at this time, Gabor texture features can be extracted. Specifically, first, randomly select a certain percentage (such as 2%) of the frame images from the synthesized video. After performing sub - screen detection, extract the color histogram features within each sub - screen region, compare the similarity of the color histogram features of each sub - screen within the same frame, and calculate the minimum value among these similarities. If this minimum value exceeds the preset threshold, it is determined that the color histogram features are sufficient for differentiation and continue to be used; otherwise, if the minimum similarity is lower than the threshold, then switch to using more complex Gabor texture features to ensure the accuracy and robustness of feature description.

[0040] The embodiments of the present invention can ensure the accuracy and robustness of feature extraction by providing another type of image feature.

[0041] In some possible embodiments of the present invention, as Figure 4 shown, determining the sub - screens to be recombined belonging to the same decomposed sub - video based on the similarity value includes: S401, determine the second sub - screen corresponding to the first sub - screen based on the similarity value, where the similarity value between the image features within the second sub - screen and the image features within the first sub - screen is the largest, and the second sub - screen and the first sub - screen are sub - screens in different synthesized video images; S402, when the similarity value between the image features within the first sub - screen and the image features within the second sub - screen is greater than the preset similarity threshold, determine that the first sub - screen and the second sub - screen are the sub - screens to be recombined belonging to the same decomposed sub - video.

[0042] In an embodiment of the present invention, the first split screen refers to the split screen for which the decomposed sub-video has been determined, and the second split screen refers to the split screen for which the decomposed sub-video has not been determined yet. When determining the decomposed sub-video to which the second split screen belongs, the similarity value between the image features in the second split screen and the image features in all split screens of other synthesized video frames can be calculated, and the first split screen with the largest similarity value can be determined. Then, it is determined whether the similarity value between the image features in the first split screen and the image features in the second split screen is greater than a preset similarity threshold. If so, the first split screen and the second split screen are the split screens to be recombined belonging to the same decomposed sub-video. Specifically, for the 0th frame of the synthesized video (i == 0), corresponding decomposed sub-videos are created according to the number of split screens N0 of the 0th frame of the synthesized video, and the split sub-images of the 0th frame of the synthesized video are used as the 0th frame of the corresponding sub-videos. That is: K = N0; the kth decomposed sub-video is represented as: sub_video_k = [sub_k_frame_u, u = 0, 1, 2,..., Mk - 1], k = 0, 1, 2,..., K - 1, Mk = 1 (at this time, there is only one frame image in the sub-video sub_video_k), and note that the sub-video numbers start from 0. sub_k_frame_0 = merge_frame_0_sub_k (the kth split sub-image of the 0th frame of the synthesized video is used as the 0th frame of the kth sub-video) sub_features_k = [sub_k_fea_u, u = 0, 1, 2,..., Mk - 1], k = 0, 1,..., K - 1, Mk = 1 sub_k_fea_0 = merge_frame_0_fea_k (the features of the kth split sub-image of the 0th frame of the synthesized video are used as the features of the 0th frame of the kth decomposed sub-video) For the subsequent frames of the synthesized video (i >= 1), each of the Ni split screens of the ith frame of the synthesized video is processed one by one. The features merge_frame_i_fea_j (j = 0, 1,..., Ni - 1) of the jth split sub-image merge_frame_i_sub_j of the ith frame of the synthesized video are compared one by one with the feature list sub_features_k (k = 0, 1,..., K - 1) of the existing sub-videos to find the maximum similarity dis_m and its corresponding sub-video sub_video_m.

[0043] If dis_m is greater than or equal to the set threshold (e.g., 0.9), then it is considered that the split-screen sub-image merge_frame_i_sub_j belongs to sub_video_m (0 <= m <= K - 1), update sub_video_m and sub_features_m, add merge_frame_i_sub_j to sub_video_m, add merge_frame_i_fea_j to sub_features_m, and increment the number of frames of sub_video_m by 1, i.e., Mm = Mm + 1.

[0044] In some possible embodiments of the present invention, when the similarity value between the image features in the first split screen and the image features in the second split screen is less than or equal to the preset similarity threshold, the composite video decomposition method includes: Create a new decomposed sub-video based on the second split screen, and use the second split screen as the split screen to be recombined in the new decomposed sub-video.

[0045] In an embodiment of the present invention, following the foregoing embodiment, if dis_m is less than the set threshold (e.g., 0.9), then create a new sub-video sub_video_K, and use merge_frame_i_sub_j as the 0th frame of the newly created sub-video sub_video_K. Since there are already sub-videos sub_video_k (k = 0, 1, 2,..., K - 1) for the 0th, 1st,..., (K - 1)th paths, the newly created sub-video is marked as sub_video_K, i.e.: sub_video_K = [sub_K_frame_u, u = 0, 1, 2,..., MK - 1], MK = 1 (the number of frames of the newly created sub-video is 1) sub_K_frame_0 = merge_frame_i_sub_j (use merge_frame_i_sub_j as the 0th frame sub_K_frame_0 of the newly created sub-video sub_video_K) sub_features_K = [sub_K_fea_u, u = 0, 1, 2,..., MK - 1], MK = 1 sub_K_fea_0 = merge_frame_i_fea_j Update K = K + 1, i.e., increment the number of decomposed sub-videos by 1.

[0046] Based on the steps of the above two embodiments, process all the frame composite video images of the composite video, and finally output the K decomposed sub-videos included in the video, sub_video_k = [sub_k_frame_u, u = 0, 1,..., Mk - 1], where sub_video_k is the k-th decomposed sub-video, and Mk is the total number of frames of the k-th decomposed sub-video, k = 0, 1, 2,..., K - 1.

[0047] In the embodiment of the present invention, by determining the split screens to be recombined belonging to the same decomposed sub-video according to the similarity value of the image features within the split screens, the accuracy of the decomposition of the composite video is ensured.

[0048] In some possible embodiments of the present invention, as Figure 5 shown, based on the time frame order of the composite video images corresponding to each split screen to be recombined in the composite video, recombine each split screen to be recombined to obtain a decomposed sub-video, including: S501, determine the time frame order of the split screens to be recombined in the decomposed sub-video based on the time frame order of the composite video images corresponding to each split screen to be recombined in the composite video; S502, synthesize each split screen to be recombined based on the time frame order of each split screen to be recombined in the decomposed sub-video to obtain a decomposed sub-video.

[0049] In the embodiment of the present invention, for the decomposition of the composite video, all the composite video images can be first subjected to split screen detection, the image features within each split screen can be determined based on the split screen detection results, and then the split screens to be recombined belonging to the same decomposed sub-video can be determined based on the method in the foregoing embodiments. For the split screens to be recombined belonging to the same decomposed sub-video, determine the time frame order of the split screens to be recombined in the decomposed sub-video based on the time frame order of the composite video images corresponding to each split screen to be recombined in the composite video, and then recombine each split screen to be recombined based on the time frame order of each split screen to be recombined in the decomposed sub-video to obtain a decomposed sub-video. Based on this, multiple decomposed sub-videos can be obtained.

[0050] In the embodiment of the present invention, the split screens to be recombined are recombined according to the time frame order of the composite video in the decomposed sub-video, ensuring the accuracy of the synthesized decomposed sub-video.

[0051] In order to better implement the composite video decomposition method in the embodiment of the present invention, correspondingly, based on the composite video decomposition method, as Figure 6 shown, the embodiment of the present invention further provides a composite video decomposition device. The composite video decomposition device 600 includes: A split screen detection module 601, configured to perform split screen detection on the composite video images to obtain the split screen areas of each split screen in the composite video images; A similarity calculation module 602, configured to extract image features within the split screen regions, calculate similarity values of the image features within each split screen region in each synthesized video image, and determine the split screens to be recombined belonging to the same decomposed sub-video based on the similarity values, where the similarity value of the image features of the split screens to be recombined is greater than a preset similarity threshold. A sub-video recombination module 603, configured to recombine each split screen to be recombined based on the time frame order of the synthesized video images corresponding to each split screen to be recombined in the synthesized video, so as to obtain a decomposed sub-video.

[0052] The synthesized video decomposition device 600 provided in the above embodiment can implement the technical solutions described in the above embodiment of the synthesized video decomposition method. For the specific implementation principles of the above modules or units, reference can be made to the corresponding content in the above embodiment of the synthesized video decomposition method, which will not be elaborated here.

[0053] As Figure 7 shown, the present invention also correspondingly provides an electronic device 700. The electronic device 700 includes a processor 701, a memory 702, and a display 703. Figure 7 Only some components of the electronic device 700 are shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0054] In some embodiments, the processor 701 may be a central processing unit (CPU), a microprocessor, or other data processing chips, and is used to run the program code stored in the memory 702 or process data, that is, the synthesized video decomposition method in the present invention.

[0055] In some embodiments, the processor 701 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processor 701 may be local or remote. In some embodiments, the processor 701 may be implemented on a cloud platform. In some embodiments, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-cloud, etc., or any combination of the above.

[0056] In some embodiments, the memory 702 may be an internal storage unit of the electronic device 700, such as the hard disk or memory of the electronic device 700. In some other embodiments, the memory 702 may also be an external storage device of the electronic device 700, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 700.

[0057] Further, the memory 702 may include both the internal storage unit of the electronic device 700 and external storage devices. The memory 702 is used to store the application software installed in the electronic device 700 and various types of data.

[0058] In some embodiments, the display 703 may be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) toucher, etc. The display 703 is used to display the information of the electronic device 700 and to display a visual user interface. The components 701 - 703 of the electronic device 700 communicate with each other through the system bus.

[0059] In some embodiments, when the processor 701 executes the composite video decomposition program in the memory 702, the following steps may be implemented: Perform split-screen detection on the composite video image to obtain the split-screen areas of each split screen in the composite video image; Extract the image features within the split-screen areas, calculate the similarity values of the image features within each split-screen area in each composite video image, and determine the split screens to be recombined belonging to the same decomposed sub-video based on the similarity values, where the similarity value of the image features of the split screens to be recombined is greater than a preset similarity threshold; Recombine the split screens to be recombined based on the time frame order of the composite video images corresponding to each split screen to be recombined in the composite video to obtain the decomposed sub-video.

[0060] It should be understood that when the processor 701 executes the composite video decomposition program in the memory 702, in addition to the above functions, other functions may also be implemented. For specific details, reference may be made to the description of the corresponding method embodiments above.

[0061] Further, the embodiments of the present invention do not specifically limit the type of the mentioned electronic device 700. The electronic device 700 may be a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop, and other portable electronic devices. Exemplary embodiments of the portable electronic device include, but are not limited to, portable electronic devices running IOS, android, microsoft, or other operating systems. The above portable electronic devices may also be other portable electronic devices, such as a laptop with a touch-sensitive surface (such as a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 700 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (such as a touch panel).

[0062] Accordingly, an embodiment of the present application further provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the programs or instructions are executed by a processor, the steps or functions in the composite video decomposition method provided by the above method embodiments can be implemented.

[0063] Those skilled in the art can understand that all or part of the processes for implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory, etc.

[0064] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A synthetic video decomposition method, characterized in that: include: Performing split-screen detection on the composite video image to obtain a split-screen area of ​​each split-screen in the composite video image; Extracting image features in the split-screen area, calculating similarity values ​​of the image features in each split-screen area in each synthetic video image, and determining the split screens to be reassembled belonging to the same decomposed sub-video based on the similarity values, wherein the similarity values ​​of the image features of the split screens to be reassembled are greater than a preset similarity threshold; The to-be-recombined split screens are recombined based on the time frame sequence of the composite video images corresponding to the to-be-recombined split screens in the composite video to obtain decomposed sub-videos.

2. The synthetic video decomposition method according to claim 1, characterized in that: The performing split-screen detection on the composite video image to obtain the split-screen area of ​​each split-screen in the composite video image includes: A preset split-screen detection algorithm is used to perform split-screen detection on each frame of the composite video image to determine the position of each split-screen area in each frame of the composite video image, wherein the position of the split-screen area is the coordinate of the split-screen area in the composite video image.

3. The synthetic video decomposition method according to claim 1, characterized in that: The extracting the image features in the split-screen area comprises: The color histogram features in the split-screen area are extracted.

4. The synthetic video decomposition method according to claim 3, characterized in that: The extracting the image features in the split-screen area comprises: When the color histogram feature does not meet the preset conditions for determining the split screen to be reorganized, the Gabor texture feature in the split screen area is extracted.

5. The synthetic video decomposition method according to claim 1, characterized in that: The determining, based on the similarity value, the split screens to be reassembled that belong to the same decomposed sub-video includes: Determining a second split screen corresponding to the first split screen based on the similarity value, wherein the similarity value between the image feature in the second split screen and the image feature in the first split screen is the largest, and the second split screen and the first split screen are split screens in different synthetic video images; When the similarity value between the image features in the first split screen and the image features in the second split screen is greater than a preset similarity threshold, the first split screen and the second split screen are determined to be split screens to be reassembled that belong to the same decomposed sub-video.

6. The synthetic video decomposition method according to claim 5, characterized in that: When the similarity value between the image feature in the first split screen and the image feature in the second split screen is less than or equal to a preset similarity threshold, the synthetic video decomposition method includes: A new decomposed sub-video is created based on the second split screen, and the second split screen is used as the split screen to be reassembled of the new decomposed sub-video.

7. The synthetic video decomposition method according to claim 1, characterized in that: The step of recombining the split screens to be recombined based on the time frame sequence of the composite video images corresponding to the split screens to be recombined in the composite video to obtain the decomposed sub-videos includes: Determining the time frame sequence of the to-be-recombined split screens in the decomposed sub-videos based on the time frame sequence of the synthesized video images corresponding to the to-be-recombined split screens in the synthesized video; The split screens to be reassembled are reassembled based on the time frame sequence of each split screen to be reassembled in the decomposed sub-video to obtain the decomposed sub-video.

8. A synthetic video decomposition device, characterized in that: include: A split screen detection module, used to perform split screen detection on the composite video image to obtain a split screen area of ​​each split screen in the composite video image; a similarity calculation module, configured to extract image features in the split-screen area, calculate similarity values ​​of the image features in each split-screen area in each synthesized video image, and determine the split-screens to be reassembled belonging to the same decomposed sub-video based on the similarity values, wherein the similarity values ​​of the image features of the split-screens to be reassembled are greater than a preset similarity threshold; The sub-video reassembly module is used to reassemble each of the to-be-reassembled split screens based on the time frame sequence of the synthetic video images corresponding to each of the to-be-reassembled split screens in the synthetic video to obtain a decomposed sub-video.

9. An electronic device, characterized in that: comprising a memory, a processor and a display, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps of the synthetic video decomposition method according to any one of claims 1 to 7; The display is used for displaying the decomposed sub-videos after the composite video is decomposed.

10. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the synthetic video decomposition method described in any one of claims 1 to 7.

Images