Video data storage method, device, electronic device and readable storage medium

By obtaining frame pair information between video clips and storing their association relationships, the problem of low material retrieval efficiency during video synthesis is solved, and the efficiency of video synthesis is improved.

CN113297416BActive Publication Date: 2024-11-12BEIJING DA MI TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202110560239.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-05-21
Publication Date
2024-11-12
Estimated Expiration
2041-05-21

AI Technical Summary

Technical Problem

In the prior art, the material retrieval efficiency during video synthesis is low, resulting in a decrease in video synthesis efficiency.

Method used

By obtaining frame pair information between video clips, characterizing their association relationships, and retaining these associations during storage, so as to achieve rapid retrieval during subsequent video synthesis.

Benefits of technology

The efficiency of video synthesis is improved, and the search time of video synthesis materials is reduced through rapid search based on association relationships.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113297416B_ABST
    Figure CN113297416B_ABST
Patent Text Reader

Abstract

The embodiments of the present application provide a video data storage method, device, electronic device and readable storage medium, which relate to the field of computer technology. Through the embodiments of the present application, before storing video clips, the electronic device used for video data storage can first determine the frame pair information between multiple video clips. Then, each video clip is used as a video node, and each frame pair information is used as the association relationship between each video node, and each video clip and each frame pair information is stored. Since the frame pair information can represent that there is an association relationship between two video clips, the embodiments of the present application can retain the association relationship between each video clip when storing each video clip. When performing video synthesis later, the rapid retrieval of video synthesis materials can be realized based on the above association relationship, thereby improving the efficiency of video synthesis.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video data storage method, device, electronic device and readable storage medium. Background Art

[0002] At present, with the development of Internet technology, the number of online service platforms is increasing. People can interact with online service platforms through terminal devices to obtain corresponding online services.

[0003] In the process of online services, in order to enable users to obtain a better experience, the online service platform can display a synthetic video on the display screen of the user-side terminal device through the corresponding application. For example, the synthetic video can be a synthetic video with a virtual character image (the virtual character image can be a virtual customer service staff displayed on the online customer service interface, or a virtual teacher displayed on the online classroom interface, etc.), and for another example, the synthetic video can also be a synthetic video with a virtual cartoon image.

[0004] However, in the related art, if the number of materials for video synthesis is too large, a lot of time will be needed to retrieve the materials for video synthesis during video synthesis, which will lead to a decrease in the efficiency of video synthesis. Therefore, how to improve the efficiency of video synthesis is a problem that needs to be solved urgently. Summary of the invention

[0005] In view of this, embodiments of the present application provide a video data storage method, device, electronic device and readable storage medium to improve the efficiency of video synthesis.

[0006] In a first aspect, a video data storage method is provided, the method being applied to an electronic device, the method comprising:

[0007] Get multiple video clips.

[0008] Frame pair information between a plurality of video segments is determined, where the frame pair information is used to represent an association relationship between two corresponding video segments.

[0009] Each of the video clips is used as a video node, each frame pair information is used as the association relationship between each video node, and each of the video clips and each frame pair information is stored.

[0010] In a second aspect, a video data storage device is provided, the device being applied to an electronic device, the device comprising:

[0011] The acquisition module is used to acquire multiple video clips.

[0012] The frame pair information module is used to determine the frame pair information between multiple video clips, and the frame pair information is used to represent the association relationship between two corresponding video clips.

[0013] The storage module is used to store the video clips and the frame pair information by taking the video clips as video nodes and the frame pair information as the association relationship between the video nodes.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in the first aspect is implemented.

[0016] Through the embodiments of the present application, before storing video clips, the electronic device for video data storage can first determine the frame pair information between multiple video clips, wherein the frame pair information can be used to characterize that there is a strong correlation between the corresponding two video clips. Then, each video clip can be used as a video node, and each frame pair information can be used as the correlation relationship between each video node to store each video clip and each frame pair information. In this way, the correlation relationship between each video clip can be retained when storing each video clip, and then, when the video is subsequently synthesized, the video synthesis material can be quickly retrieved based on the above correlation relationship, thereby improving the efficiency of video synthesis. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other purposes, features and advantages of the embodiments of the present application will become more apparent through the following description of the embodiments of the present application with reference to the accompanying drawings, in which:

[0018] Figure 1 A schematic diagram of a video data storage system according to an embodiment of the present application;

[0019] Figure 2 This is a flowchart of a video data storage method according to an embodiment of the present application;

[0020] Figure 3 A flowchart of a process for determining a video segment according to an embodiment of the present application;

[0021] Figure 4 A schematic diagram of a database of an embodiment of the present application;

[0022] Figure 5 A schematic diagram of a graphical database according to an embodiment of the present application;

[0023] Figure 6 A flowchart of a video synthesis process is determined for an embodiment of the present application;

[0024] Figure 7 A flowchart for determining a target video segment according to an embodiment of the present application;

[0025] Figure 8 This is a schematic diagram of the structure of a video data storage device according to an embodiment of the present application;

[0026] Fig. 9 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The present application is described below based on embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, some specific details are described in detail. It is possible for those skilled in the art to fully understand the present application without the description of these details. In order to avoid confusing the essence of the present application, known methods, processes, flows, components and circuits are not described in detail.

[0028] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0029] Unless the context clearly requires otherwise, words such as “include”, “including” and the like in the specification should be interpreted as including rather than being exclusive or exhaustive; that is, as meaning “including but not limited to”.

[0030] In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.

[0031] In order to solve the above problems, the present application embodiment provides a video data storage system, specifically, Figure 1 As shown, Figure 1 This is a schematic diagram of a video data storage system according to an embodiment of the present application, which includes: a video clip set 11 , an electronic device 12 for storing video data, and a database 13 .

[0032] The video segment set 11 includes multiple video segments (video segment 111, video segment 112, video segment 113 and video segment 114). In the embodiment of the present application, each video segment in the video segment set 11 can be used as a video segment to be stored.

[0033] It should be noted that the number of video clips included in the video clip set 11 is a natural number greater than or equal to 2, and the embodiment of the present application does not limit the number of video clips.

[0034] The electronic device 12 may be a terminal or a server. The terminal may be a smart phone, a tablet computer or a personal computer (PC), etc. The server may be a single server, a server cluster configured in a distributed manner, or a cloud server.

[0035] The database 13 may include a plurality of video clips as synthesis materials. In the embodiment of the present application, if the electronic device 12 stores each video clip in the video clip set 11 into the database 13, the stored video clips may be used as materials for subsequent video synthesis.

[0036] During the process of storing video data, the electronic device 12 can receive each video segment in the video segment set 11, and then determine the frame pair information between each video segment in the video segment set 11, wherein the frame pair information can be used to characterize the association relationship between two video segments, and the association relationship can be used to characterize that the corresponding two video segments have a strong association, that is, two video segments with frame pair information can be spliced ​​and synthesized.

[0037] It should be noted that in the database 13, not every two video clips have frame pair information. That is, the database 13 may include two video clips with frame pair information or two video clips without frame pair information.

[0038] After the electronic device 12 determines the frame pair information, each video segment can be used as a video node, and each frame pair information can be used as the association relationship between each video node, and each video segment and each frame pair information can be stored in the database 13.

[0039] When performing video synthesis subsequently, the electronic device for video synthesis can quickly determine the material for video synthesis according to the association relationship between the video clips in the database 13 .

[0040] Through the embodiments of the present application, before storing video clips, the electronic device for video data storage can first determine the frame pair information between multiple video clips, wherein the frame pair information can be used to characterize that there is a strong correlation between the corresponding two video clips. Then, each video clip can be used as a video node, and each frame pair information can be used as the correlation relationship between each video node to store each video clip and each frame pair information. In this way, the correlation relationship between each video clip can be retained when storing each video clip, and then, when the video is subsequently synthesized, the video synthesis material can be quickly retrieved based on the above correlation relationship, thereby improving the efficiency of video synthesis.

[0041] The video data storage method provided by the embodiment of the present application will be described in detail below in conjunction with specific implementation methods. Figure 2 As shown, the specific steps are as follows:

[0042] In step 21, a plurality of video clips are obtained.

[0043] The multiple video clips in step 21 may be different video clips from the same video, or may be different video clips from different videos.

[0044] In a preferred implementation, the embodiment of the present application can determine each video segment for an original video.

[0045] Specifically, the process may be performed as follows: determining an original video, performing target detection on the original video, determining target frames, and merging target frames based on intervals between target frames to determine video segments.

[0046] Among them, the original video is a video suitable for this application. For example, the original video can be a video recorded by a camera device. After the video is legally authorized, the embodiment of this application can use the video to determine the video segment.

[0047] In an online classroom scenario, the original video may be a video recorded by filming a teacher using a camera. After the online classroom platform receives the original video, it may determine the video clip based on the original video.

[0048] In addition, the target frame at least includes a detection object corresponding to the target detection, and the detection object can be determined by a pre-trained target detection model. Specifically, the target detection model can detect whether the video frame in the original video contains the detection object by performing region selection, feature extraction and feature classification on the video frame in the original video.

[0049] In an embodiment of the present application, the result of target detection can be used to characterize whether the detected target (i.e., the detection object) exists in each video frame of the original video. Specifically, the result of target detection can be represented by a numerical value. If the target detection result is greater than 0, it represents that the detected target exists in the video frame of the original video, otherwise it does not exist. Of course, the result of target detection can also be represented in other ways. For example, the result of target detection can be represented by a classification result, wherein the classification result can include "yes" and "no". The classification result of "yes" can represent the presence of the detected target in the video frame of the original video, and the classification result of "no" can represent the absence of the detected target in the video frame of the original video.

[0050] It should be noted that the embodiment of the present application can perform target detection on each video frame of the original video at the same time, or can perform target detection on each video frame of the original video one by one.

[0051] After determining each target frame, the embodiment of the present application can merge each target frame to determine each video segment.

[0052] Among them, each video segment determined in the embodiment of the present application can be used for subsequent video synthesis.

[0053] In practical applications, the embodiment of the present application can determine multiple target frames with relatively coherent content as a video segment. On the contrary, if the interval between two target frames is too far, the content representing the two target frames is likely to be incoherent. Therefore, the embodiment of the present application can regard the two target frames as video frames in two video segments.

[0054] In an embodiment of the present application, a target frame with a detection object can be selected based on the target detection of the original video. Then, the embodiment of the present application can merge each target frame based on the interval between each target frame to determine each video clip, wherein, since the length of the interval between the target frames can characterize whether the target frames are continuous, therefore, through the interval between each target frame, the embodiment of the present application can determine a plurality of video clips with coherent content. In this way, through the embodiment of the present application, it can be determined that both the detection object and the plurality of video clips with coherent content can be obtained, so that a higher quality composite video can be obtained when the subsequent video synthesis is performed.

[0055] On the other hand, after determining each video segment, the embodiment of the present application can also determine the category corresponding to each video segment. Specifically, the process can be performed as follows: determining the detection object corresponding to each target frame in the video segment, and taking the category of the detection object with the most occurrences as the video category of the video segment.

[0056] The video category can be represented by the category of the detected object after target detection, for example, the video category can include "OK", "waving", "nodding", etc. If the video category corresponding to the video segment A is "waving", it is characterized that the main content of the video segment A is related to the waving action.

[0057] Further, after the video category of the video clip is determined, the video clip, the frame pair information, and the video category may be stored together.

[0058] By the embodiment of the present application, it is possible to select a target frame with a detection object based on the target detection of the original video. Then, the embodiment of the present application can merge each target frame based on the interval between each target frame, determine each video clip and store it by classification, wherein, since the length of the interval between the target frames can characterize whether the target frames are continuous, therefore, by the interval between each target frame, the embodiment of the present application can determine a plurality of video clips with coherent content. In this way, it is possible to determine by the embodiment of the present application that both the detection object and a plurality of video clips to be processed with coherent content, so that a composite video with higher quality can be obtained when performing subsequent video synthesis. In addition, since the embodiment of the present application can also store video clips by classification, so, by the embodiment of the present application, it is also possible to have a higher video clip retrieval efficiency when performing subsequent video synthesis.

[0059] Combining the above method steps, such as Figure 3 As shown, Figure 3 This is a flowchart of a process of determining a video segment according to an embodiment of the present application, which specifically includes the following steps:

[0060] In step 31, the original video is determined.

[0061] Among them, the original video is a video suitable for this application. For example, the original video can be a video recorded by a camera device. After the video is legally authorized, the embodiment of this application can use the video to determine the video segment.

[0062] In step 32, target detection is performed on the original video to determine the target detection result.

[0063] Among them, the target detection result can be used to characterize whether the detected target exists in each video frame of the original video. Specifically, the target detection result can be represented by a numerical value. If the target detection result is greater than 0, it indicates that the detected target exists in the video frame of the original video, otherwise it does not exist.

[0064] It should be noted that the embodiment of the present application can perform target detection on each video frame of the original video at the same time, or can perform target detection on each video frame of the original video one by one. Figure 3 The following will be described in terms of detecting targets one by one.

[0065] In step 33 , it is determined whether the target detection result is greater than 0. If the target detection result is greater than 0, step 34 is executed. If the target detection result is less than or equal to 0, step 31 is executed.

[0066] In practical applications, the target detection result is generally represented by "0" and "1", where "0" is used to indicate that the corresponding video frame does not contain the detected object, and "1" is used to indicate that the corresponding video frame contains the detected object. That is, in step 33, if the target detection result is greater than 0, the target detection result is represented as "1", that is, the corresponding video frame contains the detected object.

[0067] In step 34, the target frame is determined, and the frame number of the target frame is added to the predetermined list.

[0068] In the embodiment of the present application, the number of video frames between adjacent target frames is used as an example to represent the interval between the two. The predetermined list is used to store the frame numbers of the target frames, that is, the video frames corresponding to the frame numbers in the predetermined list all have the detected target.

[0069] In step 35, the intervals between adjacent target frames in the predetermined list are determined.

[0070] in, Figure 3 The interval between adjacent target frames is represented by the number of video frames between adjacent target frames. In practical applications, the interval between adjacent target frames can also be represented by the time interval between adjacent target frames, or by other applicable methods.

[0071] In step 36, it is determined whether the interval between adjacent target frames is less than the interval threshold. If the interval between adjacent target frames is less than the interval threshold, step 37 is executed. If the interval between adjacent target frames is greater than or equal to the interval threshold, step 38 is executed.

[0072] The interval threshold in the judgment condition corresponding to step 36 may be represented by a numerical value. For example, the interval threshold may be 1 frame, 2 frames, 3 frames, 4 frames, and so on.

[0073] In step 37, the target frame is added to the temporary list.

[0074] In the embodiment of the present application, the temporary list is used to store target frames that meet the corresponding conditions of step 36, that is, each target frame stored in the temporary list can be used to form a continuous video clip.

[0075] In step 38, video segments are generated based on the temporary list.

[0076] In the process of generating video segments based on the temporary list, the target frames stored in the temporary list can be interpolated. Specifically, if there is a video frame interval between adjacent target frames, interpolated frames can be performed between the adjacent target frames to make the video segments have good continuity.

[0077] In step 39, the category of the video segment is determined and the video segment is stored.

[0078] By the embodiment of the present application, it is possible to select a target frame with a detection object based on the target detection of the original video. Then, the embodiment of the present application can merge each target frame based on the interval between each target frame, determine each video clip and store it by classification, wherein, since the length of the interval between the target frames can characterize whether the target frames are continuous, therefore, by the interval between each target frame, the embodiment of the present application can determine a plurality of video clips with coherent content. In this way, it is possible to determine by the embodiment of the present application that both the detection object and a plurality of video clips to be processed with coherent content, so that a composite video with higher quality can be obtained when performing subsequent video synthesis. In addition, since the embodiment of the present application can also store video clips by classification, so, by the embodiment of the present application, it is also possible to have a higher video clip retrieval efficiency when performing subsequent video synthesis.

[0079] In step 22, frame pair information between a plurality of video clips is determined.

[0080] The frame pair information is used to characterize the association relationship between the corresponding two video clips. Specifically, the frame pair information can be determined by the synthesis evaluation parameters between the two video clips to be processed, and the synthesis evaluation parameters can include pixel similarity, color similarity, ratio similarity, optical flow value, etc. The embodiment of the present application can use one or more of these parameters to perform a synthesis evaluation on whether the two video clips to be processed can be spliced.

[0081] In a preferred embodiment, the above step 22 can be executed as follows: determining the first video segment and the second video segment from each video segment, calculating the synthetic evaluation parameters between each first video frame and each second video frame respectively, and generating frame pair information between the first video segment and the second video segment in response to the synthetic evaluation parameters satisfying predetermined conditions.

[0082] The first video segment includes at least one first video frame, the second video segment includes at least one second video frame, and the synthesis evaluation parameter includes at least one of pixel similarity, color similarity, ratio similarity and optical flow value.

[0083] In the embodiment of the present application, the frame pair information is information used to characterize that two video clips can be spliced. Therefore, it can be determined whether the two video clips can be spliced ​​by using the video frames at appropriate positions in the two video clips.

[0084] For example, the first video frame may be any frame in the last n frames of the first video segment, and the second video frame may be any frame in the first m frames of the second video segment. Of course, the first video frame may also be any frame in the first n frames of the first video segment, and the second video frame may also be any frame in the last m frames of the second video segment. Wherein, m and n are natural numbers, and their values ​​may be set according to actual conditions.

[0085] After determining each first video frame and each second video frame, the embodiment of the present application can determine the synthesis evaluation parameters between each first video frame and each second video frame respectively. Based on the synthesis evaluation parameters, the embodiment of the present application can evaluate the correlation between each first video frame and each second video frame. If the synthesis evaluation parameters meet predetermined conditions, frame pair information between the first video segment and the second video segment can be generated.

[0086] Taking the color similarity in the synthesis evaluation parameters as an example, the color similarity can be used to perform a synthesis evaluation on the two video clips based on the color difference between the two video clips. If the color difference between the two video clips does not exceed the predetermined difference threshold, it means that the two video clips can be spliced. Furthermore, a frame pair information can be generated for the two video clips to characterize the association relationship between the two video clips.

[0087] In addition, if the composite evaluation parameters include multiple parameters, in one case, frame pair information is generated in response to any one of the composite evaluation parameters (ie, the composite evaluation parameters between any first video frame and any second video frame) satisfying the predetermined condition.

[0088] In another case, in response to a predetermined ratio of parameters in the composite evaluation parameters satisfying the predetermined condition, the frame pair information is generated, wherein the predetermined ratio may be a ratio set according to actual conditions, such as 50%, 70%, 90%, etc.

[0089] In another case, in response to all parameters in the composite evaluation parameters satisfying the predetermined condition, the frame pair information is generated.

[0090] By setting the frame pair information, it can be ensured that there is correlation between two adjacent materials during the video synthesis process, which increases the overall smoothness of the synthesized video. Compared with directly synthesized videos, the embodiment of the present application can obtain a synthesized video with advantages such as coherence and smoothness based on the frame pair information.

[0091] In step 23, each video segment is used as a video node, each frame pair information is used as the association relationship between each video node, and each video segment and each frame pair information are stored.

[0092] Through the embodiments of the present application, before storing video clips, the electronic device for video data storage can first determine the frame pair information between multiple video clips, wherein the frame pair information can be used to characterize that there is a strong correlation between the corresponding two video clips. Then, each video clip can be used as a video node, and each frame pair information can be used as the correlation relationship between each video node to store each video clip and each frame pair information. In this way, the correlation relationship between each video clip can be retained when storing each video clip, and then, when the video is subsequently synthesized, the video synthesis material can be quickly retrieved based on the above correlation relationship, thereby improving the efficiency of video synthesis.

[0093] In a preferred embodiment, when the embodiment of the present application stores video clips and frame pair information, the video categories corresponding to the video clips can also be stored. Specifically, step 23 can be performed as follows: determine the video categories corresponding to each video clip, and store each video clip, each frame pair information, and each video category using each video clip as a video node, each frame pair information as the association relationship between each video node, each video category as a category node, and the category belonging relationship as the association relationship between each video node and the category node.

[0094] That is to say, after the embodiment of the present application stores each video clip, the database at least includes each video clip, each frame pair information and each video category, for example, Figure 4 As shown, Figure 4 This is a schematic diagram of a database according to an embodiment of the present application, which includes: a database 41, category A, category B, category C, category D and frame pair information under the database 41.

[0095] Each category includes a plurality of video clips, and the association between the video clips can be represented by frame pair information in the database 41 .

[0096] pass Figure 4 The database shown can store each video clip, frame pair information between each video clip, and the video category corresponding to each video clip. When video synthesis is performed subsequently, fast retrieval of video synthesis materials can be achieved based on the frame pair information, thereby improving the efficiency of video synthesis.

[0097] In another preferred embodiment, the database for storing video clips may be a graph database (Graph Database, Graph DB). Specifically, the above step 23 may be performed as follows: a database is established with each video clip as a video node and each frame pair information as the association relationship between each video node.

[0098] like Figure 5 As shown, Figure 5 A schematic diagram of a graph database provided in an embodiment of the present application, the schematic diagram includes multiple nodes (node ​​51-node 58) and association relationships between the nodes.

[0099] Each node may correspond to a video clip. Figure 5 The arrow in is used to represent that there is an association relationship between two nodes, and the direction of the arrow is used to represent the order of video splicing in the corresponding association relationship.

[0100] It should be noted that Figure 5 In the illustrated graph database, there may be two nodes that have no association relationship. For example, there is no direct association relationship between the node 54 and the node 56 .

[0101] Therefore, in the embodiment of the present application, there may be an association relationship between the nodes in the graph database, or there may not be an association relationship. However, each node in the above graph database has an association relationship with at least one other node.

[0102] In the embodiment of the present application, the relationship between each video clip in the graph database can be clearly and concisely represented through the nodes and associations in the graph database. In addition, since the graph database has the advantage of simple structure compared with the traditional database, the graph database can realize fast storage and fast query. Furthermore, in the embodiment of the present application, when it is necessary to store or retrieve a large number of video clips and a large number of frame pair information, the graph database can be used to realize fast storage of video clips and fast retrieval of video clips.

[0103] In another case, the graph database may also include video segment category nodes. In this case, some nodes in the graph database may correspond to a video segment, and another part of nodes may correspond to a video segment category. In this case, the node corresponding to the video segment category is the video segment category node. The video segment category node may correspond to at least one video segment (i.e., may correspond to a node of at least one video segment).

[0104] When an electronic device performs retrieval based on video clip categories, by setting video clip category nodes in an image database, the electronic device can quickly retrieve video clips under corresponding categories, thereby improving the efficiency of video retrieval.

[0105] On the other hand, after the video segments are determined and stored, if the electronic device for video synthesis receives a video synthesis instruction, the electronic device may determine the synthesized video according to the received video synthesis instruction and the video segments.

[0106] Specifically, Figure 6As shown, the process of determining the composite video may include the following steps:

[0107] In step 61 , in response to receiving a video synthesis instruction, a plurality of target video segments are determined according to the video synthesis instruction.

[0108] The video synthesis instruction is used to specify the connection order of each target video segment.

[0109] It can be seen from the above method steps that the database for storing video clips includes each video clip and frame pair information between each video clip.

[0110] In a preferred embodiment, the database for storing video clips may also include video categories corresponding to each video clip, and then step 61 may be executed as follows: in response to receiving a video synthesis instruction, determining each target category identifier in the video synthesis instruction, and determining each target video clip under the category corresponding to each target category identifier based on each target category identifier and frame pair information.

[0111] The target category identifier is used to specify the category corresponding to the target video clip.

[0112] Specifically, Figure 7 As shown, Figure 7 A flowchart of determining a target video segment according to an embodiment of the present application.

[0113] In the process of determining the target video segment, the electronic device 72 for video synthesis may receive a video synthesis instruction 71, wherein the video synthesis instruction 71 includes a target category identifier (category A, category C, and category D) and a specified connection order (CDA).

[0114] When the electronic device 72 receives the video synthesis instruction 71 , it can retrieve and obtain the corresponding video segment from the database 73 as the target video segment 74 according to the target category identifier and the connection sequence of the video segments in the video synthesis instruction 71 .

[0115] The database 73 includes video clips of multiple categories and frame pair information corresponding to each video clip, and the target video clip 74 includes video clips a1, a3, c1, c2 and d3. In addition, the number of categories in the database 73 is not limited to Figure 7 4 categories shown.

[0116] Depend on Figure 7It can be seen that the frame pair information in the database 73 can be used to represent the association relationship between two video clips, wherein the association relationship can represent that the two video clips can be spliced, and the association relationship can also include the connection order of the two video clips. For example, the frame pair information "a1-b1" can be used to represent that the video clip a1 and the video clip b1 can be spliced, and the connection order of the video clip a1 and the video clip b1 is a1 first and b1 second.

[0117] Based on the above Figure 7 As shown in the content, the electronic device 72 can determine each target video segment 74 from the database 73 based on the target category identifier in the video synthesis instruction 71, the video segment connection order specified by the video synthesis instruction 71 and the frame pair information in the database 73.

[0118] in addition, Figure 7 The database 73 in can also be as follows Figure 5 The graph database shown.

[0119] In another preferred embodiment, the video synthesis instruction can also directly specify the video segment. Specifically, the process can be executed as follows: in response to receiving the video synthesis instruction, determining each target video identifier in the video synthesis instruction, and according to each target video identifier, determining the target video segment corresponding to each target video identifier.

[0120] The target video identifier is used to specify the target video segment.

[0121] In step 62, a synthesis operation is performed on each target video segment based on the connection order specified by the video synthesis instruction to determine a synthesized video.

[0122] Through the embodiments of the present application, since frame pair information between video clips is also stored when storing video clips, there is a strong correlation between two adjacent target video clips in the synthesized video, so that the synthesized video has a higher degree of flow.

[0123] Based on the same technical concept, the embodiment of the present application also provides a video data storage device, such as Figure 8 As shown, the device includes: an acquisition module 81, a frame pair information module 82 and a storage module 83.

[0124] The acquisition module 81 is used to acquire multiple video clips.

[0125] The frame pair information module 82 is used to determine frame pair information between multiple video segments, where the frame pair information is used to represent the association relationship between two corresponding video segments.

[0126] The storage module 83 is used to store the video clips and the frame pair information by taking the video clips as video nodes and the frame pair information as association relationships between the video nodes.

[0127] In some preferred embodiments, the storage module 83 is specifically used for:

[0128] Determine the video categories corresponding to the video clips.

[0129] Each video clip is used as a video node, each frame pair information is used as the association relationship between each video node, each video category is used as a category node, and the category belonging relationship is used as the association relationship between each video node and the category node. Each video clip, each frame pair information and each video category are stored.

[0130] In some preferred embodiments, the storage module 83 is specifically used for:

[0131] A database is established by taking each of the video clips as a video node and taking each frame pair information as the association relationship between each of the video nodes. The database is a graph database.

[0132] In some preferred embodiments, the frame pair information module 82 is specifically used to:

[0133] A first video segment and a second video segment are determined from the video segments, the first video segment including at least one first video frame, and the second video segment including at least one second video frame.

[0134] A synthesis evaluation parameter is calculated between each of the first video frames and each of the second video frames, wherein the synthesis evaluation parameter includes at least one of pixel similarity, color similarity, ratio similarity and optical flow value.

[0135] In response to the synthesis evaluation parameter satisfying a predetermined condition, frame pair information between the first video segment and the second video segment is generated.

[0136] In some preferred embodiments, the acquisition module 81 is specifically used for:

[0137] Determine the original video.

[0138] Target detection is performed on the original video to determine each target frame, where the target frame at least includes a detection object corresponding to the target detection.

[0139] The target frames are merged based on the intervals between the target frames to determine the video segments.

[0140] In some preferred embodiments, the device further comprises:

[0141] The first determination module is used to determine the detection object corresponding to each target frame in the video segment.

[0142] The video category module is used to use the category of the detected object that appears most frequently as the video category of the video clip.

[0143] In some preferred embodiments, the device further comprises:

[0144] The second determination module is configured to, in response to receiving a video synthesis instruction, determine a plurality of target video segments according to the video synthesis instruction, wherein the video synthesis instruction is used to specify a connection order of the target video segments.

[0145] The synthesis module is used to perform synthesis operations on each target video segment based on the connection sequence specified by the video synthesis instruction to determine the synthesized video.

[0146] In some preferred embodiments, the second determining module is specifically configured to:

[0147] In response to receiving the video synthesis instruction, each target category identifier in the video synthesis instruction is determined, and the target category identifier is used to specify the category corresponding to the target video segment.

[0148] According to the target category identifiers and the frame pair information, target video segments under the categories respectively corresponding to the target category identifiers are determined.

[0149] Through the embodiments of the present application, before storing video clips, the electronic device for video data storage can first determine the frame pair information between multiple video clips, wherein the frame pair information can be used to characterize that there is a strong correlation between the corresponding two video clips. Then, each video clip can be used as a video node, and each frame pair information can be used as the correlation relationship between each video node to store each video clip and each frame pair information. In this way, the correlation relationship between each video clip can be retained when storing each video clip, and then, when the video is subsequently synthesized, the video synthesis material can be quickly retrieved based on the above correlation relationship, thereby improving the efficiency of video synthesis.

[0150] Fig. 9 Schematic diagram of an electronic device according to an embodiment of the present application. Fig. 9 As shown, Fig. 9The electronic device shown is a general address query device, which includes a general computer hardware structure, which at least includes a processor 91 and a memory 92. The processor 91 and the memory 92 are connected via a bus 93. The memory 92 is suitable for storing instructions or programs executable by the processor 91. The processor 91 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 91 executes the instructions stored in the memory 92, thereby executing the method flow of the embodiment of the present application as described above to realize the processing of data and the control of other devices. The bus 93 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 94 and the display device and the input / output (I / O) device 95. The input / output (I / O) device 95 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 95 is connected to the system via an input / output (I / O) controller 96.

[0151] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0152] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0153] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0154] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0155] Another embodiment of the present application relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above method embodiments.

[0156] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by specifying relevant hardware through a program, and the program is stored in a storage medium, including several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0157] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A video data storage method, characterized in that: The method comprises: Get multiple video clips; Determining frame pair information between a plurality of video segments, wherein the frame pair information is used to represent an association relationship between two corresponding video segments; and Taking each of the video clips as a video node and each frame pair information as an association relationship between each of the video nodes, each of the video clips and each frame pair information is stored; The storing of each video segment and each frame pair information as the association relationship between each video node includes: Determine the video category corresponding to each video clip; and Taking each of the video clips as a video node, taking each frame pair information as an association relationship between each video node, taking each video category as a category node, taking the category belonging relationship as an association relationship between each video node and the category node, storing each of the video clips, each frame pair information and each video category; The step of obtaining multiple video clips includes: Determine the original video; Performing target detection on the original video to determine each target frame, wherein the target frame at least includes a detection object corresponding to the target detection; Merging target frames based on intervals between target frames to determine video segments; Wherein, the video category is determined according to the detection object corresponding to each target frame in the video clip; The determining of frame pair information between the plurality of video clips comprises: Determine a first video segment and a second video segment from the video segments, the first video segment includes at least one first video frame, and the second video segment includes at least one second video frame; Calculating synthesis evaluation parameters between each of the first video frames and each of the second video frames, the synthesis evaluation parameters comprising pixel similarity, color similarity, ratio similarity and optical flow value; and In response to the synthesis evaluation parameter satisfying a predetermined condition, frame pair information between the first video segment and the second video segment is generated.

2. The method according to claim 1, characterized in that The method of taking each video segment as a video node, taking each frame pair information as the association relationship between each video node, and storing each video segment and each frame pair information includes: A database is established by taking each of the video clips as a video node and taking each frame pair information as the association relationship between each of the video nodes. The database is a graph database.

3. The method according to claim 1, characterized in that The method further comprises: Determining a detection object corresponding to each target frame in the video clip; and The category of the detected object that appears most frequently is used as the video category of the video clip.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: In response to receiving a video synthesis instruction, determining a plurality of target video segments according to the video synthesis instruction, wherein the video synthesis instruction is used to specify a connection order of the target video segments; and Based on the connection order specified by the video synthesis instruction, a synthesis operation is performed on each target video segment to determine a synthesized video.

5. The method according to claim 4, characterized in that In response to receiving the video synthesis instruction, determining a plurality of target video segments according to the video synthesis instruction includes: In response to receiving the video synthesis instruction, determining each target category identifier in the video synthesis instruction, wherein the target category identifier is used to specify a category corresponding to the target video segment; and According to the target category identifiers and the frame pair information, target video segments under categories corresponding to the target category identifiers are determined.

6. A video data storage device, characterized in that: The device comprises: An acquisition module, used for acquiring multiple video clips; a frame pair information module, used to determine frame pair information between a plurality of video segments, wherein the frame pair information is used to represent an association relationship between two corresponding video segments; and A storage module, used to store the video clips and the frame pair information by taking the video clips as video nodes and the frame pair information as association relationships between the video nodes; The storing of each video segment and each frame pair information as the association relationship between each video node includes: Determine the video category corresponding to each video clip; and Taking each of the video clips as a video node, taking each frame pair information as an association relationship between each video node, taking each video category as a category node, taking the category belonging relationship as an association relationship between each video node and the category node, storing each of the video clips, each frame pair information and each video category; The step of obtaining multiple video clips includes: Determine the original video; Performing target detection on the original video to determine each target frame, wherein the target frame at least includes a detection object corresponding to the target detection; Merging target frames based on intervals between target frames to determine video segments, wherein the video category is determined according to the detection object corresponding to each target frame in the video segment; The determining of frame pair information between the plurality of video clips comprises: Determine a first video segment and a second video segment from the video segments, the first video segment includes at least one first video frame, and the second video segment includes at least one second video frame; Calculating synthesis evaluation parameters between each of the first video frames and each of the second video frames, the synthesis evaluation parameters comprising pixel similarity, color similarity, ratio similarity and optical flow value; and In response to the synthesis evaluation parameter satisfying a predetermined condition, frame pair information between the first video segment and the second video segment is generated.

7. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Video processing method and device and electronic device

    CN106331479A

  • Method and apparatus for processing face recognition result based on video semantics

    CN109063611A

  • Video semantic analysis method and device, storage medium and electronic equipment

    CN112712056A

  • Method and apparatus for processing video

    US20190377956A1