Strategy determination method and electronic device for generating video

Abstract

The present application relates to the field of video processing, and provides a strategy determination method and electronic device for generating a video, which obtains a first duration, a second duration, and a video analysis rate, and then determines a target analysis strategy corresponding to a candidate video from an analysis strategy set based on the first duration, the second duration, and the video analysis rate, and then analyzes and processes the selected video based on the target analysis strategy to obtain analysis results of each selected video, and finally generates a target video based on the analysis results of multiple selected videos. That is to say, the target analysis strategy corresponding to the candidate video is determined from multiple analysis strategies through the video duration of the candidate video, the analysis duration allocated to the candidate video for analysis and processing, and the video analysis rate of the electronic device itself for analyzing the video. Compared with the traditional technology of analyzing and processing all frame images in the candidate video, the time occupied for analyzing and processing the candidate video is shortened, thereby improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of video, and in particular, to a strategy determination method for generating video and an electronic device. Background Art

[0002] In some electronic devices, a "one-click blockbuster" function is provided to users. When the user clicks the "one-click blockbuster" control, the electronic device automatically selects the highlight segments of each video and / or picture from multiple videos and / or pictures selected by the user, and automatically generates a new video based on the selected highlight segments.

[0003] In the process of obtaining the highlight clips of the candidate video, the frame images in the video are usually analyzed to obtain the analysis results of each frame image, and then a new video is automatically generated based on the analysis results of each frame image. However, the number of frame images in the video is usually large, and the time required to analyze the frame images in the video is usually long, which will result in a poor user experience.

[0004] Based on this, how to reduce the analysis time of frame images in the video and improve user experience has become an urgent problem to be solved. Summary of the invention

[0005] The present application provides a strategy determination method for generating a video and an electronic device, which can improve the user experience when executing a one-click blockbuster.

[0006] In a first aspect, a method for determining a strategy for generating a video is provided, which is applied to an electronic device, and the method includes:

[0007] Obtaining a first duration, a second duration, and a video analysis rate, wherein the first duration refers to the video duration of the candidate video, and the second duration is used to indicate the analysis duration allocated to the candidate video for analysis processing;

[0008] Determining a target analysis strategy corresponding to the candidate video from the analysis strategy set based on the first duration, the second duration, and the video analysis rate, wherein the analysis strategy set includes at least two analysis strategies, and different analysis strategies are used to perform analysis on the candidate video for different durations;

[0009] Analyze and process the candidate videos based on the target analysis strategy to obtain analysis results of the candidate videos;

[0010] The target video is obtained according to the analysis results of the candidate videos.

[0011] It should be understood that different analysis strategies among at least two analysis strategies are to select all or part of the frame images from the candidate video for analysis and processing. Therefore, the time taken to analyze and process the candidate video using at least two analysis strategies is shorter than the traditional method of analyzing and processing all the frame images in the candidate video.

[0012] In an embodiment of the present application, by obtaining the first duration, the second duration and the video analysis rate, then determining the target analysis strategy corresponding to the candidate video from the analysis strategy set based on the first duration, the second duration and the video analysis rate, and then analyzing and processing the selected video based on the target analysis strategy to obtain the analysis results of each selected video, and finally, generating the target video according to the analysis results of multiple selected videos. Among them, the first duration refers to the video duration of the candidate video, the second duration is used to indicate the analysis duration allocated to the candidate video for analyzing and processing the candidate video, and the analysis strategy set includes at least two analysis strategies, and different analysis strategies are used to analyze and process the candidate video. The required duration is different. That is to say, the target analysis strategy corresponding to the candidate video is determined from multiple analysis strategies through the video duration of the candidate video, the analysis duration allocated to the candidate video for analysis and processing, and the video analysis rate of the electronic device itself for analyzing the video. Compared with the traditional technology of analyzing and processing all frame images in the candidate video, the time occupied for analyzing and processing the candidate video is shortened, so the analysis results of the candidate video can be obtained faster, and then the target video can be obtained faster, thereby improving the user experience.

[0013] Furthermore, since the second duration is the analysis time allocated to the candidate video for analysis and processing, it is equivalent to the electronic device determining the analysis time corresponding to each candidate video in advance according to the user's acceptable waiting time (that is, the waiting time that will not lead to a deterioration in user experience), so that the total analysis time required for the analysis and processing of each candidate video according to the allocated analysis time will not exceed the user's acceptable waiting time, and will not lead to a poor user experience due to excessive waiting time, thereby improving the user experience in the process of one-click blockbuster.

[0014] In combination with the first aspect, in an embodiment of the first aspect, determining a target analysis strategy corresponding to the candidate video from the analysis strategy set based on the first duration, the second duration, and the video analysis rate includes:

[0015] A first ratio is determined based on the first duration, the second duration and the video analysis rate, the first ratio being used to indicate the percentage of frame images in the candidate video that have completed analysis processing within the second duration to the total frame images in the candidate video; a target analysis strategy is determined based on the first ratio.

[0016] In combination with the first aspect, in an embodiment of the first aspect, determining a target analysis strategy corresponding to the candidate video from the analysis strategy set based on the first duration, the second duration, and the video analysis rate includes:

[0017] A third duration is determined based on the first duration and the video analysis rate, where the third duration is used to indicate the time required to complete the analysis and processing of all frame images in the candidate video. A target analysis strategy corresponding to the candidate video is determined based on the third duration and the second duration.

[0018] In combination with the first aspect, in an embodiment of the first aspect, at least two analysis strategies include a full analysis strategy, a dense key segment analysis strategy, a sparse key frequency band analysis strategy, and an I frame sampling analysis strategy.

[0019] Among them, different analysis strategies are used to analyze and process the same candidate video. The full analysis strategy takes the longest time, the intensive key segment analysis strategy takes the second longest time, the sparse key segment analysis strategy takes the third longest time, and the I-frame sampling analysis strategy takes the shortest time.

[0020] In combination with the first aspect, in an embodiment of the first aspect, the full analysis strategy refers to a strategy of analyzing and processing the frame image of each frame in the candidate video.

[0021] It should be understood that since the full analysis strategy is a strategy for analyzing and processing the frame image of each frame in the candidate video, the time taken to analyze and process the candidate video using the full analysis strategy is the longest.

[0022] In combination with the first aspect, in an embodiment of the first aspect, the intensive key segment analysis strategy refers to a strategy of dividing the candidate video into N analysis segments with the same interval duration, and performing analysis and processing on the N analysis segments, where N is a positive integer greater than 0.

[0023] Compared with the sparse key segment analysis strategy, the intensive key segment analysis strategy divides the candidate video into N analysis segments of the same duration, and the interval duration between two adjacent analysis segments is the same.

[0024] It should be understood that compared with the full analysis strategy that decodes, reduces the resolution and scores all video frames, the intensive key segment analysis strategy only needs to decode, reduce the resolution and score the video frames of a few key segments. Therefore, the time taken to analyze and process candidate videos using the intensive key segment analysis strategy is shorter than the time taken to analyze and process candidate videos using the full analysis strategy.

[0025] In combination with the first aspect, in an embodiment of the first aspect, the sparse key segment analysis strategy refers to a strategy of dividing the candidate video into N analysis segments and performing analysis and processing on the N analysis segments, wherein the intervals between the N analysis segments are of different durations.

[0026] It should be understood that the sparse key segment analysis strategy has the following two characteristics:

[0027] 1. The first analysis segment may not be the first frame of the video. For example, the first analysis segment needs to select a frame from the first video frame at the starting point of the first video, the video frame at the 2nd second position of the first video, and the video frame at the 1 / 5 position of the first video as the first frame of the first analysis segment.

[0028] 2. The interval lengths between analysis segments in the sparse key segment analysis strategy are different. As the length of the candidate video increases, the interval length between two analysis segments becomes longer.

[0029] Exemplarily, the sparse key segment analysis strategy divides the candidate video into 4 analysis segments, which are analysis segment 1, analysis segment 2, analysis segment 3 and analysis segment 4 in chronological order from front to back. Among them, the interval between analysis segment 1 and analysis segment 2 is 1s, the interval between analysis segment 2 and analysis segment 3 is 2s, and the interval between analysis segment 3 and analysis segment 4 is 4s.

[0030] It should be understood that compared with the intensive key segment analysis strategy, the sparse key segment analysis strategy focuses on analyzing the front and middle sections of the video, and the starting point of the first analysis segment may not be the first video frame, so the sparse key segment analysis strategy may take less time.

[0031] In combination with the first aspect, in an embodiment of the first aspect, the I frame sampling analysis strategy refers to selecting M frame images in the candidate video as the starting frames of the analysis segment, obtaining M analysis segments, and selecting one from the M analysis segments as the highlight segment, where M is a positive integer greater than 0.

[0032] In combination with the first aspect, in an embodiment of the first aspect, the method also includes: displaying a first interface, the first interface including a first control, the first control being used to generate a target video; in response to a first operation on the first control, displaying a second interface, the second interface being used to display multiple videos and / or pictures; in response to a second operation on the second interface, determining a candidate video, the second operation being used to select a candidate video from multiple videos and / or pictures.

[0033] It should be understood that the first interface can be Figure 3 In the interface shown in (b), the first control may be as follows Figure 3 After the user clicks the "One-click blockbuster" control, in response to the user's click operation, the following is displayed Figure 3 The second interface (interface A2) shown in (d) in FIG. 1 shows a plurality of videos and / or pictures in the second interface. The second operation may refer to a selection operation of the user on the second interface, for selecting a candidate video and / or a picture for a one-click blockbuster from the second interface. The second interface may also refer to Figure 3 The A3 interface shown in (f) in FIG.

[0034] In a second aspect, a method for determining a strategy for generating a video is provided, the method being applied to an electronic device, the electronic device comprising: a strategy monitoring module, an algorithm module and a basic capability layer, the strategy monitoring module being connected to the algorithm module and the basic capability layer respectively, the method comprising:

[0035] The policy monitoring module is used to obtain a first duration, a second duration, and a video analysis rate, wherein the first duration refers to the video duration of the candidate video, and the second duration is used to indicate the analysis duration allocated to the candidate video for analysis processing;

[0036] The strategy monitoring module is used to determine a target analysis strategy corresponding to the candidate video from the analysis strategy set based on the first duration, the second duration and the video analysis rate, wherein the analysis strategy set includes at least two analysis strategies, and different analysis strategies require different durations to analyze and process the candidate video;

[0037] The algorithm module is used to analyze and process the candidate videos based on the target analysis strategy to obtain the analysis results of the candidate videos;

[0038] The basic capability layer is used to obtain the target video based on the analysis results of the candidate videos.

[0039] It should be understood that different analysis strategies among at least two analysis strategies are to select all or part of the frame images from the candidate video for analysis and processing. Therefore, the time taken to analyze and process the candidate video using at least two analysis strategies is shorter than the traditional method of analyzing and processing all the frame images in the candidate video.

[0040] In an embodiment of the present application, the strategy monitoring module obtains the first duration, the second duration and the video analysis rate, and then determines the target analysis strategy corresponding to the candidate video from the analysis strategy set based on the first duration, the second duration and the video analysis rate, so that the algorithm module analyzes and processes the candidate video based on the target analysis strategy to obtain the analysis results of each candidate video, and finally, the basic capability layer generates the target video according to the analysis results of multiple candidate videos. Among them, the first duration refers to the video duration of the candidate video, the second duration is used to indicate the analysis duration allocated to the candidate video for analyzing and processing the candidate video, and the analysis strategy set includes at least two analysis strategies, and different analysis strategies are used to analyze and process the candidate video. The time required is different. That is to say, the target analysis strategy corresponding to the candidate video is determined from multiple analysis strategies through the video duration of the candidate video, the analysis duration allocated to the candidate video for analysis and processing, and the video analysis rate of the electronic device itself for analyzing the video. Compared with the traditional technology of analyzing and processing all frame images in the candidate video, the time occupied by analyzing and processing the candidate video is shortened, so the analysis results of the candidate video can be obtained faster, and then the target video can be obtained faster, thereby improving the user experience.

[0041] In combination with the second aspect, in one embodiment of the second aspect, the strategy monitoring module is used to determine a first ratio based on the first duration, the second duration and the video analysis rate, the first ratio being used to indicate the percentage of frame images in the candidate video that have completed analysis processing within the second duration to the total frame images in the candidate video; and the target analysis strategy is determined based on the first ratio.

[0042] In combination with the second aspect, in an embodiment of the second aspect, the strategy monitoring module is used to determine a third duration based on the first duration and the video analysis rate, and the third duration is used to indicate the time required to complete the analysis and processing of all frame images in the candidate video; based on the third duration and the second duration, determine the target analysis strategy corresponding to the candidate video.

[0043] In combination with the second aspect, in an embodiment of the second aspect, at least two analysis strategies include a full analysis strategy, a dense key segment analysis strategy, a sparse key frequency band analysis strategy, and an I frame sampling analysis strategy.

[0044] Among them, different analysis strategies are used to analyze and process the same candidate video. The full analysis strategy takes the longest time, the intensive key segment analysis strategy takes the second longest time, the sparse key segment analysis strategy takes the third longest time, and the I-frame sampling analysis strategy takes the shortest time.

[0045] In combination with the second aspect, in an embodiment of the second aspect, the full analysis strategy refers to a strategy for analyzing and processing the frame image of each frame in the candidate video.

[0046] It should be understood that since the full analysis strategy is a strategy for analyzing and processing the frame image of each frame in the candidate video, the time taken to analyze and process the candidate video using the full analysis strategy is the longest.

[0047] In combination with the second aspect, in an embodiment of the second aspect, the intensive key segment analysis strategy refers to a strategy of dividing the candidate video into N analysis segments with the same interval duration, and performing analysis and processing on the N analysis segments, where N is a positive integer greater than 0.

[0048] It should be understood that compared with the full analysis strategy that decodes, reduces the resolution and scores all video frames, the intensive key segment analysis strategy only needs to decode, reduce the resolution and score the video frames of a few key segments. Therefore, the time taken to analyze and process candidate videos using the intensive key segment analysis strategy is shorter than the time taken to analyze and process candidate videos using the full analysis strategy.

[0049] In combination with the second aspect, in an embodiment of the second aspect, the sparse key segment analysis strategy refers to a strategy of dividing the candidate video into N analysis segments and performing analysis and processing on the N analysis segments, wherein the intervals between the N analysis segments are of different durations.

[0050] It should be understood that compared with the intensive key segment analysis strategy, the sparse key segment analysis strategy focuses on analyzing the front and middle sections of the video, and the starting point of the first analysis segment may not be the first video frame, so the sparse key segment analysis strategy may take less time.

[0051] In combination with the second aspect, in an embodiment of the second aspect, the I frame sampling analysis strategy refers to selecting M frame images in the candidate video as the starting frames of the analysis segment, obtaining M analysis segments, and selecting one from the M analysis segments as the highlight segment, where M is a positive integer greater than 0.

[0052] In a third aspect, an electronic device is provided, the electronic device comprising one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute:

[0053] In a fourth aspect, an electronic device is provided, comprising one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the strategy determination method for generating a video in the first aspect or any one of the implementations of the first aspect.

[0054] In a fifth aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, and the processors are used to call computer instructions to enable the electronic device to execute the first aspect or any one of the strategy determination methods for generating a video in the first aspect.

[0055] In a sixth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by an electronic device, the electronic device executes the strategy determination method for generating a video in the first aspect or any one of the implementations of the first aspect.

[0056] In a seventh aspect, a computer program product is provided, comprising: a computer program code, which, when executed by an electronic device, enables the electronic device to execute the strategy determination method for generating a video in the first aspect or any one of the implementations of the first aspect.

[0057] The strategy determination method and electronic device for generating a video provided in the embodiment of the present application obtain the first duration, the second duration and the video analysis rate, and then determine the target analysis strategy corresponding to the candidate video from the analysis strategy set based on the first duration, the second duration and the video analysis rate, and then analyze and process the selected video based on the target analysis strategy to obtain the analysis results of each selected video, and finally, generate the target video according to the analysis results of multiple selected videos. Among them, the first duration refers to the video duration of the candidate video, the second duration is used to indicate the analysis duration allocated to the candidate video for analyzing and processing the candidate video, and the analysis strategy set includes at least two analysis strategies, and different analysis strategies are used to analyze and process the candidate video. The required duration is different. That is to say, the target analysis strategy corresponding to the candidate video is determined from multiple analysis strategies through the video duration of the candidate video, the analysis duration allocated to the candidate video for analysis and processing, and the video analysis rate of the electronic device itself for analyzing the video. Compared with the traditional technology of analyzing and processing all frame images in the candidate video, the time occupied by analyzing and processing the candidate video is shortened, so the analysis results of the candidate video can be obtained faster, and then the target video can be obtained faster, thereby improving the user experience.

[0058] Furthermore, since the second duration is the analysis time allocated to the candidate video for analysis and processing, it is equivalent to the electronic device determining the analysis time corresponding to each candidate video in advance according to the user's acceptable waiting time (that is, the waiting time that will not lead to a deterioration in user experience), so that the total analysis time required for the analysis and processing of each candidate video according to the allocated analysis time will not exceed the user's acceptable waiting time, and will not lead to a poor user experience due to excessive waiting time, thereby improving the user experience in the process of one-click blockbuster. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a schematic diagram of a hardware system of an electronic device applicable to the present application;

[0060] Figure 2 is a schematic diagram of a software system for an electronic device applicable to the present application;

[0061] Figure 3 is a schematic diagram of an interface for selecting a candidate video in an embodiment of the present application;

[0062] Figure 4 This is a schematic diagram of an interface for generating a one-click blockbuster video in an embodiment of the present application;

[0063] Figure 5 This is a schematic diagram of a display interface for a one-click blockbuster video in an embodiment of the present application;

[0064] Figure 6 This is a schematic diagram of an interface for generating a one-click blockbuster video in an embodiment of the present application;

[0065] Figure 7 It is a flowchart of a method for determining a strategy for generating a video provided in an embodiment of the present application;

[0066] Figure 8 It is a flowchart of a method for determining a strategy for generating a video provided in an embodiment of the present application;

[0067] Fig. 9 It is a flowchart of a method for determining a strategy for generating a video provided in an embodiment of the present application;

[0068] Fig.10 is a schematic diagram of the time required for different analysis strategies in the embodiments of the present application;

[0069] Fig.11 is a schematic diagram of the middle analysis segment of the intensive key segment analysis strategy in the embodiment of the present application;

[0070] Fig.12 is a schematic diagram of different parameters of the intensive key segment analysis strategy in an embodiment of the present application;

[0071] Fig.13 It is a flow chart of the intensive key segment analysis strategy in the embodiment of the present application;

[0072] Fig.14 It is a flow chart of the sparse key segment analysis strategy in the embodiment of the present application;

[0073] Fig.15 Schematic diagram of the middle analysis segment of the sparse key segment analysis strategy in the embodiment of the present application;

[0074] Fig.16 It is a flow chart of the sparse key segment analysis strategy in the embodiment of the present application;

[0075] Fig.17 is a schematic diagram of the position of a frame of image in a candidate video in the I frame sampling analysis strategy in an embodiment of the present application;

[0076] Fig.18 It is a flowchart of a method for determining a strategy for generating a video provided in an embodiment of the present application;

[0077] Fig.19 It is a flowchart of the duration allocation algorithm provided in this application;

[0078] Fig. 20 It is a flowchart of a method for determining a strategy for generating a video provided in an embodiment of the present application;

[0079] Fig.21 It is a flowchart of a method for determining a strategy for generating a video provided in an embodiment of the present application;

[0080] Fig. 22 It is a flowchart of a method for determining a strategy for generating a video provided in an embodiment of the present application;

[0081] Fig.23 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0082] In the embodiments of the present application, the following terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present embodiment, unless otherwise specified, "plurality" means two or more.

[0083] In order to facilitate the understanding of the embodiments of the present application, the relevant concepts involved in the embodiments of the present application are first briefly described.

[0084] 1. Highlight Clips

[0085] Highlight clips refer to the wonderful clips in a video. They can be clips of the user's smiling face, clips of the moment of winning, etc. For example, taking a skateboarder's competition video as an example, the wonderful moment can be the moment when the athlete performs a flip skateboard action.

[0086] 2. One-click blockbuster

[0087] In some electronic devices, a "one-click blockbuster" function is provided to users. When the user clicks the "one-click blockbuster" control, the electronic device automatically selects the highlight segments of each video and / or picture from multiple videos and / or pictures selected by the user, and automatically generates a new video based on the selected highlight segments. It should be understood that the highlight segment of a picture can refer to scoring the picture. When the score of the picture is higher than a preset threshold, the picture can be used as a highlight segment, and the highlight segment only includes one frame of the picture.

[0088] At present, some electronic devices provide users with a "one-click blockbuster" function. When the user clicks the "one-click blockbuster" control, the electronic device automatically selects the highlight segments of each video and / or picture from multiple videos and / or pictures selected by the user, and automatically generates a new video based on the selected highlight segments. In the process of selecting the highlight segments of the video, the frame images in the video are usually analyzed to obtain the analysis results of each frame image, and then a new video is automatically generated based on the analysis results of each frame image. However, the number of frame images in the video is usually large, and the time required to analyze the frame images in the video is usually long, which will result in a poor user experience.

[0089] In view of this, the embodiment of the present application provides a strategy determination method and electronic device for generating a video. In the embodiment of the present application, by obtaining a first duration, a second duration and a video analysis rate, and then determining a target analysis strategy corresponding to a candidate video from an analysis strategy set based on the first duration, the second duration and the video analysis rate, and then analyzing and processing the candidate video based on the target analysis strategy to obtain analysis results of each candidate video, and finally, generating a target video based on the analysis results of multiple candidate videos. Among them, the first duration refers to the video duration of the candidate video, the second duration is used to indicate the analysis duration allocated to the candidate video for analyzing and processing the candidate video, the analysis strategy set includes at least two analysis strategies, and different analysis strategies are used to analyze and process the candidate video. The duration required is different. That is to say, through the video length of the candidate video, the analysis time allocated to the candidate video for analysis and processing, and the video analysis rate of the electronic device itself for analyzing the video, a target analysis strategy corresponding to the candidate video is determined from multiple analysis strategies. Compared with the traditional technology of analyzing and processing all frame images in the candidate video, the time taken for analyzing and processing the candidate video is shortened, so the analysis results of the candidate video can be obtained faster, and then the target video can be obtained faster, thereby improving the user experience.

[0090] Furthermore, since the second duration is the analysis time allocated to the candidate video for analysis and processing, it is equivalent to the electronic device determining the analysis time corresponding to each candidate video in advance according to the user's acceptable waiting time (that is, the waiting time that will not lead to a deterioration in user experience), so that the total analysis time required for the analysis and processing of each candidate video according to the allocated analysis time will not exceed the user's acceptable waiting time, and will not lead to a poor user experience due to excessive waiting time, thereby improving the user experience in the process of one-click blockbuster.

[0091] The following will describe the strategy determination method and electronic device for generating a video provided in the embodiments of the present application in conjunction with the accompanying drawings.

[0092] Figure 1 A hardware system of an electronic device suitable for the present application is shown.

[0093] The electronic device 100 can be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, an in-vehicle electronic device, an augmented reality (AR) device, a virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, etc. The embodiment of the present application does not impose any restrictions on the specific type of the electronic device 100.

[0094] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0095] It should be noted that Figure 1 The structure shown does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include Figure 1 More or fewer components than those shown, or the electronic device 100 may include Figure 1 Combinations of some of the components shown, or the electronic device 100 may include Figure 1 Subassemblies of some of the components shown. Figure 1 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0096] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). Among them, different processing units may be independent devices or integrated devices. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0097] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0098] In some embodiments, the processor 110 may include one or more interfaces. For example, the processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and a USB interface.

[0099] Exemplarily, in an embodiment of the present application, the processor 110 can be used to execute the strategy determination method for generating a video provided in an embodiment of the present application; for example, obtaining a first duration, a second duration, and a video analysis rate, and then determining a target analysis strategy corresponding to the candidate video based on the first duration, the second duration, and the video analysis rate, and then analyzing and processing the candidate video based on the target analysis strategy to obtain analysis results for each candidate video, and finally, generating a target video based on the analysis results of multiple candidate videos.

[0100] Figure 1 The connection relationship between the modules shown is only a schematic illustration and does not constitute a limitation on the connection relationship between the modules of the electronic device 100. Optionally, the modules of the electronic device 100 may also adopt a combination of multiple connection modes in the above embodiments.

[0101] The wireless communication function of the electronic device 100 can be implemented through components such as the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

[0102] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0103] The electronic device 100 can realize the display function through the GPU, the display screen 194 and the application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0104] Display screen 194 may be used to display images or videos.

[0105] Optionally, the display screen 194 can be used to display images or videos. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a micro OLED (Micro OLED) or a quantum dot light emitting diode (QLED). In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0106] Exemplarily, in an embodiment of the present application, the display screen 194 may display a video or photo selected by a user; and display a processed video.

[0107] Exemplarily, the electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0108] Exemplarily, the ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted to the camera photosensitive element through the camera, and the light signal is converted into an electrical signal. The camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can perform algorithmic optimization on the noise, brightness and color of the image. The ISP can also optimize the exposure and color temperature of the shooting scene and other parameters. In some embodiments, the ISP can be set in the camera 193.

[0109] Exemplarily, the camera 193 (also referred to as a lens) is used to capture static images or videos. It can be triggered to start by an application instruction to realize the photo function, such as taking an image of any scene. The camera may include components such as an imaging lens, a filter, and an image sensor. The light emitted or reflected by the object enters the imaging lens, passes through the filter, and finally converges on the image sensor. The imaging lens is mainly used to converge the light emitted or reflected by all objects in the photographing angle (also referred to as the scene to be photographed, the target scene, or the scene image that the user expects to shoot) to form an image; the filter is mainly used to filter out excess light waves in the light (for example, light waves other than visible light, such as infrared); the image sensor can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The image sensor is mainly used to perform photoelectric conversion on the received light signal, convert it into an electrical signal, and then transmit the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, etc. format.

[0110] Exemplarily, the digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0111] Exemplarily, a video codec is used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0112] Exemplarily, the gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., the x-axis, the y-axis, and the z-axis) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used in scenes such as navigation and somatosensory games.

[0113] For example, the acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally the x-axis, y-axis, and z-axis). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. The acceleration sensor 180E can also be used to identify the posture of the electronic device 100 as an input parameter for applications such as horizontal and vertical screen switching and pedometers.

[0114] Exemplarily, the distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, for example, in a shooting scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0115] Exemplarily, the ambient light sensor 180L is used to sense the brightness of the ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0116] For example, the fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement functions such as unlocking, accessing application locks, taking photos, and answering calls.

[0117] Exemplarily, the touch sensor 180K is also referred to as a touch control device. The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also referred to as a touch control screen. The touch sensor 180K is used to detect a touch operation acting on or near it. The touch sensor 180K may pass the detected touch operation to the application processor to determine the type of touch event. A visual output related to the touch operation may be provided through the display screen 194. In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100, and may be disposed at a different position from the display screen 194.

[0118] The hardware system of the electronic device 100 is described in detail above, and the software system of the electronic device 100 is introduced below.

[0119] Figure 2 It is a schematic diagram of a software system of an electronic device provided in an embodiment of the present application.

[0120] like Figure 2As shown, the electronic device can adopt a layered architecture to divide the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the software layers of the software structure are divided from top to bottom into: application (APP) layer, media middle platform framework layer, application framework (FWK) layer and hardware abstraction layer (HAL). In one possible case, the electronic device may refer to a terminal device.

[0121] The application layer, referred to as the application layer, can include a series of application packages, such as camera, gallery, calendar, map, navigation, etc. When these application packages are running, they can access the various service modules provided by the media platform framework layer and the application framework layer through the application programming interface (API) and execute corresponding intelligent services.

[0122] In some embodiments, the camera is used to take photos, videos, slow-motion images, panoramic images, etc. in response to user operations. After these images are taken by the camera, or after the user triggers the mobile phone to take a screenshot, or after the user triggers the mobile phone to record the screen, or after the terminal device downloads images from other devices, the terminal device can save these images in the gallery, so that the user can generate videos from the images and / or videos in the gallery, such as one-click filming.

[0123] The embodiment of the present application divides the gallery into: a business layer, an application function layer and a basic function layer from top to bottom.

[0124] The business layer provides multiple services such as automatic multi-lens video recording, AI music videos, one-click video recording, and wonderful moments. These services are presented in the form of controls in the user interface (UI) of the gallery. Users can trigger the camera to perform corresponding video processing actions by operating a certain control. For example, after the user selects one or more material videos and clicks the one-click video recording control of the gallery, the gallery will automatically analyze and extract the highlight clips in the material videos through algorithms, and combine the highlight clips into a well-edited video.

[0125] The application function layer includes an automatic editing framework. Each business in the business layer can call the automatic editing framework to provide automatic editing services for pictures and videos. Exemplarily, the automatic editing framework may include functional modules such as segment optimization, storyline organization, layout splicing, and special effects beautification. Segment optimization is used to call the highlight segment analysis interface and the policy monitoring interface to extract highlight segments from the material video. Storyline organization is used to sequentially splice multiple material videos in the form of a storyline based on the content of the material video. Layout splicing is used to modulate the interface layout of the material video. Special effects beautification is used to adjust the beautification effect of the video, such as adjusting the brightness of the picture and beautifying the face of the character.

[0126] The basic function layer is used to perform basic function processing on the edited video clips after the automatic editing framework edits multiple material videos. Exemplarily, the basic function layer may include basic function modules such as video splicing, synthesis preservation, video effect rendering, and audio effect processing. Among them, the video splicing user splices multiple extracted highlight clips. Synthesis preservation is used to store the video set obtained after splicing. Video effect rendering is used to add video effects to the video set obtained after splicing, such as adding style filters and themes to the video. Audio effect processing is used to add sound effects to the video set obtained after splicing, such as adding background music.

[0127] The media middle platform framework layer is a software layer set between the application layer and the application framework. The media middle platform framework layer may include an analysis performance query interface, a highlight segment analysis interface, a policy monitoring interface, a pipeline interface, and a topic summary interface. Among them, the analysis performance query interface is used to calculate the total duration of all material videos according to the video analysis speed. The highlight segment analysis interface is used to call the policy monitoring interface to extract highlight segments. The policy monitoring interface is used to configure the available analysis time for each material video according to the total duration of all material videos, and to set the dynamic analysis strategy for each material video based on the available analysis time and the duration of each video. The pipeline interface is used to reduce the resolution of the video file according to the file description issued by the policy monitoring interface in accordance with the analysis strategy, and forward the data address of the video file after the resolution reduction to the hardware abstraction layer through the application framework, and then report the analysis results of the highlight segment returned by the hardware abstraction layer to the policy monitoring interface. The topic summary interface is used to obtain the topic corresponding to the content of the highlight segment.

[0128] The application framework layer, referred to as the framework layer, can be used to support the operation of various modules in the media platform framework layer. For example, the framework layer can include a one-click film interface, a parameter management interface, an image data transmission interface, a theme analysis interface, and a performance analysis interface.

[0129] The hardware abstraction layer is an encapsulation of the Linux kernel driver, providing an interface to the upper layer. It hides the hardware interface details of a specific platform and provides a virtual hardware platform for the operating system, making it hardware-independent and portable on multiple platforms. For example, the hardware abstraction layer may include a highlight segment algorithm, a face detection algorithm, a video acceleration algorithm, and an image super-resolution algorithm. Among them, the highlight segment algorithm is an image processing algorithm provided by the image chip. The algorithm can score each frame of the image based on the image color, image texture features, image quality, frame interpolation with previous and next frames, and edge change rate values. The scoring result can be used as a basis for evaluating whether a frame of the image is a highlight segment. For example, when the scoring result of a frame of the image is greater than or equal to 60 points, the frame of the image is a highlight segment.

[0130] It should be noted that Figure 2 The layers in the software structure shown and the components included in each layer do not constitute a specific limitation on the terminal device. In other embodiments, the terminal device may include more layers than shown in the figure, such as a system library (FWK LIB) layer and a kernel layer. In addition, each layer may include more or fewer components than shown in the figure, which is not limited in this application.

[0131] It is understandable that in order to implement the strategy determination method for generating videos in the embodiments of the present application, the terminal device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments.

[0132] Combine the following Figures 3 to 6 The relevant interface schematic diagram of the strategy determination method for generating a video provided in an embodiment of the present application is described in detail.

[0133] In an exemplary application scenario, a user uses a mobile phone to shoot multiple video materials in advance, and the mobile phone stores the multiple video materials in a gallery. Figure 3 As shown in Figure (a) of FIG. 1 , a gallery control is displayed on the desktop of the mobile phone. When the user wants the mobile phone to generate a clipped video clip based on multiple video materials, the user can click the gallery control on the mobile phone desktop. In response to the user's click operation on the gallery control, the mobile phone displays a gallery interface A1, as shown in FIG. Figure 3 As shown in Figure (b) in FIG. 1 , the gallery interface A1 provides a “one-click blockbuster” control, such as Figure 3As shown in Figure (c) in FIG. 1 , the user can click the “One-click blockbuster” control. In response to the user clicking the “One-click blockbuster” control, the mobile phone displays the gallery interface A2, as shown in FIG. Figure 3 As shown in FIG. 5(d), the gallery interface A2 may include multiple recently shot video materials, such as multiple video materials shot on November 10, 2022. The user may click on any one of the multiple video materials, such as Figure 3 As shown in Figure (e) in FIG. 1 , the user clicks on the first video material among the multiple video materials. In response to the user's click operation on any of the video materials, the mobile phone displays a gallery interface A3, which includes all the materials in the gallery. In this way, the user can select the video material required to generate a video clip in the gallery interface A3, for example, Figure 3 As shown in Figure (f) in FIG. 1 , it is assumed that the user has selected 5 video materials. The gallery interface A3 also includes video generation controls, such as Figure 3 As shown in Figure (f) in the figure, the video generation control is a check mark “√”.

[0134] Continue to refer Figure 4 In Figure (a), after selecting a video material, the user can click on the video generation control, such as clicking on the check mark “√”. Figure 4 As shown in Figure (b), in response to the user clicking the check mark "√" control, the mobile phone starts to analyze the five video materials selected by the user in the gallery interface A3, selects highlight clips (i.e., wonderful clips, such as user smiling face clips, winning moment clips, etc.) from each video material, and generates video clips based on the selected highlight clips. During this process, the mobile phone can display the progress of analyzing the materials in the gallery interface A3 so that the user can intuitively view the progress of the analysis. In an example, Figure 5 As shown in Figure (a) in FIG. 1 , after the video clip is generated, the mobile phone displays the gallery interface A4, which includes the generated video clip, and the mobile phone can automatically play the video clip. In addition, a video export control can be provided in the gallery interface A4, such as Figure 5 As shown in FIG. 1 (b), the user can click on the video export control. In response to the user's click operation on the video export option, the mobile phone exports the video clip, such as Figure 5 As shown in Figure (c) in the figure, the video clip is stored in the gallery, so that the user can view the video clip from the gallery.

[0135] In another exemplary application scenario, a user uses a mobile phone to shoot multiple video materials and multiple image materials in advance, and the mobile phone stores the multiple video materials and multiple image materials in a gallery. In this way, the user can select video materials and image materials in the gallery, so that the mobile phone generates a video clip based on the video materials and image materials. In one example, Figure 6As shown in Figure (a), after entering the gallery interface A2, the gallery interface A2 includes multiple video materials and multiple image materials that have been recently shot. The user can click on any one of the multiple video materials and multiple image materials, such as the first video material. In response to the user's click operation on any one of the materials, the mobile phone displays the gallery interface A3, which includes all the materials in the gallery. In this way, the user can select the material required to generate a video clip in the gallery interface A3, such as Figure 6 As shown in Figure (b) of FIG. 1 , it is assumed that the user has selected 4 video materials and 2 picture materials. The gallery interface A3 also includes a video generation control, such as Figure 6 As shown in Figure (b) of the video generation control, the icon is a check mark "√". Figure 6 In Figure (c), after selecting the material, the user can click on the video generation control, such as clicking on the check mark “√”. Figure 6 As shown in Figure (d), in response to the user clicking the check mark "√", the mobile phone starts to analyze the 4 video materials and 2 picture materials selected by the user in the gallery interface A3, selects the highlight clips from each video material, and selects the highlight pictures (i.e., wonderful pictures) from the 2 picture materials. During this process, the mobile phone can display the material analysis progress in the gallery interface A3 so that the user can intuitively view the analysis progress. In an example, Figure 5 As shown in Figure (a) in FIG. 1 , after the video clip is generated, the mobile phone displays the gallery interface A4, which includes the generated video clip, and the mobile phone can automatically play the video clip. In addition, a video export control can be provided in the gallery interface A4, such as Figure 5 As shown in Figure (b), the user can click on the video export control. In response to the user's click operation on the video export control, the mobile phone exports the video clip, such as Figure 5 As shown in Figure (c) in the figure, the video clip is stored in the gallery, so that the user can view the video clip from the gallery.

[0136] In one example, if the number of materials selected by the user is small, the mobile phone can prompt the user during the process of selecting materials so that the user can know how many materials are appropriate to select. Figure 3 As shown in Figure (f), a prompt message "More than 6 materials will produce better results" is displayed in the gallery interface A3, so that the user can know how many materials are needed to generate a better video clip.

[0137] In one example, if the number of materials selected by the user is large, the mobile phone can prompt the user during the process of selecting materials so that the user can know how many materials can be selected at most. Figure 6As shown in FIG. 5( b ) , a prompt message “up to 30 materials can be selected” is displayed in the gallery interface A3 , so that the user can know how many materials can be selected.

[0138] In one example, after the video clip is generated, other functional controls may be provided in the application interface A4 displaying the video clip, so that the user can edit, add special effects, analyze, etc. the generated video clip based on these functional controls, for example Figure 5 As shown in Figure (a), the other functional controls may include but are not limited to templates, music, clips, sharing and other controls.

[0139] In one example, after the electronic device detects that the user clicks the "One-click Blockbuster" control operation, the electronic device can execute the strategy determination method for generating videos provided in the embodiment of the present application, analyze and process multiple videos selected by the user, and display the video as follows: Figure 5 Display interface A4 shown in (a).

[0140] It should be understood that the above application scenarios are examples of application scenarios of the embodiments of the present application and do not constitute a limitation on the application scenarios of the embodiments of the present application.

[0141] Combine the following Figures 7 to 22 The strategy determination method for generating a video provided in an embodiment of the present application is described in detail.

[0142] Figure 7 is a schematic flow chart of a method for determining a strategy for generating a video provided by an embodiment of the present application. The method can be Figure 1 The electronic device shown executes; the strategy determination method for generating a video includes steps S100 to S300, and steps S100 to 300 are described in detail below.

[0143] Step 100: In response to a first operation of the user, an initialization operation is performed to obtain a video analysis rate.

[0144] Step 200: Analyze and process each candidate video based on the video analysis rate to obtain analysis results of each candidate video.

[0145] Step 300: Generate a target video based on the analysis results of multiple candidate videos.

[0146] The following is a detailed description of step 100 "in response to a first operation of the user, performing an initialization operation to obtain a video analysis rate".

[0147] The first operation may refer to the operation of the user clicking the "One-click Movie" control. The specific steps of the initialization operation can be found in Figure 8 shown.

[0148] It should be understood that the first operation may be a click operation or a voice operation, and the embodiment of the present application does not limit this.

[0149] Figure 8 Schematic diagram of the process of initializing the operation in response to the user clicking the "One-click Movie" control in the embodiment of the present application. Figure 8 As shown, including:

[0150] S101. The video editing service layer of the application layer receives an operation by a user to enable a one-key video-making function.

[0151] S102, the video editing business layer loads and displays pictures and videos.

[0152] S103: The video editing service layer receives the video and picture selection from the user and confirms the one-click creation operation.

[0153] S104. The video editing business layer calls the media middle platform framework to initialize each interface.

[0154] Among them, the video convenience service layer can initialize the image highlight segment analysis interface, main body summary interface and analysis performance query interface in the media middle platform framework layer through the application function layer in the application layer.

[0155] S105. The media middle platform framework sends initialization parameters to the one-click film-making module of the FWR layer.

[0156] Among them, the analysis performance interface in the media platform framework can send initialization parameters to the one-click film module of the FWR layer through the channel interface in the media platform framework. The sent initialization parameters include the highlight segment analysis algorithm corresponding to the one-click film function and the corresponding initialization interface.

[0157] S106. The one-key film forming module instructs the algorithm module of the HAL layer to perform an initialization operation.

[0158] S107: The algorithm module initializes the highlight segment analysis algorithm.

[0159] S108. The algorithm module returns a message indicating that initialization is successful to the one-key film formation module.

[0160] S109, the one-key chip module calls the chip analysis speed interface from the algorithm module.

[0161] S110, the algorithm module sends the chip analysis speed to the one-key chip formation module.

[0162] The chip analysis speed refers to the rate at which the chip in the electronic device analyzes the video, and can be used to indicate the number of frame images that the electronic device can analyze per second. It is a performance parameter related to the chip in the electronic device, and is usually fixed. Optionally, the chip analysis speed can also be called the video analysis rate.

[0163] Exemplarily, the video analysis rate is 4.

[0164] Exemplarily, the video analysis rate is 5.

[0165] S111. The one-click film module sends the chip analysis speed to the analysis performance query interface in the media middle platform framework layer.

[0166] S112. The analysis performance query interface returns an instruction indicating successful initialization to the video editing service layer.

[0167] After completing the initialization operation, the video editing business layer of the application layer obtains the video analysis rate returned by the algorithm module, and then can analyze the video and / or picture selected by the user according to the video analysis rate to obtain the analysis result, that is, perform the relevant operations of step 200.

[0168] Next, step 200 of "analyzing and processing each candidate video based on the video analysis rate to obtain analysis results of each candidate video" is described.

[0169] The specific steps of analyzing and processing each candidate video to obtain the corresponding analysis results can be as follows: Fig. 9 As shown, including:

[0170] S201. The video editing business layer of the application layer queries the analysis performance interface of the media middle platform framework layer for the analysis speed of video 1.

[0171] Among them, video 1 can refer to Figure 3 The video selected in the interface A3 shown in (f) is the video material for the one-key blockbuster function. The analysis speed of the video can be queried through the file descriptor of the video. Exemplarily, the file descriptor of video 1 is fd1.

[0172] S202. The analysis performance interface of the media middle platform framework layer obtains the resolution and rate of video 1 according to the file descriptor fd1 of video 1.

[0173] S203. The analysis performance interface calculates the analysis rate of video 1 according to the resolution, frame rate and chip analysis speed of video 1.

[0174] S204. The analysis performance interface returns the analysis rate of video 1 to the video editing service layer.

[0175] After receiving the analysis rate of video 1, the video editing service layer continues to execute S201 to S204 until the analysis speeds of all selected videos are queried.

[0176] S205: The video editing service layer calculates the target total analysis duration according to the analysis speed of all returned videos.

[0177] The target total analysis duration may refer to the duration of analyzing all selected videos.

[0178] S206. The video editing business layer calls the image highlight segment analysis interface to send the target analysis total duration and the file descriptors of all the material files to be analyzed to the policy monitoring module.

[0179] Among them, all the material files to be analyzed are included in Figure 3 The video (ie, candidate video) and picture selected in interface A3 in (f) .

[0180] For example, the user may only select a picture to perform a one-click blockbuster operation and obtain a target video corresponding to the one-click blockbuster function.

[0181] Exemplarily, the user may only select a video to perform a one-click blockbuster operation and obtain a target video corresponding to the one-click blockbuster function.

[0182] Exemplarily, the user can select pictures and videos to perform a one-click blockbuster operation and obtain a target video corresponding to the one-click blockbuster function.

[0183] It should be understood that since the picture material usually only has one frame of image, the time required to analyze the picture material is usually very short, so the picture material can be directly analyzed and processed without selecting different analysis strategies.

[0184] For the analysis and processing of images, please refer to Fig. 9 The steps from S207 to S211 in the embodiment include:

[0185] S207 . The policy monitoring module instructs the channel interface to analyze picture 1 .

[0186] Here, picture 1 refers to one of all pictures to be analyzed.

[0187] S208 . The channel interface decodes, reduces the resolution, and performs format conversion processing on the picture 1 to obtain a processed picture 1 , and stores the processed picture 1 .

[0188] S209. The channel interface sends the frame data address of the processed picture 1 to the algorithm module in the HAL layer.

[0189] S210 , the algorithm module in the HAL layer obtains the processed picture 1 according to the frame data address, and analyzes and processes the processed picture 1 through the analysis algorithm pre-stored in the algorithm module to obtain the analysis result of the picture 1 .

[0190] S211. The algorithm module returns the analysis result of picture 1 to the strategy monitoring module.

[0191] After receiving the analysis result of picture 1, the policy monitoring module continues to execute S207 to S211 until the analysis of all selected pictures is completed.

[0192] The analysis result of the image may refer to the score of the processed image obtained by the analysis algorithm. The higher the score, the higher the probability that the image is the image corresponding to the highlight segment. The lower the score, the lower the probability that the image is the image corresponding to the highlight segment.

[0193] It should be understood that analyzing and processing the video material generally refers to analyzing and processing the frame images in the video material. It can be analyzing and processing the frame images of each frame in the video material, or analyzing and processing the frame images of some frames, which is not limited in the embodiment of the present application. Figure 3 As for the video material selected in the interface A3 shown in (f) in the figure, the more frame images are processed, the longer the time consumed. Since the analysis time allocated to each candidate video is fixed, the analysis strategy corresponding to each candidate video can be selected according to the analysis time allocated to each candidate video and the video time of each candidate video itself. Different analysis strategies correspond to different numbers of processed frame images.

[0194] For video analysis and processing, please refer to Fig. 9 S212 to S218, including:

[0195] S212: The policy monitoring module allocates the analysis duration of each video according to the expected time.

[0196] The expected time is the difference between the total time of target analysis and the time consumed for image analysis. Each video may refer to a candidate video.

[0197] S213. The strategy monitoring module dynamically sets the analysis strategy.

[0198] S214. The policy monitoring module sends the file description of video 1 to the interface channel, which complies with the set analysis policy.

[0199] S215 , the interface channel performs processing such as decoding, format conversion, and resolution reduction on the video 1 , and stores the processed video 1 .

[0200] S216. The interface channel sends the frame data address of the processed video 1 to the algorithm module of the HAL layer.

[0201] S217 , the algorithm module analyzes the position of the highlight segment of video 1.

[0202] It should be understood that the analysis result of video 1 may refer to the position of the highlight segment in video 1. The position of the highlight segment includes the start time point and the end time point of the highlight segment in video 1.

[0203] S218. The algorithm module returns the analysis result of video 1 to the strategy monitoring module.

[0204] After receiving the analysis result of video 1, the policy monitoring module continues to analyze the next video (that is, continues to execute S212 to S218) until the analysis of all candidate videos is completed.

[0205] It should be understood that after the analysis of all selected videos and pictures is completed, the analysis results of all selected videos and pictures can be returned to the application function layer in the application layer. Specifically, it includes:

[0206] S219, the policy monitoring module calls the image highlight segment analysis interface, notifies the application function layer in the application layer that the analysis of all material files has been completed, and reports the analysis demerits to the application function layer.

[0207] The process of S213 “strategy monitoring module dynamically sets analysis strategy” is described in detail below.

[0208] Among them, the analysis strategies for analyzing and processing candidate videos include full analysis strategy, intensive key segment analysis strategy, sparse key segment analysis strategy and I frame sampling analysis strategy. Among them, for the same candidate video, different analysis strategies correspond to different numbers of processed frame images. The different analysis strategies are arranged from high to low in terms of the number of processed frame images: full analysis strategy, intensive key segment analysis strategy, sparse key segment analysis strategy and I frame sampling analysis strategy. In other words, different analysis strategies are used to analyze and process the same candidate video, such as Fig.10 As shown, the full analysis strategy takes the longest time, the intensive key segment analysis strategy takes the second longest time, the sparse key segment analysis strategy takes the third longest time, and the I-frame sampling analysis strategy takes the shortest time.

[0209] Exemplarily, the full analysis strategy means that when analyzing a candidate video, each frame image in the candidate video is analyzed to obtain highlight segments.

[0210] Exemplarily, the intensive key segment analysis strategy refers to an analysis strategy for obtaining highlight segments by analyzing and processing N analysis segments in the candidate video, wherein the candidate video includes N analysis segments, and adjacent analysis segments include interval segments.

[0211] Exemplarily, N is a positive integer less than or equal to 5.

[0212] For example, the intensive key segment analysis strategy can first calculate three target parameters: the number of analysis segments N of the candidate video 分析 , the duration of the analysis segment T 分析 , the interval between adjacent segments T 间隔 Then, according to the three target parameters, the analysis segments are evenly distributed so that the analysis segments cover the highlight segments at different positions of the candidate videos as much as possible.

[0213] For example, Fig.11 As shown, starting from the first frame of the i-th video (one of the candidate videos), analysis segments are evenly distributed to obtain analysis segments N1, analysis segments N2, analysis segments N3, etc. Among them, the duration of the analysis segments N1, analysis segments N2, analysis segments N3, etc. is T 分析 , the duration of the last analysis segment Nn is T 剩余 .

[0214] In the embodiment of the present application, the above three target parameters can be obtained by Fig.12 The five initial parameters shown are calculated.

[0215] Minimum length of highlight clip T 短高光 ;

[0216] Maximum duration of highlight clips T 长高光 ;

[0217] Minimum interval T of highlight fragments 最小间隔 ;

[0218] The available analysis time T allocated for the i-th video 预期分析 , also known as the expected analysis duration;

[0219] and the total duration T of the i-th video 总 .

[0220] Among them, the minimum length of the highlight segment is T 短高光 is the expected minimum duration configured for each highlight segment. The maximum duration of the highlight segment T 长高光 is the expected maximum duration configured for each highlight segment. The minimum interval T of the highlight segment 最小间隔It is the minimum time interval between highlight clips of the same shot. These three parameters can be configured by the application layer and sent to the policy monitoring module.

[0221] The available analysis time T allocated for the i-th video 预期分析 , is the total expected duration allocated by the policy monitoring module to the i-th video for analyzing the video according to the expected duration.

[0222] The total duration of the i-th video is T 总 is the video duration of the ith video. In the embodiment of the present application, the total duration of a video may be set to not exceed 30 seconds, which may reduce the amount of data calculation of the highlight segment interface.

[0223] It should be noted that in Fig.11 The line segment lengths corresponding to the various durations are only exemplary. In actual implementation, the specific values ​​of the various durations can be adjusted according to actual needs.

[0224] Combine the following Fig.13 , the principle of calculating 3 target parameters based on 5 initial parameters is illustrated by example.

[0225] (1) According to the total duration T of the i-th video 总 , the available analysis time T allocated for the i-th video 预期分析 , and the minimum interval T of highlight fragments 最小间隔 , determine the maximum number of analyzed fragments N 上限 , which is the upper limit of the analyzed fragment.

[0226] In one example, N 上限 It can be calculated by the following relationship:

[0227]

[0228] in, The floor symbol.

[0229] For example, N 上限 The maximum value of is in the range of [1 to 5], and N 上限 is an integer. For example, when the terminal device pre-sets N 上限 When the maximum value of is set to 4, if the above relationship is used to calculate N 上限 =5, then N 上限 The final value is set to 4.

[0230] It should be understood that the total duration T of the i-th video 总 and the available analysis time T allocated for the i-th video 预期分析The difference is the total duration of all interval segments. The total duration of all interval segments is divided by the minimum interval T of the highlight segment. 最小间隔 , we can get the upper limit of the spacer segment. Since a spacer segment is set between two analysis segments, the number of analysis segments is one more than the number of spacer segments. 总 , T 预期分析 and T 最小间隔 The upper limit value of the analyzed fragment can be obtained.

[0231] (2) According to the maximum number of analyzed fragments N 上限 , and the available analysis time T allocated for the i-th video 预期分析 , determine the minimum length T of the analyzed fragment 下限 .

[0232] In one example, T 下限 It can be calculated by the following relationship:

[0233]

[0234] It should be understood that the available analysis time T allocated for the i-th video 预期分析 When the number of analysis segments is set to the maximum number N, 上限 When , the calculated duration of the analysis segment is the minimum duration T 下限 .

[0235] (3) Based on the minimum length T of the analyzed segment 下限 , and the maximum duration T of the highlight segment 长高光 , determine the duration T of the analysis segment 分析 .

[0236] In one implementation, there is no need to consider the minimum length T of the analyzed segment. 下限 , by default, the maximum duration of the highlight clip is T 长高光 As the duration of the analysis segment T 分析 , then the following relationship exists:

[0237] T 分析 =T 长高光 .

[0238] In another implementation, it is necessary to consider the minimum length T of the analyzed segment. 下限 , the minimum length T of the analyzed fragment 下限 and the maximum duration T of the highlight segment 长高光 The larger of the two is taken as the duration T of the analysis segment. 分析 , then the following relationship exists:

[0239] T 分析 =max(T长高光 ,T 下限 ).

[0240] Among them, max is the symbol for taking the maximum number.

[0241] It should be understood that the minimum length T of the fragment to be analyzed 下限 and the maximum duration T of the highlight segment 长高光 The larger of the two is taken as the duration T of the analysis segment. 分析 , the duration of the analysis segment can be appropriately increased to cover as many locations of highlight segments as possible, thereby improving the accuracy of the highlight segments finally extracted.

[0242] (4) According to the duration T of the analysis segment 分析 , and the available analysis time T allocated for the i-th video 预期分析 , determine the number of analyzed fragments N 分析 .

[0243] In one implementation, N 分析 It can be calculated by the following relationship:

[0244]

[0245] in, The floor symbol.

[0246] In another implementation, after N is calculated in the above manner, 分析 After that, the available analysis time T allocated to the i-th video 预期分析 There may be a time T that is shorter than the analyzed segment. 分析 The remaining analysis time T 剩余 :

[0247] T 剩余 =T 预期分析 -N 分析 *T 分析 .

[0248] Remaining analysis time T 剩余 May be greater than the minimum length T of the highlight segment 短高光 In this case, N 分析 Add another analysis segment based on the 剩余 As the duration of the last analysis segment.

[0249] Specifically, N 分析 It can be calculated by the following relationship:

[0250]

[0251] It should be understood that when the remaining analysis time is T 剩余 Greater than the minimum length T of the highlight segment 短高光 By adding an analysis segment, all the expected analysis time can be effectively utilized to cover more locations of highlight segments, thereby improving the accuracy of the highlight segments finally extracted.

[0252] (5) According to the number of analyzed fragments N 分析 , the total duration of the i-th video is T 总 , and the available analysis time T allocated for the i-th video 预期分析 , determine the interval length T of the analysis segment 间隔 .

[0253] In one implementation, the minimum interval T of highlight segments is not considered. 最小间隔 , T 间隔 It can be directly calculated through the following relationship:

[0254]

[0255] In another implementation, the minimum interval T of the highlight fragments needs to be considered. 最小间隔 , the T calculated in the above way is 间隔 Minimum interval T from highlight fragment 最小间隔 The larger one is taken as the interval length T of the analysis segment. 间隔 , then the following relationship exists:

[0256] T 间隔 =max(T 间隔 ,T 最小间隔 ).

[0257] It should be understood that the minimum interval T of the highlight fragments 最小间隔 is a minimum interval calculated by the application layer. In order to avoid the problem of too small interval leading to too concentrated analysis fragments, when T 间隔 Smaller than the minimum interval T of highlight fragments 最小间隔 When 间隔 Set to T 最小间隔 .

[0258] So far, the policy monitoring module has obtained three target parameters: the number of analysis segments of the material video N 分析 , the duration of the analysis segment T 分析 , the interval between adjacent segments T 间隔 Then, according to these three target parameters, each analysis segment is evenly distributed. Specifically, Fig.15 As shown, the analysis segments are allocated at equal intervals starting from the first frame of the i-th video file. The duration of the analysis segments N1, N2, N3, etc. is T分析 , the duration of the last analysis segment Nn is T 剩余 .

[0259] It should be understood that compared with the full analysis strategy that decodes, reduces the resolution and scores all video frames, the intensive key segment analysis strategy only needs to decode, reduce the resolution and score the video frames of a few key segments. Therefore, the intensive key segment analysis strategy can obtain highlight segments faster and generate a video set composed of highlight segments.

[0260] Exemplarily, the sparse key segment analysis strategy refers to a strategy of dividing the candidate video into N analysis segments and performing analysis and processing on the N analysis segments, wherein the intervals between the N analysis segments are of different durations.

[0261] For example, the sparse key segment analysis strategy can be to first calculate the minimum length T of the highlight segment. 短高光 , the maximum duration of the highlight segment T 长高光 , the minimum interval T of highlight fragments 最小间隔 , the available analysis time T allocated for the i-th video 预期分析 , and the total duration T of the i-th video 总 , calculate three target parameters: the number of analysis segments N of the source video 分析 , the duration of the analysis segment T 分析 , the interval between adjacent segments T 间隔 (like Fig.13 Then, determine the starting point of the first analysis segment (as shown in Fig.14 Then, according to the three target parameters, the intervals between adjacent segments are distributed unequally, and the analysis segments are mainly distributed in the front and middle sections of the video, and the tail section is appropriately distributed. Finally, each analysis segment of the i-th video is scored to obtain the analysis results of the highlight segments.

[0262] For example, Fig.15 As shown in the figure, the first frame of the i-th video, or the I frame at the 2nd second position, or a frame in the I frame at the 1 / 5 position of the video that meets the picture quality requirements is used as the starting point of the first analysis segment N1. Then, starting from the starting point, the duration is set to T according to different intervals. 分析 The interval between analysis segment N1 and analysis segment N2 is T1, the interval between analysis segment N2 and analysis segment N3 is T2 (T2 = 2T1) ... and so on, the duration of the analysis segment of the last analysis segment Nn is set to T 剩余 .

[0263] It should be noted that the number of analysis segments N for the source video is 分析 , the duration of the analysis segment T分析 , the interval between adjacent segments T 间隔 The specific method of obtaining these three target parameters is the same as the method of obtaining the three target parameters by the intensive key segment analysis strategy.

[0264] Combine the following Fig.14 , the process of determining the starting point of the first analysis segment is illustrated.

[0265] D1, the strategy monitoring module sends the file descriptor fdi of the i-th video to the pipeline interface, and the analysis strategy (sparse key segment analysis strategy) set for the i-th video.

[0266] D2, the pipeline interface decodes the three key frames of the i-th video according to the file descriptor fdi of the i-th video and the analysis strategy (sparse key segment analysis strategy) set for the i-th video.

[0267] The policy monitoring module sends down the analysis policy (sparse key segment analysis policy) set for the i-th video, so that the pipeline interface can determine that the video frames to be decoded are the three key frames of the i-th video according to the policy.

[0268] For example, the three key frames may include the 1st frame located at the start point of the ith video, an I frame located at the 2nd second position, and an I frame located at the 1 / 5 position of the ith video.

[0269] It should be noted that, unlike the simple I-frame analysis strategy, in which the third key frame is "the I-frame at the 1 / 3 position of the i-th video", the third key frame of the sparse key segment analysis strategy is "the I-frame at the 1 / 5 position of the i-th video". This is because the simple I-frame analysis strategy directly determines a highlight segment, while the sparse key segment analysis strategy needs to first determine multiple analysis segments, and then select the highlight segment from the multiple analysis segments. In order to make multiple analysis segments cover the front and middle sections of the i-th video as much as possible, the third key frame of the sparse key segment analysis strategy is closer to the starting position of the video.

[0270] In addition, the embodiment of the present application is illustrated by taking the third key frame of the sparse key segment analysis strategy as "the I frame located at the 1 / 5 position of the i-th video" as an example, and it can also be "the I frame located at the 1 / 4 position of the i-th video", "the I frame located at the 1 / 6 position of the i-th video", etc., and can be adjusted according to actual needs.

[0271] The D3,pipeline interface converts the three decoded key frames from the first format to the second format.

[0272] Exemplarily, the first format is the nv12 format, and the second format is the i420 format.

[0273] D4, the pipeline interface reduces the three keyframes after changing the format from the first resolution to the second resolution.

[0274] Exemplarily, the first resolution is 1080p and the second resolution is 480p.

[0275] In some embodiments, the second resolutions corresponding to different analysis strategies may be different or the same. Therefore, in the above D2, the strategy monitoring module sends the analysis strategy (sparse key segment analysis strategy) set for the i-th video, so that the pipeline interface can determine the second resolution corresponding to the sparse key segment analysis strategy.

[0276] D5, the pipeline interface converts the three key frames after reducing the resolution from the second format to the first format.

[0277] In order to improve the processing speed of the highlight segment algorithm interface for the three key frames, the pipeline interface can reduce the resolution of the video frame before performing highlight analysis on the video frame. In addition, since the resolution reduction algorithm only supports the second format, it is necessary to first convert the three key frames from the first format to the second format, then reduce the resolution of the video frame, and then convert the reduced resolution video frame to the original first format.

[0278] D6: The pipeline interface stores the three key frames converted again into the first format in a memory (buffer).

[0279] D7, the pipeline interface sends the data address of the first frame located at the starting point of the i-th video to the highlight segment algorithm interface through the one-key segment interface.

[0280] D8, the highlight segment algorithm interface obtains the first frame according to the data address of the first frame, and then analyzes the first frame based on the preset highlight segment algorithm to obtain the score of the first frame, and then returns the score of the first frame to the policy monitoring module through the one-key segment interface and the pipeline interface.

[0281] D9, the policy monitoring module determines whether the score of the first frame is greater than or equal to a preset value.

[0282] Take the preset value of 60 points as an example. If the score of the first frame is greater than or equal to 60 points, the first frame can be used as the starting point of the first analysis segment, and D16 is executed without executing D10-D15. If the score of the first frame is less than 60 points, the following D10 can be executed.

[0283] D10, the policy monitoring module notifies the pipeline interface to send the data address of the I frame at the 2nd second position to the highlight segment algorithm interface. The pipeline interface sends the data address of the I frame at the 2nd second position to the highlight segment algorithm interface through the one-key segment interface.

[0284] D11, the highlight segment algorithm interface obtains the I frame at the 2nd second position according to the data address of the I frame at the 2nd second position, and then analyzes the I frame at the 2nd second position based on the preset highlight segment algorithm to obtain the score of the I frame at the 2nd second position, and then returns the score of the I frame at the 2nd second position to the policy monitoring module through the one-key film interface and the pipeline interface.

[0285] D12, the policy monitoring module determines whether the score of the I frame at the 2nd second position is greater than or equal to a preset value.

[0286] Still taking the preset value of 60 points as an example, if the score of the I frame at the 2nd second position is greater than or equal to 60 points, then the I frame at the 2nd second position can be used as the starting point of the first analysis segment, and D16 is executed without executing D13-D15. If the score of the I frame at the 2nd second position is less than 60 points, then the following D13 can be executed.

[0287] D13, the policy monitoring module notifies the pipeline interface to send the data address of the I frame at the 1 / 5 position to the highlight segment algorithm interface. The pipeline interface sends the data address of the I frame at the 1 / 5 position to the highlight segment algorithm interface through the one-key segment interface.

[0288] D14, the highlight segment algorithm interface obtains the I frame at the 1 / 5 position according to the data address of the I frame at the 1 / 5 position, and then analyzes the I frame at the 1 / 5 position based on the preset highlight segment algorithm to obtain the score of the I frame at the 1 / 5 position, and then returns the score of the I frame at the 1 / 5 position to the policy monitoring module through the one-key segmentation interface and the pipeline interface.

[0289] D15, the policy monitoring module determines whether the score of the I frame located at the 1 / 5 position is greater than or equal to a preset value.

[0290] Still taking the preset value of 60 points as an example, if the score of the I frame at the 1 / 5 position is greater than or equal to 60 points, then the I frame at the 1 / 5 position can be used as the starting point of the first analysis segment, and D16 is executed. If the score of the I frame at the 1 / 5 position is also less than 60 points, then the video frame with the highest score among the three key frames can be used as the starting point of the first analysis segment, and B16 is executed.

[0291] It should be noted that the above description of D9, D12 and D15 is based on the example that the preset values ​​of the three positions are equal. In actual implementation, the three preset values ​​may not be completely equal, and the embodiment of the present application does not limit this.

[0292] D16, the policy monitoring module determines one of the three key frames as the starting point of the first analysis segment.

[0293] After determining the starting point of the first analysis segment, the strategy monitoring module also needs to calculate the interval length T of adjacent segments. 间隔 , and the minimum interval T of highlight fragments 最小间隔 , calculate the intervals of each analysis segment.

[0294] Specifically, the first interval duration T1 set between the first analysis segment N1 and the second analysis segment N2 is equal to max(T 间隔 / n,T 最小间隔 ). The interval length T after the second analysis segment N2 i =min(m*T i-1 , T 间隔 ), T i-1 T i The length of the previous interval of . Where n and m are both greater than 1, and i is an integer greater than or equal to 2.

[0295] The maximum number of analyzed fragments N is 上限 =5, n=4, m=2, combined with Fig.16 , an example is given to illustrate the process of the policy monitoring module calculating the intervals of each analysis segment.

[0296] When the number of analyzed fragments N 分析 ≥2 (for example, N 分析 =2, 3, 4 or 5), a first interval duration T1 is set between the first analysis segment N1 and the second analysis segment N2.

[0297] The process of obtaining the first interval duration T1 is as follows Fig.16 As shown in (a):

[0298] The strategy monitoring module determines T 间隔 Is / 4 greater than T? 最小间隔 ;

[0299] If T 间隔 / 4>T 最小间隔 , then T1=T 间隔 / 4;

[0300] If T 间隔 / 4≤T 最小间隔 , then T1=T 最小间隔.

[0301] It should be understood that the minimum interval T of the highlight fragments 最小间隔 is a minimum interval calculated by the application layer. In order to avoid the problem of too small interval leading to too concentrated analysis fragments, when T 间隔 / 4 is less than or equal to the minimum interval T of the highlight fragment 最小间隔 When the first interval duration T1 is set to T 最小间隔 .

[0302] When the number of analyzed fragments N 分析 ≥3 (for example, N 分析 =3, 4 or 5), a second interval duration T2 is set between the second analysis segment N2 and the third analysis segment N3.

[0303] The process of obtaining the second interval duration T2 is as follows Fig.16 As shown in (b):

[0304] The strategy monitoring module determines whether 2*T1 is less than T 间隔 ;

[0305] If 2*T1 <T 间隔 , then T2=2*T1;

[0306] If 2*T1≥T 间隔 , then T2=T 间隔 .

[0307] It should be understood that by 间隔 Taking the smaller value as T2 can shorten the interval between the second analysis segment N2 and the third analysis segment N3, so that the analysis segments are more concentrated in the front and middle sections of the video. Since the front and middle sections of the video are more likely to be the content of interest to the user, this setting is more conducive to extracting the highlight segments more accurately and quickly.

[0308] When the number of analyzed fragments N 分析 ≥4 (for example, N 分析 =4 or 5), a third interval duration T3 is set between the third analysis segment N3 and the fourth analysis segment N4.

[0309] The process of obtaining the third interval duration T3 is as follows Fig.16 As shown in (c):

[0310] The strategy monitoring module determines whether 2*T2 is less than T 间隔 ;

[0311] If 2*T2 <T 间隔 , then T3=2*T2;

[0312] If 2*T2≥T间隔 , then T3=T 间隔 .

[0313] It should be understood that by 间隔 Taking the smaller value as T3 can shorten the interval between the third analysis segment N3 and the fourth analysis segment N4, so that the analysis segments are more concentrated in the front and middle sections of the video. Since the front and middle sections of the video are more likely to be the content of interest to the user, this setting is more conducive to extracting the highlight segments more accurately and quickly.

[0314] When the number of analyzed fragments N 分析 ≥5 (for example, N 分析 =5), a fourth interval duration T4 is set between the fourth analysis segment N4 and the fifth analysis segment N5. The process of obtaining the fourth interval duration T4 is as follows: Fig.16 As shown in (d):

[0315] The process of obtaining the fourth interval duration T4 is as follows: Fig.16 As shown in (d):

[0316] The strategy monitoring module determines whether 2*T3 is less than T 间隔 ;

[0317] If 2*T3 <T 间隔 , then T4=2*T3;

[0318] If 2*T3≥T 间隔 , then T4=T 间隔 .

[0319] It should be understood that by 间隔 Taking the smaller value as T4 can shorten the interval between the fourth analysis segment N4 and the fifth analysis segment N5, so that the analysis segments are more concentrated in the front and middle sections of the video. Since the front and middle sections of the video are more likely to be the content of interest to the user, this setting is more conducive to extracting the highlight segments more accurately and quickly.

[0320] It should be noted that, in some cases, after setting the segments and segment intervals, the remaining duration of the last analysis segment Nn is less than the duration T of an analysis segment. 分析 For example, Fig.16 As shown in (e) in the figure, when N 分析 =4 or 5, the segment corresponding to the remaining length of the i-th video can be used as the last analysis segment Nn.

[0321] In summary, the sparse key segment analysis strategy has two characteristics:

[0322] 1. The first analysis segment may not be the first frame of the video. It is necessary to select a frame from the first video frame at the starting point of the first video, the video frame at the 2nd second position of the first video, and the video frame at the 1 / 5 position of the first video as the first frame of the analysis segment.

[0323] 2. The interval lengths between analysis segments in the sparse key segment analysis strategy are different. As the length of the candidate video increases, the interval length between two analysis segments becomes longer.

[0324] Exemplarily, the sparse key segment analysis strategy divides the candidate video into 4 analysis segments, which are analysis segment 1, analysis segment 2, analysis segment 3 and analysis segment 4 in chronological order from front to back. Among them, the interval between analysis segment 1 and analysis segment 2 is 1s, the interval between analysis segment 2 and analysis segment 3 is 2s, and the interval between analysis segment 3 and analysis segment 4 is 4s.

[0325] It should be understood that compared with the intensive key segment analysis strategy, the sparse key segment analysis strategy focuses on analyzing the front and middle sections of the video, and the starting point of the first analyzed segment may not be the first video frame. Therefore, the sparse key segment analysis strategy may be less time-consuming, so that highlight segments can be obtained faster and a video set composed of highlight segments can be generated.

[0326] Exemplarily, the I-frame sampling analysis strategy refers to selecting M frames of images in the candidate video as the starting frames of the analysis segment, obtaining M analysis segments, and selecting one of the M analysis segments as the highlight segment. In one possible case, the M frames of images refer to 3 frames of images.

[0327] For example, the I-frame sampling and analysis strategy may be to sequentially select the first frame of the candidate video, the second frame, and the one-third frame at the total length of the candidate video, that is, to select a total of three frames, analyze and process these three frames to obtain highlight segments.

[0328] The frame image at the 2nd second may refer to the frame image at the 2nd second of the candidate video or a frame image adjacent to the frame image at the 2nd second.

[0329] For example, Fig.17 As shown in (a) in FIG. 1 , the frame image at the 2nd second may refer to the frame image at the 2nd second of the candidate video.

[0330] For example, Fig.17 As shown in (b), the frame image at the 2nd second may refer to the frame image of the frame before the frame image at the 2nd second of the candidate video.

[0331] For example, Fig.17As shown in (c), the frame image at the second second may refer to the frame image of the candidate video 3 frames before the frame image at the second second.

[0332] For example, Fig.17 As shown in (d) in FIG. 1 , the frame image at the 2nd second may refer to the frame image of the frame after the frame image at the 2nd second of the candidate video.

[0333] For example, Fig.17 As shown in (e) in FIG. 1 , the frame image at the 2nd second may refer to the frame image 2 frames after the frame image at the 2nd second of the candidate video.

[0334] It should be understood that compared with the sparse key segment analysis strategy, the I-frame sampling analysis strategy only analyzes and processes M-frame images, so the I-frame sampling analysis strategy may be less time-consuming, thereby obtaining highlight segments faster and generating a video set consisting of highlight segments.

[0335] Below through Fig.18 The illustrated embodiment describes in detail the specific process of how to select a target analysis strategy for each candidate video.

[0336] Fig.18 A flow chart of a method for determining a video strategy for generating a video provided in an embodiment of the present application is shown in FIG. Fig.18 As shown, the method includes:

[0337] S1. Obtain a first duration, a second duration, and a video analysis rate.

[0338] S2. Determine a target analysis strategy corresponding to the candidate video based on the first duration, the second duration, and the video analysis rate.

[0339] The following is a detailed description of S1 "obtaining the first duration, the second duration and the video analysis rate".

[0340] The first duration refers to the video duration of the candidate video, and the second duration is used to indicate the analysis duration allocated to the candidate video for analyzing and processing the candidate video.

[0341] The candidate videos may refer to Figure 3 The video material selected by the user in the interface A3 shown in (f) is the video material used to generate the one-click blockbuster.

[0342] Among them, the video editing business layer of the application layer can send the file descriptor of the candidate video to the policy monitoring module in the media platform framework layer. Exemplarily, the file descriptor of candidate video 1 is fd1, and the video editing business layer sends the file descriptor fd1 to the policy monitoring module. After the policy monitoring module receives the file descriptor fd1, it can read the video duration corresponding to candidate video 1 based on the file descriptor fd1, that is, the first duration.

[0343] It should be understood that after the policy monitoring module receives the total target analysis duration sent by the video editing business layer, it subtracts the duration occupied by the image analysis from the total target analysis duration to obtain the total analysis duration that can be allocated to all candidate videos, and then divides the total analysis duration into multiple analysis durations according to the video duration of each candidate video, that is, the analysis duration (second duration) allocated to each candidate video. Among them, the analysis duration allocated to each candidate video can be obtained through a duration allocation algorithm.

[0344] The duration allocation algorithm provided in the embodiments of the present application is described in detail below.

[0345] In one possible case, the total target analysis time received by the policy monitoring module from the video editing business layer is 20 seconds, of which the time occupied by the image analysis is 2 seconds, and the total analysis time that can be allocated to all candidate videos is 18 seconds. Take the example that all candidate videos are 3 videos, and the above 3 videos include video 1, video 2 and video 3. Among them, the video length (first length) of video 1 is 15 seconds, the video length (first length) of video 2 is 10 seconds, and the video length (first length) of video 3 is 20 seconds. The total analysis time of 18 seconds is allocated to each video according to the video length of each video, and the analysis time (second length) of video 1 is 6 seconds, the analysis time (second length) of video 2 is 4 seconds, and the analysis time (second length) of video 3 is 8 seconds.

[0346] In one possible case, you can also Fig.19 The duration allocation algorithm shown is used to obtain the actual analysis duration allocated to each video material (candidate video).

[0347] Let's combine Fig.19 The input parameters and output parameters of the duration allocation algorithm are explained, and then the logic of the duration allocation algorithm is explained. Finally, an example is given to illustrate how to allocate the actual analysis duration of each video material according to the duration allocation algorithm.

[0348] refer to Fig.19 As shown, in the embodiment of the present application, for multiple video material files selected by the user, based on the total number n of video material files, the duration L of each video n , the sum of all video durations L A, the analysis speed of each video S n , the sum of all video analysis speeds S A , Total target analysis time T A , the actual analysis time allocated to each video material file can be determined by using the time allocation algorithm for calculation.

[0349] First, the main input parameters of the duration allocation algorithm are explained. Fig.19 As shown, the input data of the duration allocation algorithm includes the following six items:

[0350] (1) The total number of video material files (denoted as n).

[0351] (2) The duration of each video (denoted as L n ).

[0352] For example, the duration of the first video is L1, the duration of the second video is L2, and so on. The duration of the nth video is L n . Among them, by parsing a single video material file, the duration of the video material file can be obtained.

[0353] (3) Total video length (denoted as L) A ).

[0354] The total video duration refers to the sum of all video durations, i.e. L A =L1+…+L n .

[0355] (4) The analysis speed of each video (denoted as S n ).

[0356] For example, the analysis speed of the first video is S1, the analysis speed of the second video is S2, and so on. The analysis speed of the nth video is S n The analysis speed of a single video may be obtained from the HAL layer. For example, the analysis speed of a single video may be 3 times, 4 times, 5 times, or other analysis speeds.

[0357] The video analysis speed is related to the video resolution and frame rate. For example, the higher the video resolution, the lower the video analysis speed. The higher the video frame rate, the lower the video analysis speed. Since the resolution and frame rate of each video material file are different, the analysis speed of each video material file may be different.

[0358] In the embodiment of the present application, the analysis speed of a single video ultimately depends on the video decoding speed and the chip analysis speed. In other words, the analysis speed of a single video can be determined based on the chip analysis speed and the video decoding speed. The following is a detailed description of how to calculate the analysis speed S of each video.n process.

[0359] In the embodiment of the present application, the analysis speed S of a single video can be calculated using the following equation: n :

[0360] S n =min(V1, V2)×α; (Equation 1-1).

[0361] Wherein, V1 represents the video decoding speed (decodeSpeed), V2 represents the chip analysis speed (chipestSpeed), α represents the tolerance factor, and α is taken as 0.9 as an example. min(V1, V2) means taking the smaller value of V1 and V2.

[0362] In the embodiment of the present application, the chip analysis speed V2 is a performance parameter obtained from the HAL layer during initialization. For example, the chip analysis speed V2 can be 3 times, 4 times, or 5 times, which can be determined according to the actual situation of the chip. For the sake of convenience, the embodiment of the present application takes the chip analysis speed V2 as 5 times as an example for exemplary description.

[0363] In the embodiment of the present application, the default video decoding speed (denoted as V d ) is used to calculate the video decoding speed V1. The process of calculating the video decoding speed V1 is described in detail below.

[0364] First, the video decoding speed V1 can be calculated according to the resolution and frame rate of the video using the following equation:

[0365] V1=V d ×(R d / R A )×(F d / F A ); (Equation 1-2).

[0366] Among them, V d Indicates the default video decoding speed, R d Indicates the default video resolution, R A Indicates the actual video resolution. d Indicates the default video frame rate, F A Indicates the actual video frame rate.

[0367] For example, the default video decoding speed V d Can be 4 times faster, the default video resolution is R d You can choose 1080p (i.e. 1920×1080), the default video frame rate F d It can take 30 frames per second.

[0368] At the actual video resolution R A is 720p (i.e. 1280×720), the actual video frame rate F A Taking a video with a frame rate of 60 frames per second as an example, the above parameters are substituted into equation 1-2 to calculate the video decoding speed V1:

[0369] V1=V d ×(R d / R A )×(R d / R A )=4×(1920×1080) / (1280×720)×(30 / 60)=4.5.

[0370] Therefore, the video decoding speed V1 is calculated by the above equation 1-2.

[0371] Next, we use the above equation 1-1 to calculate the analysis speed S of a single video: n Here we assume that the chip analysis speed V2 is 5 times faster. Substitute the video decoding speed V1 and the chip analysis speed V2 into equation 1-1 to calculate the single video analysis speed S n :

[0372] S n =min(4.5,5)×0.9=4.5×0.9≈4.

[0373] It can be seen that the present application solution determines the analysis speed of a single video based on the chip analysis speed and the video decoding speed; wherein the video decoding speed is determined based on the resolution and frame rate of the single video. In other words, the present application solution calculates the analysis speed of the single video based on the chip analysis speed and the resolution and frame rate of the single video.

[0374] (5) The sum of all video analysis speeds (denoted as S A ).

[0375] S A It is the sum of the analysis speeds of all videos to be analyzed.

[0376] (6) Total time of target analysis (denoted as T A ).

[0377] Total target analysis time T A The time it takes to complete the analysis of all video material files, which is specifically issued by the application layer. First, calculate the analysis speed of all video material files, and then preliminarily calculate the total target analysis time T based on the analysis speed of all video material files. A Then, according to the duration allocation algorithm provided in the embodiment of the present application, the total duration T of the target analysis isA Assigned to all video footage files.

[0378] It should be noted that in the embodiment of the present application, the multiple video material files may all be videos, or may include one or more videos and one or more pictures, or may all be pictures. For videos, the query analysis speed is required; for pictures, the query analysis speed is not required.

[0379] Scenario 1: Multiple video material files only include video materials.

[0380] In some embodiments, when the plurality of video material files are all videos, the target analysis total duration T is calculated according to the following equation: A :

[0381] T A =∑T Vn ; (Equation 2-1).

[0382] T Vn =min(L n / S n , β); (Equation 2-2).

[0383] Among them, min(L n / S n , β) represents taking L n / S n The minimum value among L and β. n Indicates the video length of a single video, S n Indicates the analysis speed of the single video, T Vn It represents the expected analysis time of a single video in multiple video material files, which can be calculated based on the video time and analysis speed. Wherein, β is the upper limit of the expected analysis time of a single video, so that the expected analysis time of a single video can be guaranteed to be within the limit. It can be understood that in actual implementation, the upper limit of the expected analysis time of a single video can be set according to actual usage requirements, and the embodiments of the present application are not limited thereto.

[0384] For example, β is 20 seconds (s). The expected analysis duration of a single video is expressed as:

[0385] T Vn =min(L n / S n , 20s); (Equation 2-3).

[0386] For example, the video length L of a video n is 60s, the analysis speed of the video is S n is 5 times faster, then L n / S nis 12 seconds. According to the above equation 2-3, it can be judged that the expected analysis time of this video is T Vn =min(12s, 20s)=12s.

[0387] For example, the video length L of a video n is 200s, the analysis speed of the video is S n is 5 times faster, then L n / S n is 40 seconds. According to the above equation 2-3, the expected analysis time of the video is T Vn =min(40s, 20s)=20s.

[0388] That is, when multiple video material files are all videos, the target total analysis time T is calculated based on the sum of the expected analysis time of all videos to be analyzed. A For example, suppose the user selects 5 videos as the video material files for one-click filming, queries the analysis speed of each video, and calculates the expected analysis time of each video based on the analysis speed of each video. Then, the expected analysis time of these 5 videos is summed up to obtain the target total analysis time T. A For example, the total time T of the target analysis is A Taking 20 seconds as an example, the embodiment of the present application will allocate 20 seconds to the five videos according to the duration allocation algorithm. How to calculate the actual analysis duration allocated to each video will be described in detail below.

[0389] Scenario 2: Multiple video material files include video materials and picture materials.

[0390] In some other embodiments, when the plurality of video material files include one or more videos and one or more pictures, the target analysis total duration T is calculated according to the following equation: A :

[0391] T A =∑T Vn +∑T P ; (Equation 3-1).

[0392] T Vn =min(L n / S n , β); (Equation 3-2).

[0393] T P =γ; (Equation 3-3).

[0394] Among them, T Vn Indicates the expected analysis duration of a single video in multiple video material files, T PIndicates the expected analysis time of a single picture in multiple video material files, γ is a preset value, and the value of γ can be determined according to actual usage requirements, and is not limited in the embodiment of the present application. For example, γ is 1, which means that the expected analysis time of a single picture is 1 second. The following description is based on the expected analysis time of a single picture being 1 second.

[0395] Assuming that the multiple video material files include n videos and m pictures, the sum of the expected analysis durations of the m pictures is m. Accordingly, the target total analysis duration T can be calculated according to the following equation: A :

[0396] T A =∑min(L n / S n , β)+m; (Equation 3-4).

[0397] It can be seen that when multiple video material files include videos and pictures, the sum of the expected analysis durations of all videos and the sum of the expected analysis durations of all pictures can be added to obtain the target total analysis duration T A .

[0398] It should be noted that in the embodiment of the present application, the expected analysis duration of each picture can be a fixed duration, that is, the expected analysis duration assigned to each picture is fixed. The expected analysis duration of each picture can be set according to actual usage requirements, and the embodiment of the present application is not limited. For example, the expected analysis duration of each picture is 1 second. In this case, the highlight segment analysis algorithm will take 1 second to perform highlight analysis on each picture to determine whether the picture contains a highlight moment (also called a wonderful moment).

[0399] In the embodiment of the present application, the total target analysis time T calculated above is A It is the duration of analyzing all video material files by the highlight segment analysis algorithm. The actual analysis duration allocated to each video in all video material files needs to be calculated according to the duration allocation algorithm. Assuming that the actual analysis duration allocated to a certain video is 10 seconds, in this case, the highlight segment analysis algorithm will take 10 seconds to perform highlight analysis on the video to determine the highlight segments (also called wonderful segments) in the video, and then the highlight segments in the video will be intercepted as the video segment content for one-click filming.

[0400] Combination of the above Fig.19 The various input parameters of the duration allocation algorithm are explained, and then refer to Fig.19 As shown, the output data of the duration allocation algorithm is: the actual analysis duration allocated to each video material file (denoted as T nFor example, the actual analysis time allocated to the first video material file is T1, the actual analysis time allocated to the second video material file is T1, and so on, the actual analysis time allocated to the nth video material file is T n .

[0401] Combination of the above Fig.19 The input parameters and output parameters of the duration allocation algorithm are described. The algorithm rules of the duration allocation algorithm in the embodiment of the present application are described in detail below, as well as how to calculate the actual analysis duration T of each video material file according to the duration allocation algorithm. n process.

[0402] In the embodiment of the present application, the overall idea of ​​the duration allocation algorithm can be: the expected analysis duration of all videos (denoted as T V ) is divided into two parts for allocation. First, one part is evenly allocated to each video, and this part is called the basic analysis time. The other part is allocated according to the weight ratio of the video, and this part is called the weight-based analysis time. In other words, the actual analysis time allocated to a single video material file consists of the following two parts:

[0403] T n =T n1 +T n2 ; (Equation 4-1).

[0404] Among them, T n1 Indicates the basic analysis duration, T n2 Indicates the duration of the analysis based on the weight.

[0405] It should be noted that, when multiple video material files are all videos, the video to be allocated time T V The total time T for target analysis in the above scenario 1 A In the case where multiple video material files include videos and pictures, the video to be allocated time T V is the sum of the expected analysis duration of all videos in the above scenario 2 ∑T Vn .

[0406] Optionally, a portion of the video duration to be allocated in a preset ratio can be evenly distributed to each video, and the remaining portion of the video duration to be allocated can be allocated to each video according to the weight ratio corresponding to the video. The preset ratio can be set according to actual use requirements, and the embodiment of the present application is not limited thereto. For example, the preset ratio is 50%. That is, 50% of the video duration to be allocated is evenly distributed to each video, and then the other 50% of the video duration to be allocated is allocated to each video according to the weight ratio.

[0407] The basic analysis time T is calculated by the following equation: n1:

[0408] T n1 =T V ×0.5 / n; (Equation 4-2).

[0409] Where n represents the number of videos to be analyzed.

[0410] The weighted analysis duration T is calculated by the following equation: n2 :

[0411] T n2 =T V ×0.5×(W n / W A );(Equation 4-3).

[0412] W n =log2(1.5+L n / L A ) / log2(1.5+S n / S A );(Equation 4-4).

[0413] W A =W1+…+W n ; (Equation 4-5).

[0414] Among them, W n Represents the weight value of a single video. For example, W1 represents the weight of the first video, W2 represents the weight of the second video, and so on. n W represents the weight of the nth video. A It should be noted that 1.5 in equation 4-4 is a preset value of the algorithm, which can be set according to actual use requirements and is not limited in the embodiment of the present application.

[0415] In the embodiment of the present application, the longer the video material file is, the longer the actual analysis time should be allocated. In other words, the video length L of a certain video n The sum of all video durations L A The ratio L n / L A The larger the value, the more actual analysis time is allocated to the video.

[0416] In the embodiment of the present application, the faster the analysis speed of a video material file, the shorter the actual analysis time allocated. In other words, the analysis speed S of a video n The sum of all video analysis speeds S A The ratio S n / S A The larger the value, the less actual analysis time is allocated to the video.

[0417] Among them, the weight value of a single video is W n It is specifically calculated by the above equation 4-4.

[0418] For example, a plurality of video material files including two videos are used as an example for illustration. The sum of the duration of all videos L A is 200 seconds, the sum of all video analysis speeds S A The speed is 10 times. Assume that the video duration to be allocated is T V for 20 seconds.

[0419] Assuming that the video length L1 of the first video is 60 seconds and the analysis speed S1 of the video is 5 times the speed, the weight value W1 of the first video can be calculated by the above equation 4-4:

[0420] W1=log2(1.5+L1 / L A ) / log2(1.5+S1 / S A )=log2(1.5+60 / 200) / log2(1.5+5 / 10)=0.85.

[0421] Assume that the video length L2 of the second video is 140 seconds and the analysis speed S2 of the video is 5 times the speed. The weight value W2 of the second video can be calculated by the above equation 4-4:

[0422] W2=log2(1.5+L2 / L A ) / log2(1.5+S2 / S A )=log2(1.5+140 / 200) / log2(1.5+5 / 10)=1.14.

[0423] The sum of the weight values ​​of all videos can be calculated by the above equation 4-5: A Equal to 1.99.

[0424] The weighted analysis duration T of the first video is calculated using the above equation 4-3 n2 :

[0425] T n2 =T V ×0.5×(W1 / W A )=20×0.5×(0.85 / 1.99)=4.

[0426] The basic analysis time T of the first video is calculated by the above equation 4-2 n1 :

[0427] T n1 =T V ×0.5 / n=20×0.5 / 2=5.

[0428] Then the actual analysis time T1 allocated for the first video is calculated using the above equation 4-1:

[0429] T n =T n1 +T n2 =4+5=9.

[0430] It can be understood that, in other words, the actual analysis time T1 allocated to the first video is 9 seconds. V The actual analysis time allocated for the second video is 20-9=11 seconds.

[0431] It should be noted that the example of allocating time for two videos according to the time allocation algorithm is used here for illustrative explanation. It can be understood that in actual implementation, when the user selects three or more videos, the present application can allocate time for a large number of videos according to the above-mentioned time allocation algorithm, thereby allocating appropriate actual analysis time to each video.

[0432] It should be understood that during the initialization process, the algorithm module of the HAL layer returns the video analysis rate to the video editing service layer of the application layer, so the video editing service layer of the application layer can send the video analysis rate to the policy monitoring module in the media platform framework layer. Among them, the video analysis rate is a performance parameter of the electronic device itself, which is usually fixed.

[0433] The following is a detailed description of S2 "determining a target analysis strategy corresponding to the candidate video based on the first duration, the second duration and the video analysis rate".

[0434] In one possible case, after the policy monitoring module of the media platform framework layer obtains the first duration, the second duration and the video analysis rate, it can determine the percentage of frame images of the candidate video that can be processed in the analysis time allocated to each candidate video based on the first duration, the second duration and the video analysis rate, and then determine the target analysis strategy for each candidate video based on the percentage.

[0435] Exemplarily, after the video duration (first duration) of the candidate video obtained by the policy monitoring module is 10s, the analysis duration (second duration) allocated to the candidate video is 1s, and the video processing rate is 5, the analysis time required to analyze all the frame images of the candidate video can be obtained according to the video duration and video processing rate of the candidate video. For example, the analysis time required to analyze all the frame images of the candidate video is the video duration of the candidate video divided by the video processing rate, which is 10s / 5=2s. The analysis duration allocated to the candidate video is 1s. This is equivalent to the percentage of the candidate video that can be analyzed within the analysis duration allocated to the candidate video, 1s / 2s=50%.

[0436] Optionally, when the analysis time allocated to the candidate video is capable of processing a percentage of frame images in the candidate video that is greater than a first preset threshold, the target analysis strategy allocated to the candidate video is a full analysis strategy. Exemplarily, the first preset threshold is 100%.

[0437] For example, the analysis time allocated to a candidate video is 2.2s, and the analysis time required to analyze all frame images in the candidate video is 2s. The percentage of frame images in the candidate video that can be processed by the analysis time allocated to the candidate video is 2.2 / 2=110%, which is greater than the first preset threshold of 100%. Therefore, the target analysis strategy assigned to the candidate video is the full analysis strategy.

[0438] Optionally, when the analysis time allocated to the candidate video is greater than the second preset threshold and less than the first preset threshold, the target analysis strategy allocated to the candidate video is an intensive key segment analysis strategy. The second preset threshold is less than the first preset threshold. Exemplarily, the first preset threshold is 100% and the second preset threshold is 60%.

[0439] For example, the analysis time allocated to a candidate video is 1.6s, and the analysis time required to analyze all frame images of the candidate video is 2s. The percentage of frame images in the candidate video that can be processed by the analysis time allocated to the candidate video is 1.6 / 2=80%, which is less than the first preset threshold of 100% and greater than the second preset threshold of 60%. Therefore, the target analysis strategy assigned to the candidate video is an intensive key segment analysis strategy.

[0440] Optionally, when the analysis duration allocated to the candidate video is greater than a third preset threshold and less than a second preset threshold in which the percentage of frame images in the candidate video can be processed, the target analysis strategy assigned to the candidate video is a sparse key segment analysis strategy. The third preset threshold is less than the second preset threshold. Exemplarily, the third preset threshold may be a percentage obtained based on the minimum highlight duration, wherein the minimum highlight duration may refer to the shortest duration required to generate a highlight segment, and the third preset threshold may be a percentage obtained by dividing the minimum highlight duration by the video duration of the candidate video. The second preset threshold may be 60%.

[0441] Exemplarily, the minimum highlight duration t0 can be obtained based on the following formula:

[0442] t0=min((t3+t1*1.2) / v, (t3+t2*0.5) / v).

[0443] Among them, t0 refers to the minimum highlight duration, t3 refers to the analysis duration of the minimum number of frame images required for a highlight frequency band, t1 refers to the minimum duration of a highlight segment, and t2 refers to the maximum duration of a highlight segment. v refers to the video analysis rate.

[0444] For example, generating a highlight segment usually requires at least 3 frames of images, so at least 3 frames of images need to be analyzed to obtain a highlight frequency band. In one possible case, the computing performance of the terminal device requires a duration t3 of 0.2s to analyze 3 frames of images. The terminal device usually sets a duration range for the highlight segment, including a minimum and maximum duration of the highlight segment. For example, the duration range of the highlight frequency band set by the terminal device is 1s-2s. That is, the minimum duration t1 of the highlight segment is 1s, and the maximum duration t2 of the highlight segment is 2s. The video analysis rate is 5. Therefore, based on the above formula, the minimum highlight duration t0=min((0.2+1*1.2) / 5, (0.2+2*0.5) / 5)=min(0.44, 0.6)=0.44s can be obtained. The third preset threshold can be 0.44 / 2=22%.

[0445] For example, the analysis time allocated to a candidate video is 1s, and the analysis time required to analyze all frame images of the candidate video is 2s. The percentage of frame images in the candidate video that can be processed by the analysis time allocated to the candidate video is 1 / 2=50%, which is less than the second preset threshold of 60% and greater than the third preset threshold of 22%. Therefore, the target analysis strategy assigned to the candidate video is a sparse key segment analysis strategy.

[0446] Optionally, when the analysis duration allocated to the candidate video is capable of processing a percentage of frame images in the candidate video that is less than a third preset threshold, the target analysis strategy allocated to the candidate video is an I-frame sampling analysis strategy, wherein the third preset threshold may be a percentage obtained according to the minimum highlight duration.

[0447] For example, the analysis time allocated to a candidate video is 0.4s, and the analysis time required to analyze all frame images of the candidate video is 2s. The percentage of frame images in the candidate video that can be processed by the analysis time allocated to the candidate video is 0.4 / 2=20%, which is less than the third preset threshold of 22%. Therefore, the target analysis strategy assigned to the candidate video is the I frame sampling analysis strategy.

[0448] In a possible case, the target analysis strategy for each candidate video can also be determined directly according to the analysis duration (second duration) allocated to the candidate video and the analysis time required to analyze all the frame images of the candidate video. The analysis time required to analyze all the frame images of the candidate video can be obtained by the video duration (first duration) and the video processing rate of the candidate video.

[0449] Exemplarily, after the video duration (first duration) of the candidate video acquired by the strategy monitoring module is 10s, the analysis duration (second duration) allocated to the candidate video is 1s, and the video processing rate is 5, the analysis duration required to analyze all the frame images of the candidate video can be obtained by dividing the video duration of the candidate video 10s by the video processing rate 5, which is 10 / 5=2s. Then, the analysis duration allocated to the candidate video and the analysis duration required to analyze all the frame images of the candidate video are compared to determine the target analysis strategy of the candidate video.

[0450] Optionally, when the analysis time allocated to the candidate video is greater than the analysis time required to analyze all frame images of the candidate video, the target analysis strategy allocated to the candidate video is the full analysis strategy.

[0451] For example, the analysis time allocated to the candidate video is 2.2 seconds, and the analysis time required to analyze all frame images in the candidate video is 2 seconds, then the target analysis strategy allocated to the candidate video is the full analysis strategy.

[0452] Optionally, when the analysis time allocated to the candidate video is less than the analysis time required to analyze all frame images of the candidate video, and is greater than the product of the analysis time required to analyze all frame images of the candidate video and a fourth preset threshold, the target analysis strategy assigned to the candidate video is an intensive key segment analysis strategy.

[0453] For example, the analysis time allocated to the candidate video is 1.6s, the fourth preset threshold is 60%, the analysis time required to analyze all the frame images in the candidate video is 2s, the analysis time allocated to the candidate video is 1.6s, which is less than the analysis time required to analyze all the frame images in the candidate video (2s), and is greater than the analysis time required to analyze all the frame images in the candidate video (2s) multiplied by the fourth preset threshold of 60% (2*60%=1.2s). Then the target analysis strategy allocated to the candidate video is an intensive key segment analysis strategy.

[0454] Optionally, when the analysis time allocated to the candidate video is less than the product of the analysis time required to analyze all the frame images of the candidate video and the fourth preset threshold, and is greater than the fifth preset threshold, the target analysis strategy allocated to the candidate video is a sparse key segment analysis strategy. The fifth preset threshold may refer to the minimum highlight duration.

[0455] Exemplarily, the minimum highlight duration t0 can be obtained based on the following formula:

[0456] t0=min((t3+t1*1.2) / v, (t3+t2*0.5) / v).

[0457] Among them, t0 refers to the minimum browsing time, t3 refers to the analysis time of the minimum number of frame images required for a highlight frequency band, t1 refers to the minimum duration of a highlight segment, and t2 refers to the maximum duration of a highlight segment. v can refer to the video analysis rate.

[0458] For example, generating a highlight segment usually requires at least 3 frames of images, so at least 3 frames of images need to be analyzed to obtain a highlight frequency band. In one possible case, the computing performance of the terminal device requires a duration t3 of 0.2s to analyze 3 frames of images. The terminal device usually sets a duration range for the highlight segment, including a minimum and maximum duration of the highlight segment. For example, the duration range of the highlight frequency band set by the terminal device is 1s-2s. That is, the minimum duration t1 of the highlight segment is 1s, and the maximum duration t2 of the highlight segment is 2s. The video analysis rate is 5. Therefore, based on the above formula, the minimum highlight duration t0=min((0.2+1*1.2) / 5, (0.2+2*0.5) / 5)=min(0.44, 0.6)=0.44s can be obtained. That is, the fifth preset threshold is 0.44s.

[0459] For example, the analysis time allocated to the candidate video is 1s, the analysis time required to analyze all the frame images of the candidate video is 2s, the fourth preset threshold is 60%, and the fifth preset threshold is 0.44s. The analysis time allocated to the candidate video is 1s, which is less than the product of 2s and 60% of the fourth preset threshold (2*60%=1.2s), and is greater than the fifth preset threshold 0.44, then the target analysis strategy allocated to the candidate video is the I frame sampling analysis strategy.

[0460] Optionally, when the analysis duration allocated to the candidate video is less than a fifth preset threshold, the target analysis strategy allocated to the candidate video is a sparse key segment analysis strategy, wherein the fifth preset threshold may refer to a minimum highlight duration.

[0461] For example, the analysis time allocated to the candidate video is 0.4s, and the fifth preset threshold is 0.44s. The analysis time allocated to the candidate video, 0.4s, is less than the fifth preset threshold 0.44s, so the target analysis strategy allocated to the candidate video is the I frame sampling analysis strategy.

[0462] The specific process of step 300 "generating a target video based on the analysis results of multiple videos to be selected" is described below.

[0463] Among them, based on the analysis results of multiple candidate videos, generating a target video can be as follows: Fig. 20 As shown, including:

[0464] S301. The application function layer in the application layer edits and screens the material files selected by the user according to the analysis results.

[0465] The material files selected by the user include candidate videos and pictures.

[0466] S302. The application function layer calls the topic summary interface in the media platform framework layer to obtain the topic.

[0467] S303: The theme summary interface determines a theme template that matches the content according to the screen scene.

[0468] S304. The theme summary interface sends the theme template to the application function layer.

[0469] S305: The application function layer sends the theme, the edited video clips, and the screened pictures to the basic capability layer in the application layer.

[0470] S306: The basic capability layer splices the clipped videos and the screened pictures, and applies a theme template to obtain a target video.

[0471] S307: The basic capability layer instructs the video editing service layer to display the target video.

[0472] S308: The video editing service layer displays the target video in the album.

[0473] In one embodiment of the present application, Fig.21 As shown, a method for determining a strategy for generating a video is provided, and the method is applied in an electronic device, comprising:

[0474] S401. Obtain a first duration, a second duration, and a video analysis rate.

[0475] The first duration refers to the video duration of the candidate video, and the second duration is used to indicate the analysis duration allocated to the candidate video for analysis and processing.

[0476] S402: Determine a target analysis strategy corresponding to the candidate video from an analysis strategy set based on the first duration, the second duration, and the video analysis rate.

[0477] The analysis strategy set includes at least two analysis strategies, and different analysis strategies may take different time periods to analyze and process the candidate videos.

[0478] S403: Analyze and process the candidate videos based on the target analysis strategy to obtain analysis results of the candidate videos.

[0479] S404: Obtain a target video according to the analysis results of the candidate videos.

[0480] In an embodiment of the present application, by obtaining the first duration, the second duration and the video analysis rate, then determining the target analysis strategy corresponding to the candidate video from the analysis strategy set based on the first duration, the second duration and the video analysis rate, and then analyzing and processing the selected video based on the target analysis strategy to obtain the analysis results of each selected video, and finally, generating the target video according to the analysis results of multiple selected videos. Among them, the first duration refers to the video duration of the candidate video, the second duration is used to indicate the analysis duration allocated to the candidate video for analyzing and processing the candidate video, and the analysis strategy set includes at least two analysis strategies, and different analysis strategies are used to analyze and process the candidate video. The required duration is different. That is to say, the target analysis strategy corresponding to the candidate video is determined from multiple analysis strategies through the video duration of the candidate video, the analysis duration allocated to the candidate video for analysis and processing, and the video analysis rate of the electronic device itself for analyzing the video. Compared with the traditional technology of analyzing and processing all frame images in the candidate video, the time occupied by analyzing and processing the candidate video is shortened, so the analysis results of the candidate video can be obtained faster, and then the target video can be obtained faster, thereby improving the user experience.

[0481] In the embodiments of the present application, Fig. 22 As shown, a method for determining a strategy for generating a video is provided, and the method is applied in an electronic device, comprising:

[0482] S501: Display a first interface, where the first interface includes a first control.

[0483] The first control is used to generate a target video. For example, the first control may be a control for "one-click blockbuster".

[0484] S502: In response to a first operation on a first control, display a second interface.

[0485] The second interface is used to display multiple videos and / or pictures.

[0486] S503: In response to a second operation on the second interface, determine candidate videos.

[0487] The second operation is used to select a candidate video from multiple videos and / or pictures.

[0488] S504: Obtain a first duration, a second duration, and a video analysis rate.

[0489] The first duration refers to the video duration of the candidate video, and the second duration is used to indicate the analysis duration allocated to the candidate video for analysis and processing.

[0490] S505: Determine a target analysis strategy corresponding to the candidate video from the analysis strategy set based on the first duration, the second duration, and the video analysis rate.

[0491] The analysis strategy set includes at least two analysis strategies, and different analysis strategies may take different time periods to analyze and process the candidate videos.

[0492] S506: Analyze and process the candidate videos based on the target analysis strategy to obtain analysis results of the candidate videos.

[0493] S507: Obtain a target video according to the analysis results of the candidate videos.

[0494] It should be understood that, although each step in the flow chart in the above-described embodiment is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is clear explanation in this article, the execution of these steps does not have strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0495] It should be understood that the above examples are intended to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the specific numerical values ​​or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0496] Combination of the above Figures 1 to 22 The strategy determination method for generating a video provided by the embodiment of the present application is described in detail; Figure 2 The device embodiments of the present application are described in detail. It should be understood that the device in the embodiments of the present application can execute the various methods of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0497] Figure 2 : is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes a policy monitoring module, an algorithm module and a basic capability layer, wherein:

[0498] The policy monitoring module is used to obtain a first duration, a second duration, and a video analysis rate, wherein the first duration refers to the video duration of the candidate video, and the second duration is used to indicate the analysis duration allocated to the candidate video for analysis processing;

[0499] The strategy monitoring module is used to determine a target analysis strategy corresponding to the candidate video from the analysis strategy set based on the first duration, the second duration and the video analysis rate, wherein the analysis strategy set includes at least two analysis strategies, and different analysis strategies require different durations to analyze and process the candidate video;

[0500] The algorithm module is used to analyze and process the candidate videos based on the target analysis strategy to obtain the analysis results of the candidate videos;

[0501] The basic capability layer is used to obtain the target video based on the analysis results of the candidate videos.

[0502] It should be understood that different analysis strategies among at least two analysis strategies are to select all or part of the frame images from the candidate video for analysis and processing. Therefore, the time taken to analyze and process the candidate video using at least two analysis strategies is shorter than the traditional method of analyzing and processing all the frame images in the candidate video.

[0503] In one embodiment, the strategy monitoring module is used to determine a first ratio based on a first duration, a second duration and a video analysis rate, wherein the first ratio is used to indicate the percentage of frame images in the candidate video that have completed analysis processing within the second duration to the total frame images in the candidate video; and determine the target analysis strategy based on the first ratio.

[0504] In one embodiment, the strategy monitoring module is used to determine a third duration based on the first duration and the video analysis rate, where the third duration is used to indicate the time required to complete the analysis and processing of all frame images in the candidate video; based on the third duration and the second duration, determine the target analysis strategy corresponding to the candidate video.

[0505] In one embodiment, the at least two analysis strategies include a full analysis strategy, a dense key segment analysis strategy, a sparse key frequency band analysis strategy, and an I frame sampling analysis strategy.

[0506] In one embodiment, the full analysis strategy refers to a strategy of analyzing and processing the frame image of each frame in the candidate video.

[0507] It should be understood that since the full analysis strategy is a strategy for analyzing and processing the frame image of each frame in the candidate video, the time taken to analyze and process the candidate video using the full analysis strategy is the longest.

[0508] In one embodiment, the intensive key segment analysis strategy refers to a strategy of dividing the candidate video into N analysis segments with equal interval durations and performing analysis processing on the N analysis segments, where N is a positive integer greater than 0.

[0509] It should be understood that compared with the full analysis strategy that decodes, reduces the resolution and scores all video frames, the intensive key segment analysis strategy only needs to decode, reduce the resolution and score the video frames of a few key segments. Therefore, the time taken to analyze and process candidate videos using the intensive key segment analysis strategy is shorter than the time taken to analyze and process candidate videos using the full analysis strategy.

[0510] In one embodiment, the sparse key segment analysis strategy refers to a strategy of dividing the candidate video into N analysis segments and performing analysis processing on the N analysis segments, wherein the intervals between the N analysis segments are of different durations.

[0511] It should be understood that compared with the intensive key segment analysis strategy, the sparse key segment analysis strategy focuses on analyzing the front and middle sections of the video, and the starting point of the first analysis segment may not be the first video frame, so the sparse key segment analysis strategy may take less time.

[0512] In one embodiment, the I-frame sampling analysis strategy refers to selecting M frames of images in the candidate video as the starting frames of the analysis segment, obtaining M analysis segments, and selecting one from the M analysis segments as the highlight segment, where M is a positive integer greater than 0.

[0513] It should be noted that the above electronic device is embodied in the form of a functional module. The term "module" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0514] For example, a "module" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combined logic circuit, and / or other suitable components that support the described functions.

[0515] Therefore, the units of each example described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0516] Fig.23 A structural schematic diagram of an electronic device provided by the present application is shown. Fig.23The dotted line in the figure indicates that the unit or the module is optional; the electronic device 900 can be used to implement the strategy determination method for generating a video described in the above method embodiment.

[0517] The electronic device 900 includes one or more processors 901, which can support the electronic device 900 to implement the strategy determination method for generating a video in the method embodiment. The processor 901 can be a general-purpose processor or a special-purpose processor. For example, the processor 901 can be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0518] Optionally, the processor 901 may be used to control the electronic device 900, execute software programs, and process data of the software programs. The electronic device 900 may also include a communication unit 905 to implement input (reception) and output (transmission) of signals.

[0519] For example, the electronic device 900 may be a chip, the communication unit 905 may be an input and / or output circuit of the chip, or the communication unit 905 may be a communication interface of the chip, and the chip may be a component of a terminal device or other electronic devices.

[0520] For another example, the electronic device 900 may be a terminal device, and the communication unit 905 may be a transceiver of the terminal device, or the communication unit 905 may include one or more memories 902 in 900, on which a program 904 is stored, and the program 904 can be executed by the processor 901 to generate instructions 903, so that the processor 901 executes the strategy determination method for generating a video described in the above method embodiment according to the instructions 903.

[0521] Optionally, data may also be stored in the memory 902 .

[0522] Optionally, the processor 901 may also read data stored in the memory 902 . The data may be stored at the same storage address as the program 904 , or may be stored at a different storage address from the program 904 .

[0523] Optionally, the processor 901 and the memory 902 may be provided separately or integrated together, for example, integrated on a system on chip (SOC) of the terminal device.

[0524] Exemplarily, the memory 902 can be used to store the relevant program 904 of the strategy determination method for generating a video provided in the embodiment of the present application, and the processor 901 can be used to call the relevant program 904 of the strategy determination method for generating a video stored in the memory 902 when executing the strategy determination method for generating a video, and execute the strategy determination method for generating a video in the embodiment of the present application; for example, obtain a first duration, a second duration and a video analysis rate, wherein the first duration refers to the video duration of the candidate video, and the second duration is used to indicate the analysis duration allocated to the candidate video for analysis and processing; determine the target analysis strategy corresponding to the candidate video from the analysis strategy set based on the first duration, the second duration and the video analysis rate, wherein the analysis strategy set includes at least two analysis strategies, and different analysis strategies are used to perform analysis and processing on the candidate video. The duration required is different; the candidate video is analyzed and processed based on the target analysis strategy to obtain an analysis result of the candidate video; and the target video is obtained according to the analysis result of the candidate video.

[0525] Optionally, the present application further provides a computer program product, which, when executed by the processor 901, implements the strategy determination method for generating a video in any method embodiment of the present application.

[0526] For example, the computer program product may be stored in the memory 902 , such as a program 904 , which is finally converted into an executable target file that can be executed by the processor 901 after preprocessing, compiling, assembling, and linking.

[0527] Optionally, the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a computer, the strategy determination method for generating a video in any method embodiment of the present application is implemented. The computer program can be a high-level language program or an executable target program.

[0528] For example, the computer-readable storage medium is, for example, a memory 902. The memory 902 may be a volatile memory or a non-volatile memory, or the memory 902 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0529] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0530] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0531] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the embodiments of the electronic device described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0532] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0533] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0534] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0535] In addition, the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0536] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0537] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims. In short, the above is only a preferred embodiment of the technical solution of the present application, and is not used to limit the protection scope of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for determining a strategy for generating a video, characterized in that: The method is applied in an electronic device, and the method comprises: Obtaining a first duration, a second duration, and a video analysis rate, wherein the first duration refers to the video duration of the candidate video, and the second duration is used to indicate the analysis duration allocated to the candidate video for analysis processing; Determining a target analysis strategy corresponding to the candidate video from an analysis strategy set based on the first duration, the second duration, and the video analysis rate, wherein the analysis strategy set includes at least two analysis strategies, and different analysis strategies are used to perform analysis on the candidate video for different durations; Analyze and process the candidate video based on the target analysis strategy to obtain an analysis result of the candidate video; Obtaining a target video according to the analysis results of the candidate videos; The step of determining a target analysis strategy corresponding to the candidate video from an analysis strategy set based on the first duration, the second duration, and the video analysis rate includes: When it is determined based on the first duration and the second duration that the percentage of frame images in the candidate video processed within the second duration is greater than a first preset threshold, the target analysis strategy is a full analysis strategy.

2. The method according to claim 1, characterized in that The determining, from a set of analysis strategies based on the first duration, the second duration, and the video analysis rate, a target analysis strategy corresponding to the candidate video includes: Determine a first ratio based on the first duration, the second duration, and the video analysis rate, wherein the first ratio is used to indicate a percentage of frame images in the candidate video that have completed analysis processing within the second duration to a total number of frame images in the candidate video; The target analysis strategy is determined according to the first ratio.

3. The method according to claim 1, characterized in that The determining, from a set of analysis strategies based on the first duration, the second duration, and the video analysis rate, a target analysis strategy corresponding to the candidate video includes: Determine a third duration according to the first duration and the video analysis rate, wherein the third duration is used to indicate the duration required to complete the analysis process of all frame images in the candidate video; A target analysis strategy corresponding to the candidate video is determined based on the third duration and the second duration.

4. The method according to any one of claims 1 to 3, characterized in that: The at least two analysis strategies include a full analysis strategy, a dense key segment analysis strategy, a sparse key frequency band analysis strategy and an I frame sampling analysis strategy.

5. The method according to claim 4, characterized in that The full analysis strategy refers to a strategy for analyzing and processing the frame image of each frame in the candidate video.

6. The method according to claim 4, characterized in that The intensive key segment analysis strategy refers to a strategy of dividing the candidate video into N analysis segments with the same interval duration, and performing analysis and processing on the N analysis segments, wherein N is a positive integer greater than 0.

7. The method according to claim 4, characterized in that The sparse key segment analysis strategy refers to a strategy of dividing the candidate video into N analysis segments and performing analysis and processing on the N analysis segments, wherein the intervals between the N analysis segments are of different durations.

8. The method according to claim 4, characterized in that The I-frame sampling analysis strategy refers to a strategy of selecting M frames of images in the candidate video as the starting frames of the analysis segment, obtaining M analysis segments, and selecting one from the M analysis segments as the highlight segment, where M is a positive integer greater than 0.

9. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Displaying a first interface, wherein the first interface includes a first control, and the first control is used to generate the target video; In response to a first operation on the first control, displaying a second interface, where the second interface is used to display a plurality of videos and / or pictures; In response to a second operation on the second interface, the candidate video is determined, and the second operation is used to select the candidate video from the multiple videos and / or pictures.

10. A method for determining a strategy for generating a video, characterized in that: The method is applied in an electronic device, the electronic device comprising: a policy monitoring module, an algorithm module and a basic capability layer, the policy monitoring module is connected to the algorithm module and the basic capability layer respectively, and the method comprises: The policy monitoring module is used to obtain a first duration, a second duration and a video analysis rate, wherein the first duration refers to the video duration of the candidate video, and the second duration is used to indicate the analysis duration allocated to the candidate video for analysis processing; The strategy monitoring module is used to determine the target analysis strategy corresponding to the candidate video from the analysis strategy set based on the first duration, the second duration and the video analysis rate, wherein the analysis strategy set includes at least two analysis strategies, and different analysis strategies require different durations for analyzing and processing the candidate video; The algorithm module is used to analyze and process the candidate video based on the target analysis strategy to obtain the analysis result of the candidate video; The basic capability layer is used to obtain a target video according to the analysis results of the candidate videos; Among them, the strategy monitoring module is specifically used to determine, based on the first duration and the second duration, that when the percentage of frame images in the candidate video processed within the second duration is greater than a first preset threshold, the target analysis strategy is a full analysis strategy.

11. The method according to claim 10, characterized in that The strategy monitoring module is used to determine a first ratio based on the first duration, the second duration and the video analysis rate, wherein the first ratio is used to indicate the percentage of frame images in the candidate video that have completed analysis processing within the second duration to the total frame images in the candidate video; and determine the target analysis strategy based on the first ratio.

12. The method according to claim 10, characterized in that The policy monitoring module is used to determine a third duration according to the first duration and the video analysis rate, wherein the third duration is used to indicate the duration required to complete the analysis and processing of all frame images in the candidate video; A target analysis strategy corresponding to the candidate video is determined based on the third duration and the second duration.

13. An electronic device, characterized in that: include: one or more processors and memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, wherein the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the strategy determination method for generating a video as described in any one of claims 1 to 9.

14. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the strategy determination method for generating a video as described in any one of claims 1 to 9.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the method for determining a strategy for generating a video according to any one of claims 1 to 9.

Images