A video decoding method and electronic device

Abstract

The application provides a video decoding method and an electronic device, relates to the technical field of video data processing, and is used for improving the decoding efficiency of a to-be-analyzed material video when the electronic device uses a one-key large piece function to analyze the to-be-analyzed material video. The method comprises the following steps: after an electronic device acquires a first encoding format and a first media parameter (i.e. a video parameter) of a to-be-analyzed material video, the electronic device determines whether a first decoder in a working state and having the same decoding parameter as the video parameter of the to-be-analyzed material video exists. If the first decoder exists, the electronic device directly uses the first decoder to decode the to-be-analyzed material video. If the first decoder does not exist in the electronic device, when the electronic device determines that a second decoder in an idle state and having the same encoding format as the first encoding format exists, the electronic device sets the media parameter of the second decoder to the first media parameter, and then enables the second decoder to decode the to-be-analyzed material video.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of video data processing, and in particular, to a video decoding method and an electronic device. BACKGROUND

[0002] Currently, some electronic devices can provide a one-key film function. Specifically, in response to a user's operation of using the one-key film function after selecting a material video, the electronic device can automatically analyze and extract a highlight segment in the material video through an algorithm, and automatically combine the highlight segment into a finished video.

[0003] Among them, the electronic device needs to perform decoding processing on the material video before performing algorithmic automatic analysis on the material video. The existing video decoding method is low in efficiency and time-consuming, which leads to a longer time spent on generating the finished video. SUMMARY

[0004] The present application provides a video decoding method and an electronic device for improving the decoding efficiency of the material video to be analyzed when the electronic device uses the one-key film function to analyze the material video to be analyzed.

[0005] To achieve the above-mentioned purpose, embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a video decoding method is provided, which includes: first, an electronic device obtains a first encoding format and a first media parameter (i.e. video parameter) of a material video to be analyzed. Then, the electronic device queries whether there is a first decoder in the electronic device which is in a working state and has the same decoding parameter as the video parameter of the material video to be analyzed. Here, the decoding parameter of the first decoder is the same as the video parameter of the material video to be analyzed, which means that the second encoding format in the decoding parameter is the same as the first encoding format, and the second media parameter in the decoding parameter is the same as the first media parameter. Next, if the first decoder exists in the electronic device, the electronic device directly decodes the material video to be analyzed using the first decoder. Next, if the first decoder does not exist in the electronic device, the electronic device queries whether there is a second decoder in the electronic device which is in an idle state and has the same encoding format as the first encoding format. Next, if the second decoder exists in the electronic device, the electronic device sets the media parameter of the second decoder to the first media parameter and enables the second decoder to decode the material video to be analyzed.

[0007] In summary, after the electronic device obtains the first encoding parameter and the first media parameter of the video material to be analyzed, the electronic device queries whether there is a decoder that can be reused. For example, the decoder that can be reused refers to a first decoder in a working state, the encoding parameter of which is the same as the first encoding parameter, and the media parameter of which is the same as the first media parameter, or a second decoder in an idle state, the encoding parameter of which is the same as the first encoding parameter. For the first decoder, the electronic device can directly reuse the first decoder to decode the video material to be analyzed, without switching the first decoder between the working state and the idle state. Generally, in the process of switching the decoder from the idle state to the working state, a configuration operation needs to be performed. In this application, the first decoder does not need to be switched between the working state and the idle state, so the configuration operation does not need to be frequently performed, thereby improving the decoding efficiency. For the second decoder, the electronic device only needs to set the media parameter of the second decoder to the first media parameter (that is, complete the configuration operation), and then enable the second decoder to decode the video material to be analyzed, thereby saving the time for creating the decoder, and further improving the decoding efficiency.

[0008] In a possible implementation manner of the first aspect, if the second decoder does not exist in the electronic device, the electronic device creates a third decoder with the first encoding format, sets the media parameter of the third decoder to the first media parameter, and then enables the third decoder to decode the video material to be analyzed.

[0009] In another possible implementation manner of the first aspect, the creation of the third decoder with the first encoding format can include the following steps. First, the electronic device queries whether the current number of decoders in the decoder cache pool of the electronic device is equal to the maximum number of decoders in the decoder cache pool (that is, N, N is a positive integer greater than or equal to 2). Then, if the current number of decoders in the decoder cache pool is equal to the maximum number of decoders in the decoder cache pool, the electronic device needs to release the decoder with the longest idle time in the decoder cache pool, and then create the third decoder in the decoder cache pool. If the current number of decoders in the decoder cache pool is less than the maximum number of decoders in the decoder cache pool, the electronic device can directly create the third decoder.

[0010] In another possible implementation manner of the first aspect, after the electronic device obtains the video parameter of the video material to be analyzed, the video decoding method provided in this application can further include the following steps. First, the electronic device queries whether there is a fifth decoder in the electronic device, which is in a working state, and has the same encoding format as the first encoding format, but has a different media parameter from the first media parameter. Then, if the fifth decoder exists in the electronic device, the electronic device sets the media parameter of the fifth decoder to the first media parameter after the fifth decoder is in an idle state, and then enables the fifth decoder to decode the video material to be analyzed.

[0011] In this implementation, there can be multiple decoders in working state in the electronic device.

[0012] In a possible implementation of the first aspect, the electronic device can include multiple videos to be analyzed. Before the electronic device queries whether the first decoder exists in the electronic device, the video decoding method provided in the present application can further include: the electronic device divides the multiple videos to be analyzed in the same first encoding format into a group to obtain at least one video group.

[0013] Each video group can include at least one video subgroup, and the media parameters of the videos to be analyzed in each video subgroup are the same. The electronic device can decode the videos to be analyzed in each video subgroup in sequence. For each video subgroup: if the first decoder exists in the electronic device, the electronic device can directly use the first decoder to decode the videos to be analyzed in the video subgroup. Alternatively, if the second decoder exists, the electronic device sets the media parameters of the second decoder to the first media parameters and then enables the second decoder to decode the videos to be analyzed in the video subgroup. Alternatively, the electronic device creates the third decoder, sets the media parameters of the third decoder to the first media parameters, and then enables the third decoder to decode the videos to be analyzed. Alternatively, if the fifth decoder exists, the electronic device sets the media parameters of the fifth decoder to the first media parameters after the fifth decoder is in an idle state, and then enables the fifth decoder to decode the videos to be analyzed.

[0014] Based on this, for the videos to be analyzed in the same video subgroup, the electronic device can decode multiple videos to be analyzed in a video subgroup by using the same decoder in the above decoders. In this way, the decoder can work all the time (i.e., without frequent switching between working state and idle state) and does not need to repeat the configuration operation, which can further save time and improve the video decoding efficiency.

[0015] In a possible implementation of the first aspect, the electronic device can arrange the videos to be analyzed with the same first media parameters in each video group adjacently, so as to divide each video group into at least one video subgroup.

[0016] In a possible implementation manner of the first aspect, before the video parameter of the material video to be analyzed is acquired, the video decoding method provided by the embodiment of the present application further includes: the electronic device acquires the material video to be analyzed and a file descriptor of the material video to be analyzed. The file descriptor of the material video to be analyzed is used to uniquely identify the material video to be analyzed. The acquisition of the video parameter of the material video to be analyzed can include: first, the electronic device creates a media extractor corresponding to the file descriptor of the material video to be analyzed. Then, the electronic device acquires video data of the material video to be analyzed from the material video to be analyzed by using the media extractor, and extracts the video parameter of the material video to be analyzed from the video data.

[0017] In a possible implementation manner of the first aspect, the decoding of the material video to be analyzed by using the decoder (such as at least one of the first decoder, the second decoder, the third decoder, and the fifth decoder) in the electronic device includes: the electronic device decodes each of the first image frames in the first quantity of the first image frames in the material video to be analyzed by using the decoder.

[0018] The first quantity of the first image frames can be uniformly distributed at various positions of the material video to be analyzed, or the first quantity of the first image frames are I frames in the material video to be analyzed.

[0019] In a possible implementation manner of the first aspect, after the decoding of each of the first image frames in the first quantity of the first image frames in the material video to be analyzed by using the decoder in the electronic device, the video decoding method provided by the embodiment of the present application further includes: first, the electronic device performs image analysis on each of the decoded first image frames to acquire an image score of each of the corresponding first image frames. Then, the electronic device determines a region in which a plurality of first image frames that are continuous within a second preset time length before and after a first target image frame are located as a target region of the material video to be analyzed. At this time, the first target image frame is a first image frame with the highest image score in all the first image frames in the material video to be analyzed. Then, the electronic device acquires a second quantity of second image frames from the target region. Finally, the electronic device decodes each of the second image frames in the second quantity of the second image frames in the target region by using the decoder in the electronic device. The second quantity of the image frames are uniformly distributed at various positions of the target region of the material video to be analyzed, or the second quantity of the image frames are I frames in the target region of the material video to be analyzed.

[0020] In a second aspect, an electronic device is provided. The electronic device includes a memory and one or more processors. The memory is coupled to the processors. The memory stores computer program code including instructions. When the instructions are executed by the processors, the electronic device performs the method of any one of the above first aspect.

[0021] In a third aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores instructions which, when executed on an electronic device, cause the electronic device to perform the method of any one of the first aspect.

[0022] In a fourth aspect, a computer program product is provided, and the computer program product stores instructions which, when executed on an electronic device, cause the electronic device to perform the method of any one of the first aspect.

[0023] In a fifth aspect, an embodiment of the present application provides a chip, and the chip includes a processor configured to invoke a computer program in a memory to perform the method of the first aspect.

[0024] It can be understood that the electronic device of the second aspect, the computer-readable storage medium of the third aspect, the computer program product of the fourth aspect, and the chip of the fifth aspect can achieve the beneficial effects of the first aspect and any possible design of the first aspect, which will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 An interface schematic diagram is provided for an embodiment of the present application.

[0026] Figure 2 A hardware structure schematic diagram of an electronic device is provided for an embodiment of the present application.

[0027] Figure 3 A software architecture schematic diagram of an electronic device is provided for an embodiment of the present application.

[0028] Figure 4 One of flow schematic diagrams of a video decoding method is provided for an embodiment of the present application.

[0029] Figure 5 Another of flow schematic diagrams of a video decoding method is provided for an embodiment of the present application.

[0030] Figure 6 Still another of flow schematic diagrams of a video decoding method is provided for an embodiment of the present application.

[0031] Figure 7 Still another of flow schematic diagrams of a video decoding method is provided for an embodiment of the present application.

[0032] Figure 8 Still another of flow schematic diagrams of a video decoding method is provided for an embodiment of the present application.

[0033] Figure 9 Still another of flow schematic diagrams of a video decoding method is provided for an embodiment of the present application.

[0034] Figure 10 FIG. 7 is a flowchart of a video decoding method according to an embodiment of the present application;

[0035] Figure 11 FIG. 8 is a flowchart of a video decoding method according to an embodiment of the present application;

[0036] Figure 12 FIG. 9 is a flowchart of a video decoding method according to an embodiment of the present application;

[0037] Figure 13 FIG. 10 is a schematic structural diagram of a chip system according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] The terms “first”, “second”, and the like in embodiments of the present application are only used for the purpose of distinguishing the same type of features, and should not be understood as indicating relative importance, quantity, order, and the like.

[0039] The terms “exemplary” or “for example” in embodiments of the present application are used to represent that an embodiment or design scheme is an example, illustration or description. Any embodiment or design scheme described as “exemplary” or “for example” in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the terms “exemplary” or “for example” are used in the sense of presenting a related concept in a specific manner.

[0040] The terms “coupled” and “connected” in embodiments of the present application should be interpreted in a broad sense, for example, can refer to a direct physical connection or an indirect connection through electronic devices, such as a connection through resistors, inductors, capacitors or other electronic devices.

[0041] The electronic device can provide a one-key video function. For example, an image processing application (such as a gallery) is installed on the electronic device, and the image processing application can support processing operations for editing pictures and videos and the like. The electronic device can decode the to-be-analyzed material video through the image processing application, automatically analyze the to-be-analyzed material video through an algorithm, and thereby extract a highlight segment in the to-be-analyzed material video. Finally, the electronic device splices all the highlight segments extracted from the to-be-analyzed material video into a finished video through the image processing application, and displays the finished video on a display interface.

[0042] The video decoding method in the conventional technology is low in efficiency and long in time, which leads to a long time spent in generating a finished video based on a to-be-analyzed material video and a poor user experience.

[0043] To this end, an embodiment of the present application provides a video decoding method. After an electronic device obtains a first encoding parameter and a first media parameter of a material video to be analyzed, the electronic device preferentially queries whether a decoder that can be reused exists. For example, the decoder that can be reused refers to a first decoder that is in a working state, has the same encoding parameter as the first encoding parameter, and has the same media parameter as the first media parameter, or a second decoder that is in an idle state and has the same encoding parameter as the first encoding parameter. For the first decoder, the electronic device can directly reuse the first decoder to decode the material video to be analyzed. For the second decoder, the electronic device needs to set the media parameter of the second decoder to the first media parameter (that is, complete a configuration operation), and then enable the second decoder to decode the material video to be analyzed.

[0044] Based on this, the reused decoder can avoid frequent repeated creation, configuration, and release of the decoder, thereby improving the video decoding efficiency.

[0045] The electronic device related to the embodiments of the present application can also be referred to as a terminal, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), and the like. The electronic device can be a mobile phone, a smart television, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, or a wireless terminal in smart home, and the like. The embodiments of the present application do not limit the specific technology and specific device form of the electronic device.

[0046] Figure 1 An interface schematic diagram provided by an embodiment of the present application is shown. Hereinafter, the electronic device is taken as a mobile phone as an example, and the application of the video decoding method provided by the embodiments of the present application is described in combination with Figure 1 The application scenario of the video decoding method provided by the embodiments of the present application is exemplarily described.

[0047] In an embodiment, as shown in A in Figure 1 The desktop 110 of the mobile phone displays an application icon 111 of a gallery. In response to a click operation on the application icon 111 of the gallery, the mobile phone displays a gallery interface 120 as shown in B inFigure 1 As shown in B of FIG. 1C, the mobile phone displays a first gallery interface 120. The first gallery interface 120 includes a "one-click big picture" control 121. In response to a click operation on the "one-click big picture" control 121, the mobile phone displays a second gallery interface 130, as shown in C of FIG. 1C. The second gallery interface 130 can include multiple videos and multiple pictures. In response to a click operation on any video in the second gallery interface 130, the mobile phone displays a third gallery interface 140, as shown in D of FIG. 1C. The third gallery interface 140 includes all videos and all pictures. In response to a selection operation on at least one video in the third gallery interface 140, the mobile phone displays a first pop-up window 141 in the third gallery interface 140, as shown in D of FIG. 1C. The first pop-up window 141 includes the selected videos and a video generation control 142, for example, a one-click big picture control. It should be understood that the one-click big picture control in the first pop-up window 141 is different from the "one-click big picture" control 121 in the first gallery interface 120. In response to a click operation on the video generation control 142, the mobile phone analyzes the selected videos, filters out highlight clips from the selected videos, and generates a finished video according to the highlight clips. In this process, the mobile phone can display a second pop-up window 143 in the third gallery interface 140, as shown in E of FIG. 1C. The second pop-up window 143 includes an analysis progress, so that the user can intuitively view the analysis progress. When the mobile phone completes the analysis of the selected videos, the mobile phone displays a fourth gallery interface 150, as shown in F of FIG. 1C. The fourth gallery interface 150 displays the finished video. Figure 1 Figure 1 Figure 1 Figure 1 Figure 1

[0048] In another embodiment, the mobile phone displays a second gallery interface 130, as shown in C of FIG. 1C. The second gallery interface 130 includes multiple videos and multiple pictures. In response to a click operation on any video in the second gallery interface 130, the mobile phone displays a third gallery interface 140, as shown in D of FIG. 1C. The third gallery interface 140 includes all videos and all pictures. In response to a selection operation on at least one video in the third gallery interface 140, the mobile phone displays a first pop-up window 141 in the third gallery interface 140, as shown in D of FIG. 1C. The first pop-up window 141 includes the selected videos and a video generation control 142, for example, a one-click big picture control. It should be understood that the one-click big picture control in the first pop-up window 141 is different from the "one-click big picture" control 121 in the first gallery interface 120. In response to a click operation on the video generation control 142, the mobile phone analyzes the selected videos, filters out highlight clips from the selected videos, and generates a finished video according to the highlight clips. In this process, the mobile phone can display a second pop-up window 143 in the third gallery interface 140, as shown in E of FIG. 1C. The second pop-up window 143 includes an analysis progress, so that the user can intuitively view the analysis progress. When the mobile phone completes the analysis of the selected videos, the mobile phone displays a fourth gallery interface 150, as shown in F of FIG. 1C. The fourth gallery interface 150 displays the finished video. Figure 1 Figure 1

[0049]

[0050] Figure 2 FIG. 1A shows a hardware structure of an electronic device according to an embodiment of the present application.​​​​​​​​Figure 2 The hardware structure of the electronic device is introduced.

[0051] Taking the electronic device as a mobile phone for example, as shown in the figure, the electronic device 200 can include a processor 210, a memory 220, a universal serial bus (USB) interface 230, a power management module 240, an antenna, a communication module 250, a display screen 260, an audio module 270, a camera 280, a sensor module 290, etc. Figure 2

[0052] The processor 210 can include one or more processing units, for example: the processor 210 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors. The controller can be the nerve center and command center of the electronic device 200. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching instructions and executing instructions.

[0053] The memory 220 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 210 executes various functions of the electronic device and data processing by running the instructions stored in the memory 220. The memory 220 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an interface display function, etc.). The data storage area can store data created during the use of the electronic device (such as notification messages) and the like. In addition, the memory 220 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0054] ​The power management module 240 is configured to connect the battery to the processor 210. The power management module 240 receives power from the battery and / or a power source to supply power to the processor 210, the memory 220, the communication module 250, the display 260, the audio module 270, the camera 280, and the like. The power management module 240 can also be configured to monitor parameters such as the battery capacity, the number of battery cycles, the state of health of the battery (leakage, impedance), and the like. In some embodiments, the power management module 240 can also be disposed in the processor 210.

[0055] The communication module 250 can provide wireless communication solutions including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), and the like. The communication module 250 can be one or more devices that integrate at least one communication processing module. The communication module 250 receives electromagnetic waves via an antenna, frequency modulates and filters the electromagnetic wave signals, and transmits the processed signals to the processor 210. The communication module 250 can also receive signals to be transmitted from the processor 210, frequency modulate them, amplify them, and radiate them as electromagnetic waves via the antenna.

[0056] In some embodiments, the antenna and the communication module 250 of the electronic device 200 are coupled such that the electronic device 200 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, global navigation satellite system (GNSS), WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include global positioning system (GPS), beidou navigation satellite system (BDS), global navigation satellite system (GLONASS), and / or Galileo satellite navigation system (GALILEO).

[0057] The electronic device 200 implements a display function through a GPU, a display 260, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display 260 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 can include one or more GPUs, which execute program instructions to generate or change display information.

[0058] The display screen 260 is configured to display images, videos (e.g., a video clip), and the like. The display screen 260 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini-LED, a Micro-OLED, a quantum dot light emitting diode (QLED), or the like. In some embodiments, the display screen 260 can be configured to display a video clip.

[0059] The electronic device 200 can implement a photographing function through an ISP, the camera 280, a video codec, a GPU, the display screen 260, and an application processor.

[0060] The audio module 270 is configured to convert digital audio information into an analog audio signal for output and to convert an analog audio input into a digital audio signal. The audio module 270 can also be configured to encode and decode audio signals. In some embodiments, the audio module 270 can be disposed in the processor 210 or some of the functions of the audio module 270 can be disposed in the processor 210.

[0061] The camera 280 is configured to capture still images or videos. An object generates an optical image through a lens and projects the optical image onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal and then transmits the electrical signal to an ISP to convert the electrical signal into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into a standard image signal in an RGB, YUV, or the like format.

[0062] The sensor module 290 can include a pressure sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, and a bone conduction sensor, among others.

[0063] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 200. In other embodiments, the electronic device 200 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0064] Generally speaking, the realization of the one-click blockbuster function in electronic devices requires not only hardware support but also software cooperation. The software system of the electronic device can adopt a layered architecture, event-driven architecture, micro-core architecture, micro-service architecture, or cloud architecture. For example, combined with Figure 3 The software architecture of the electronic device involved in the embodiments of the present application is introduced.

[0065] Figure 3 A schematic diagram of the software architecture of an electronic device provided in an embodiment of the present application is shown.

[0066] like Figure 3 As shown, the layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android operating system It is divided into four layers, from top to bottom: application (APP) layer, media middle platform framework layer, application framework (FWK) layer and hardware abstraction layer (HAL).

[0067] The application layer can include a series of application packages, such as camera, gallery, third-party video editing software, calendar, map, and navigation. When these application packages are running, they can access the various service modules provided by the media platform framework layer and application framework layer through the application programming interface (API) and execute corresponding intelligent services.

[0068] In one embodiment, the camera is used to capture photos, videos, slow-motion images, and panoramic images in response to user operations. After these images are captured by the camera, or after the user triggers the phone to take a screenshot, or after the user triggers the phone to record the screen, or after the terminal device downloads images from other devices, the terminal device can save these images in a gallery, so that the user can perform video editing operations on the images in the gallery, such as a one-click blockbuster operation.

[0069] The embodiment of the present application divides the gallery into the following layers from top to bottom: business layer, application function layer and basic function layer.

[0070] Among them, the service layer is also called a video editing service layer. The service layer includes multiple modules such as multi-camera video automatic film making, one-way multi-AI music short film, one-key blockbuster, and highlight moment, which are respectively used to provide corresponding services. These services are presented in the form of controls in the user interface (UI) of the gallery, also known as the gallery interface. The user can trigger the camera to perform the corresponding video processing action by operating a certain control. For example, after the user selects one or more materials (the materials include material videos and / or material pictures) and clicks the one-key blockbuster control of the gallery, the electronic device decodes the to-be-analyzed materials, automatically analyzes and extracts the highlight segments in the material videos and the highlight pictures in the material pictures through an algorithm, and then combines all the highlight segments and all the highlight pictures into a finished film video.

[0071] Among them, the application function layer includes an automatic editing framework. Each service in the service layer can call the automatic editing framework to provide automatic editing services for the materials. Exemplarily, the automatic editing framework can include functional modules such as segment optimization, story line organization, layout splicing, and special effect beautification. The segment optimization is used to call a channel interface to perform decoding, resolution reduction, format conversion, and the like on the to-be-analyzed materials, and store the processed to-be-analyzed materials. The segment optimization is also used to call an image highlight segment analysis interface, a strategy monitoring module, and the like to extract highlight segments and / or highlight pictures from the to-be-analyzed materials. The story line organization is used to sequentially splice multiple to-be-analyzed materials in the form of a story line based on the content of the to-be-analyzed materials. The layout splicing is used to adjust the interface layout of the to-be-analyzed materials. The special effect beautification is used to adjust the beautification effect of the to-be-analyzed materials, such as adjusting the picture brightness and beautifying the faces of characters.

[0072] Among them, the basic function layer is used to perform basic function processing on the edited video segments after the automatic editing framework edits the multiple to-be-analyzed materials. Exemplarily, the basic function layer can include basic function modules such as video splicing, synthesis saving, video speed changing, and audio / video effect processing. Among them, the video splicing is used to splice all the extracted highlight segments and / or highlight pictures to obtain a finished film video. The synthesis saving is used to store the finished film video obtained after splicing. The video speed changing is used to perform speed changing processing on the finished film video. For example, the video speed changing is used to add a slow playback effect to the part including the highlight action in the spliced finished film video. The audio / video effect processing module is used to add video effects and sound effects to the finished film video obtained after splicing. For example, style filters and themes are added to the finished film video, and background music is added.

[0073] The media platform framework layer is a software layer set up between the application layer and the application framework. The media platform framework layer can include an analysis performance query interface, a policy monitoring module, a channel (pipeline) interface, a topic summary interface, an initialization interface, an image highlight segment analysis interface, and a decoder buffer pool.

[0074] The analysis performance query interface is used to obtain the estimated analysis duration of each video to be analyzed.

[0075] Among them, the strategy monitoring module is used to obtain the material to be analyzed from all materials, determine the image frames that need to be analyzed in each material video to be analyzed (such as the first image frame, second image frame, etc. mentioned below), determine the target area of ​​each material video to be analyzed, determine the highlight clips of each material video to be analyzed, etc.

[0076] Among them, the channel interface is used to decode, convert the format, reduce the resolution, etc. of the material to be analyzed, and forward the frame data address of the processed material to be analyzed to the hardware abstraction layer through the application framework layer, and then report the analysis results of the analyzed image frame returned by the hardware abstraction layer to the application function layer.

[0077] The channel interface is also used to reduce the frame rate of the decoded video material to be analyzed, retaining some image frames in each video material to be analyzed. This can reduce the time required by the electronic equipment to perform operations such as format conversion and resolution reduction, thereby improving efficiency.

[0078] The theme summary interface is used to call the theme algorithm, obtain the theme type that matches all highlight clips and / or highlight images, and send the theme type to the application function layer. The application function layer obtains the theme template corresponding to the theme type.

[0079] The initialization interface is used to initialize the related algorithms of the hardware abstraction layer.

[0080] The decoder buffer pool is used to create a decoder and record decoder status, decoding parameters, etc.

[0081] It should be noted that this application is described using the one-click blockbuster function provided by the gallery as an example, which does not limit the embodiments of this application. In actual implementation, third-party video editing software can use the video clip processing method provided in the embodiments of this application to combine multiple pictures and videos selected by the user into a complete video with one click.

[0082] The application framework layer, also referred to as the framework layer, supports the operation of various modules in the media platform framework layer. For example, the framework layer may include service interfaces such as the one-click blockbuster interface, parameter management interface, image data transmission interface, theme analysis interface, and performance analysis interface.

[0083] The hardware abstraction layer can include a capability query interface and algorithm modules. The capability query interface is used to obtain algorithm capabilities supported by the hardware abstraction layer. The algorithm modules can include a highlight fragment algorithm, a face detection algorithm, a video acceleration algorithm, and the like.

[0084] The highlight fragment algorithm is used to perform image analysis on image frames in a video to be analyzed to determine whether the image frames belong to a highlight fragment. The highlight fragment algorithm is also used to perform image analysis on pictures in a picture to be analyzed to determine whether the pictures belong to a highlight picture.

[0085] The face detection algorithm is used to perform face detection on the image frames (or pictures).

[0086] The video acceleration algorithm is used to perform acceleration processing on a video (such as the first video).

[0087] It should be noted that, Figure 3 The layers in the software structure and the components included in each layer do not constitute a specific limitation on the terminal device. In other embodiments, the terminal device can include more layers than those shown, such as a system library (FWK LIB) layer and a kernel layer. Each layer can include more or fewer components than those shown. In addition, the various functional modules described above can be combined into one functional module, and the various layers can be combined into one layer, such as the media middleware framework layer, which can be disposed in the application framework layer.

[0088] It can be understood that the electronic device includes corresponding hardware and / or software modules for performing each function in order to implement the method in the embodiments of the present application. The algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or computer software that drives hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments.

[0089] Figure 4 A flowchart of one of the video decoding methods provided by the embodiments of the present application is shown. As Figure 4 shown, the method can include:

[0090] S401, the electronic device displays at least one candidate video and candidate picture.

[0091] For example, as Figure 1 shown in C of FIG. 13, the mobile phone displays a second gallery interface 130. The second gallery interface 130 includes at least one video and at least one picture. The video can be referred to as a candidate video, and the picture can be referred to as a candidate picture.

[0092] The candidate video can be obtained by photographing of the electronic device, screen recording, or received from another electronic device. The candidate picture can be obtained by photographing of the electronic device, screen capturing, or received from another electronic device. The source of the candidate video and the candidate picture is not limited in the present application.

[0093] S402, in response to the selection operation on the candidate video, the electronic device obtains a material video from the candidate video, and in response to the selection operation on the candidate picture, the electronic device obtains a material picture from the candidate picture.

[0094] The material video is the selected candidate video. In another embodiment, the electronic device can obtain the material picture from the candidate picture in response to the selection operation on the candidate picture. The material picture is the selected candidate picture.

[0095] In some embodiments, the electronic device can obtain the material video from the candidate video only in response to the selection operation on the candidate video. Alternatively, the electronic device can obtain the material picture from the candidate picture only in response to the selection operation on the candidate picture.

[0096] It should be noted that when the electronic device obtains a material video, the file descriptor (FD) of the material video is also obtained. The file descriptor of a material video can uniquely identify the material video. That is, the file descriptor and the material video are one-to-one correspondence. Similarly, when the electronic device obtains a material picture, the file descriptor of the material picture is also obtained.

[0097] S403, the electronic device obtains the material video to be analyzed and the material picture to be analyzed from all materials according to the analysis total duration and the analysis total duration upper limit suggestion value.

[0098] The analysis total duration can represent the sum of the durations required to analyze all the materials selected by the user. The analysis total duration upper limit suggestion value represents the suggestion value of the maximum duration to analyze all the materials. The analysis total duration upper limit suggestion value can be calculated according to the estimated analysis duration of each material. Generally, the analysis total duration upper limit suggestion value is greater than the analysis total duration.

[0099] In the embodiment of the present application, the estimated analysis duration can be determined according to the following manner:

[0100] In one embodiment, the estimated analysis duration of a material video can be the duration of analyzing one image frame in the material video, that is, the minimum analysis duration of the material video. For example, the duration of analyzing one image frame by the algorithm module is 200ms, and the estimated analysis duration of each material video is 200ms.

[0101] In another embodiment, the estimated analysis duration of a material video can be the duration of analyzing all image frames in the material video.

[0102] In another embodiment, the estimated analysis duration of a material video can be the duration of analyzing M image frames in the material video. Wherein M is a positive integer greater than 1, and M is less than the number of all image frames in the material video.

[0103] In another embodiment, the estimated analysis duration of a material picture is the duration of analyzing one image frame.

[0104] In the embodiments of the present application, the total analysis duration can be determined according to the following manner:

[0105] In one embodiment, the material only includes material pictures. At this time, the processing duration of processing one material picture can be directly determined according to the algorithm capability. The product of the number of material pictures and the processing duration of processing one material picture (i.e. the estimated analysis duration) is used to determine the sum of the estimated analysis durations of all material pictures. At this time, the total analysis duration is the sum of the estimated analysis durations of all material pictures.

[0106] In another embodiment, the material only includes material videos. At this time, the sum of the estimated analysis durations of all material videos can be accumulated to obtain the sum of the estimated analysis durations of all material videos. At this time, the total analysis duration is the sum of the estimated analysis durations of all material videos.

[0107] In another embodiment, the material includes material videos and material pictures. The sum of the estimated analysis durations of all material videos and the sum of the estimated analysis durations of all material pictures are accumulated to obtain the total analysis duration of all materials.

[0108] Wherein, the material to be analyzed can be all materials, or part of the materials. Taking the material including material pictures and material videos as an example: the material to be analyzed can include all material videos and all material pictures; or, the material to be analyzed can include part of the material pictures and part of the material videos. For example, taking the material including material pictures as an example: the material to be analyzed can include all material pictures, or part of the material pictures. For example, taking the material including material videos as an example: the material to be analyzed can include all material videos, or part of the material videos.

[0109] Hereinafter, taking the material including material pictures and material videos as an example, the material to be analyzed is introduced:

[0110] In one embodiment, if the total analysis duration is less than or equal to the upper limit of the total analysis duration, the material to be analyzed can be all material pictures and all material videos.

[0111] In another embodiment, if the analysis total time length is greater than the analysis total time length upper limit suggestion value, the to-be-analyzed material can be a part of material pictures randomly selected from all material pictures based on a random selection strategy, and a part of material videos randomly selected from all material videos. The random selection strategy is related to the importance of the material pictures and the material videos.

[0112] For example, the importance of the material pictures is prior to the importance of the material videos. At this time, the random selection strategy can be: selecting P material pictures and then selecting Q material videos, until the analysis total time length of the selected materials reaches the analysis time length upper limit suggestion value, or the material pictures have been selected completely. Q < P, and P and Q are positive integers.

[0113] For example, the importance of the material videos is prior to the importance of the material pictures. At this time, the random selection strategy can be: selecting A material videos and then selecting B material pictures, until the analysis total time length of the selected materials reaches the analysis time length upper limit suggestion value, or the material videos have been selected completely. B < A, and A and B are positive integers.

[0114] It should be noted that the to-be-analyzed material is further processed below, and the non-to-be-analyzed material is not further processed. That is, the highlight segment is derived from the to-be-analyzed material.

[0115] S404, the electronic device performs image analysis on each to-be-analyzed material picture, and selects a highlight picture from all to-be-analyzed material pictures.

[0116] Specifically, the electronic device sequentially performs decoding, format conversion, resolution reduction, image analysis and other processing operations on each to-be-analyzed material picture. For details, refer to the detailed description below, and the embodiments of the present application will not be repeated here.

[0117] S405, the electronic device obtains the video parameters of the to-be-analyzed material video.

[0118] The video parameters of the to-be-analyzed material video can include the first encoding format and the first media parameter of the to-be-analyzed material video.

[0119] Specifically, the encoding format refers to the format used to store or transmit the video. For example, the first encoding format can include H26X series, but is not limited thereto. X = 4, 5, and the like.

[0120] Specifically, the media parameters refer to basic parameters of the video. For example, the first media parameters can include resolution, frame rate, rotation angle, etc. The resolution is used to indicate the number of pixels in the horizontal direction and the vertical direction of the video. The higher the resolution, the more details and the higher definition of the video. The frame rate is used to indicate the number of image frames transmitted per second. The higher the frame rate, the smoother the video. The rotation angle is used to indicate whether the angle of the video is a horizontal screen angle or a vertical screen angle. Generally, the horizontal screen angle and the vertical screen angle differ by 90°.

[0121] In an embodiment, the electronic device acquires the video parameters of the to-be-analyzed material video, which can include the following steps. First, the electronic device creates a media extractor corresponding to the file descriptor of the to-be-analyzed material video. Then, the electronic device separates the video data and the audio data of the to-be-analyzed material video by using the media extractor. Finally, the electronic device extracts the video parameters of the to-be-analyzed material video from the video data of the to-be-analyzed material video by using the media extractor.

[0122] Since the media extractor is one-to-one corresponding to the file descriptor, and the file descriptor is one-to-one corresponding to the to-be-analyzed material video, one media extractor can only extract the video parameters of one corresponding to-be-analyzed material video. Based on this, when the electronic device acquires the video parameters of one corresponding to-be-analyzed material video by using one media extractor, the electronic device releases the media extractor.

[0123] S406, the electronic device queries whether there is a first decoder in a working state and the decoding parameters are the same as the video parameters of the to-be-analyzed material video.

[0124] The decoding parameters being the same as the video parameters of the to-be-analyzed material video can include that the second encoding format in the decoding parameters is the same as the first encoding format, and the second media parameters in the decoding parameters are the same as the first media parameters. As introduced above, the second encoding format can include H26X series, but is not limited to this. As introduced above, the second media parameters can include resolution, frame rate, rotation angle, etc.

[0125] S407, if the first decoder exists, the electronic device decodes the to-be-analyzed material video by using the first decoder based on the target analysis strategy.

[0126] In an embodiment, when the electronic device decodes the to-be-analyzed material video segment using the first decoder, the electronic device has completed a current decoding task by the first decoder. Then, the electronic device can directly decode the to-be-analyzed material video by the first decoder. In this process, the electronic device does not need to idle the first decoder (i.e., stop the first decoder) and then start the first decoder to decode the to-be-analyzed material video. In this way, the electronic device can not only ensure that the decoding operation of each to-be-decoded video is completed, but also does not need to frequently switch the first decoder between the working state and the idle state. Generally, the process of switching the decoder from the idle state to the working state needs to perform a configuration operation. In the embodiment, the first decoder does not need to be switched between the working state and the idle state, and thus the configuration operation does not need to be frequently performed. Based on this, the video decoding efficiency can be improved.

[0127] The configuration operation refers to that the electronic device sets the media parameters of the to-be-decoded video as the media parameters of the decoder. For example, the resolution of the to-be-decoded video is set as the resolution of the decoder, the frame rate of the to-be-decoded video is set as the frame rate of the decoder, and the rotation angle of the to-be-decoded video is set as the rotation angle of the decoder. The to-be-decoded video is the to-be-analyzed material video that needs to be decoded.

[0128] In an embodiment, the target analysis strategy can be an overview analysis-focal region fine analysis strategy. Specifically, the overview analysis refers to performing image analysis on a first number of image frames in a to-be-analyzed material video to obtain a target region of the to-be-analyzed material video. The focal region fine analysis refers to performing image analysis on a second number of image frames in the target region of the to-be-analyzed material video to obtain a highlight segment from the target region.

[0129] The first number can be greater than the second number, or the first number can be equal to the second number, or the first number can be equal to the second number, which is not limited in the embodiments of the present application.

[0130] The first number of image frames can be uniformly distributed at various positions of the to-be-analyzed material video, or the first number of image frames can be arbitrarily distributed at various positions of the to-be-analyzed material video, or the first number of image frames are I frames in the to-be-analyzed material video. Similarly, the second number of image frames can be uniformly distributed at various positions of the target region of the to-be-analyzed material video, or the second number of image frames can be arbitrarily distributed at various positions of the to-be-analyzed material video, or the second number of image frames are I frames in the to-be-analyzed material video.

[0131] As known from the above, the electronic device needs to decode the same to-be-analyzed material video twice. The first decoding is in the overview analysis stage. At this time, the electronic device can decode the first quantity of image frames in the to-be-analyzed material video. The second decoding is in the key region fine analysis stage. At this time, the electronic device can decode the second quantity of image frames in the target region of the to-be-analyzed material video.

[0132] For the convenience of description, each image frame in the first quantity of image frames is referred to as a first image frame, and each image frame in the second quantity of image frames is referred to as a second image frame in the present document.

[0133] It should be noted that the overview analysis process and the key region fine analysis process will be described in detail below, and the embodiments of the present application will not be described here. That is, the acquisition process of the first quantity of first image frames in each to-be-analyzed material video and the determination process of the target region in each to-be-analyzed material video will be described in detail below, and will not be described here.

[0134] In the present embodiment, at the same time, the electronic device can simultaneously decode the corresponding plurality of to-be-decoded videos through the plurality of decoders respectively. That is, at the same time, there can be a plurality of decoders in the electronic device in a working state. Alternatively, at the same time, the electronic device can decode one to-be-decoded video through one decoder. That is, at the same time, only one decoder in the electronic device is in a working state.

[0135] S408, if the first decoder does not exist, the electronic device queries whether a second decoder in an idle state and having the same encoding format as the first encoding format exists.

[0136] In the embodiments of the present application, a decoder cache pool is provided in the electronic device. The decoder cache pool can be used to create decoders, record the states of the decoders, and the decoding parameters of the decoders, etc. When the electronic device does not query the first decoder, the electronic device will query whether the second decoder exists in the decoder cache pool.

[0137] S409, if the second decoder exists, the electronic device sets the media parameters of the second decoder to the first media parameters, and then enables the second decoder to decode the to-be-analyzed material video based on the target analysis strategy.

[0138] When the electronic device needs to enable a decoder in an idle state, the electronic device needs to perform a configuration operation on the decoder in the idle state, and then enable the configured decoder to decode the to-be-decoded video. Therefore, in the present embodiment, if the electronic device queries the second decoder, before using the second decoder to decode the to-be-analyzed material video, the electronic device needs to set the media parameters of the second decoder to the first media parameters of the to-be-analyzed material video, and then enable the second decoder.

[0139] S410, if the second decoder does not exist, the electronic device creates a third decoder whose encoding format is the first encoding format, sets the media parameters of the third decoder to the first media parameters, and enables the third decoder to decode the video to be analyzed based on the target analysis strategy.

[0140] When the electronic device does not query the second decoder in the decoder cache pool, it indicates that the second decoder does not exist in the electronic device. At this time, the electronic device needs to create a decoder with the same encoding format as the video to be decoded. Then, the electronic device configures the newly created decoder using the media parameters of the video to be decoded, and then enables the configured decoder to decode the video to be decoded.

[0141] Optionally, the decoder cache pool can store up to N decoders, where N is a positive integer greater than or equal to 2. That is, N is the maximum number of decoders in the decoder cache pool. When the electronic device creates a new decoder in the decoder cache pool, the electronic device needs to determine whether the current number of decoders in the decoder cache pool is equal to the maximum number.

[0142] In an embodiment, if the electronic device determines that the current number of decoders in the decoder cache pool is equal to the maximum number, it indicates that the decoder cache pool cannot continue to store new decoders. At this time, the electronic device needs to release the fourth decoder in the decoder cache pool and create a third decoder in the decoder cache pool. If the electronic device determines that the current number of decoders in the decoder cache pool is less than the maximum number, it indicates that the decoder cache pool can continue to store new decoders. At this time, the electronic device directly creates a third decoder in the decoder cache pool.

[0143] Among them, the fourth decoder performs decoding operation for the longest time between the current time and the time. That is, the fourth decoder is idle in the decoder cache pool for the longest time. At this time, in order to release the cache space of the decoder cache pool, the fourth decoder can be released to store the newly created third decoder.

[0144] In an embodiment, after the electronic device decodes the video to be analyzed, it also performs frame reduction processing to retain part of the image frames in each video to be analyzed. For example, the video to be analyzed 1 second includes 60 frames. After frame reduction, the video to be analyzed 1 second includes 30 frames. In this way, the time for the electronic device to perform format conversion, reduce resolution, etc. can be reduced, and efficiency can be improved.

[0145] S411, the electronic device performs image analysis on the decoded video to be analyzed to obtain all highlight segments.

[0146] Specifically, as described above in S407, the electronic device decodes the video twice using the decoder. The following describes the two cases:

[0147] In an embodiment, after decoding the video for the first time, the electronic device obtains frame data addresses of each first image frame in the video. Then, the electronic device obtains each first image frame from the video based on the frame data addresses of each first image frame, and performs image analysis on each first image frame to obtain an analysis result of each first image frame. The analysis result of each first image frame includes an image score of the first image frame. Then, the electronic device finds a first target image frame with the highest image score from each first image frame. Finally, the electronic device determines a region of a plurality of first image frames within a first preset time period before and after the first target image frame as the target region of the video.

[0148] The first preset time period is related to the actual time length of the video, the highlight segment suggestion time length, and the first number.

[0149] In an embodiment, after decoding the video for the second time, the electronic device obtains frame data addresses of each second image frame in the target region of the video. Then, the electronic device obtains each second image frame from the video based on the frame data addresses of each second image frame, and performs image analysis on each second image frame to obtain an analysis result of each second image frame. The analysis result of each second image frame includes an image score of the second image frame. Then, the electronic device finds a second target image frame with the highest image score from each second image frame. Finally, the electronic device determines a region of a plurality of second image frames within a second preset time period before and after the second target image frame as the highlight segment of the video.

[0150] As can be seen, there is no strict order between the steps in the embodiments of the present application. For example, part of the process in S411 can be interleaved with part of the steps in S407-S410.

[0151] S412, the electronic device obtains a first video from all the highlight segments and all the highlight pictures.

[0152] In an embodiment, after obtaining a plurality of highlight segments, the electronic device can directly splice the plurality of highlight segments to obtain a first video.

[0153] In another embodiment, after obtaining all highlight clips and all highlight images, the electronic device first obtains a theme template corresponding to the content of all highlight clips and all highlight images. The electronic device then directly applies the theme template to all highlight clips and all highlight images after splicing them together, thereby obtaining a first finished video. The theme template may be obtained by the electronic device applying a theme algorithm to analyze all highlight clips and all highlight images.

[0154] In another embodiment, after obtaining all highlight clips and all highlight images, as well as the corresponding theme templates, the electronic device concatenates all highlight clips and all highlight images to produce a second complete video. The electronic device then performs video re-editing on the second complete video, such as speed change and adding special effects, to produce a third complete video. Finally, the electronic device applies the theme template to the third complete video to produce the first complete video.

[0155] S413: The electronic device displays the first completed video.

[0156] For example, Figure 1 As shown in F in FIG, the mobile phone displays the fourth gallery interface 150. The fourth gallery interface 150 includes the finished videos.

[0157] Optionally, after S405 and before S406 , the electronic device may group the multiple materials to be analyzed so that the multiple material videos to be analyzed with the same video parameters are in one video group.

[0158] In one embodiment, the electronic device may group multiple to-be-analyzed material videos having the same first encoding format into a group, thereby obtaining at least one video group; wherein each video group includes at least one video subgroup, and the to-be-analyzed material videos in each video subgroup have the same media parameters. For example, the electronic device may arrange the to-be-analyzed material videos having the same first media parameters adjacently in each video group, thereby obtaining at least one video subgroup in each video group.

[0159] Figure 5 FIG2 shows a second flow chart of a video decoding method provided by an embodiment of the present application. Figure 4 ,like Figure 5 As shown, the electronic device may include multiple material videos. After the above S405, the method may include:

[0160] S501: The electronic device groups the to-be-analyzed material videos having the same first encoding format into a group to obtain at least one video group.

[0161] In this embodiment, only one decoder in the electronic device is in working state at the same time, and the electronic device is taken as an example for description. Based on this, after the electronic device obtains the video parameters of each to-be-analyzed material video, the electronic device first divides the to-be-analyzed material videos of the same first encoding format into a group to obtain at least one video group. In this way, the electronic device can reuse the decoder of the same encoding format to decode all the to-be-analyzed material videos in a video group, thereby saving the time of the electronic device querying the decoder and further improving the video decoding efficiency.

[0162] In S502, the electronic device arranges the to-be-analyzed material videos with the same first media parameter in each video group adjacently.

[0163] To further improve the video decoding efficiency, the electronic device can sort the to-be-analyzed material videos in the same video group based on the first media parameter of each to-be-analyzed material video in the same video group, so that the to-be-analyzed material videos with the same first media parameter in the same video group are arranged adjacently, and each video group is divided into at least one video subgroup. Then, the electronic device can decode the to-be-analyzed material videos in each video subgroup in turn. Based on this, for the to-be-analyzed material videos in the same video subgroup, the electronic device can decode the to-be-analyzed material videos in the video subgroup by using the same decoder in the above-described decoders.

[0164] For example, for each video subgroup: if the first decoder exists in the electronic device, the electronic device directly uses the first decoder to decode the to-be-analyzed material videos in the video subgroup. Or, if the second decoder exists, the electronic device sets the media parameter of the second decoder to the first media parameter and then enables the second decoder to decode the to-be-analyzed material videos in the video subgroup. Or, the electronic device creates a third decoder, sets the media parameter of the third decoder to the first media parameter, and then enables the third decoder to decode the to-be-analyzed material videos. Or, if the fifth decoder exists, the electronic device enables the fifth decoder to decode the to-be-analyzed material videos after setting the media parameter of the fifth decoder to the first media parameter and putting the fifth decoder in an idle state.

[0165] In this process, the decoder can work all the time (i.e., without frequent switching between the working state and the idle state) and without repeatedly performing the configuration operation. In this way, the electronic device can further save time and further improve the video decoding efficiency.

[0166] It should be noted that for the first to-be-analyzed material video in each video group, the electronic device can configure the decoder according to the specific conditions of the found decoder, put the decoder in a working state, or put the decoder in an idle state, etc. However, for the non-first to-be-analyzed material video in each video group, the electronic device can directly use the decoder used for the first to-be-analyzed material video in the corresponding video group to decode.

[0167] For example, the first media parameter can include resolution, frame rate, and rotation angle. For multiple to-be-analyzed material videos in the same video group, first, the electronic device can arrange the to-be-analyzed material videos in descending order of resolution size; then, the electronic device can arrange the to-be-analyzed material videos in descending order of frame rate size; and finally, the electronic device can arrange the to-be-analyzed material videos in descending order of rotation angle size.

[0168] Figure 6 A third flowchart of a video decoding method provided by an embodiment of the application is shown. In combination with Figure 4 As shown in S408, the method can include: Figure 6

[0169] S601, the electronic device queries whether there is a fifth decoder in a working state and having the same first encoding format as the first encoding format of the to-be-analyzed material video, but different media parameters from the first media parameters of the to-be-analyzed material video.

[0170] In this embodiment, at the same time, multiple decoders in a working state can exist in the electronic device. Therefore, after the electronic device does not find the first decoder in a working state and having the same video parameter as the video parameter of the to-be-analyzed material video, the electronic device can query whether there is a decoder in a working state and having different decoding parameters from the video parameter of the to-be-analyzed material video.

[0171] The decoding parameters of the decoder being different from the video parameters can include that the encoding format of the decoder is different from the first encoding format, but the media parameters of the decoder are the same as the first media parameters; or the encoding format of the decoder is the same as the first encoding format, but the media parameters of the decoder are different from the first media parameters; or the encoding format of the decoder is different from the first encoding format, and the media parameters of the decoder are different from the first media parameters. In this embodiment, a decoder in a working state and having the same first encoding format as the first encoding format of the to-be-analyzed material video, but different media parameters from the first media parameters of the to-be-analyzed material video is referred to as a fifth decoder.

[0172] ​That is, the electronic device first queries whether the fifth decoder in the working state exists after the first decoder is not queried, instead of directly querying whether the second decoder in the idle state exists.

[0173] S602, if the fifth decoder exists, the electronic device sets the media parameter of the fifth encoder as the first media parameter after the fifth decoder is in the idle state, and then enables the fifth decoder to decode the video to be analyzed based on the target analysis strategy.

[0174] This step can be referred to the description in S407, and the embodiments of the present application will not be described here.

[0175] Optionally, if the fifth decoder does not exist, the electronic device can execute the steps after S408, and the embodiments of the present application will not be described here.

[0176] It should be noted that in this embodiment, after the electronic device obtains the video parameters of all the videos to be analyzed, the electronic device can group all the videos to be analyzed according to the above-mentioned S501 and S502, or not group, and the embodiments of the present application will not be limited.

[0177] The above is to introduce the video decoding method provided by the embodiments of the present application with the electronic device as the execution subject. In the specific implementation process, the support of various software and hardware modules in the electronic device is needed. In the following Figures 7-12 , by the interaction of various software and hardware modules in the electronic device, the video decoding method provided by the embodiments of the present application is introduced.

[0178] Figure 7 Fig. 4 shows a flow diagram of a video decoding method provided by an embodiment of the present application. As shown in Fig. 4, the method can include: Figure 7

[0179] S01, the one-key movie module of the service layer receives the operation of enabling the one-key movie function by the user.

[0180] In an embodiment, as shown in B in Fig. 1, the operation of enabling the one-key movie function can be a click operation on the one-key movie control 121 in the first gallery interface 120. Figure 1 S02, the one-key movie module of the service layer loads and displays the candidate pictures and candidate videos.

[0181] In an embodiment, as shown in Fig. 1, the operation of enabling the one-key movie function can be a click operation on the one-key movie control 121 in the first gallery interface 120.

[0182] Figure 1 ​​As shown in Figure C, in response to a click on the one-click blockbuster control 121 in the first gallery interface 120, the one-click blockbuster module loads and displays multiple images and multiple videos in the second gallery interface 130. The videos in the second gallery interface 130 are candidate videos. The images in the second gallery interface 130 are candidate images.

[0183] In another embodiment, when there are only videos in the gallery, S02 is for the one-key blockbuster module of the business layer to load and display candidate videos. Alternatively, when there are only pictures in the gallery, S02 is for the one-key blockbuster module of the business layer to load and display candidate pictures.

[0184] S03. The one-click movie module of the business layer receives the user's operation of selecting the material video, and determines the operation of executing the one-click blockbuster function.

[0185] The stock video is a candidate video selected by the user. Of course, the one-click blockbuster module of the business layer in this step can receive the user's operation of selecting a stock picture. The stock picture is a candidate picture selected by the user.

[0186] For example, Figure 1 As shown in D in FIG, the operation of selecting the material video and material picture can be: clicking the candidate video and candidate picture in the third gallery interface 140. Figure 1 As shown in D in FIG, the operation of determining the execution of the one-key film-making function may be: a click operation on the video generation control 142 in the third gallery interface 140 .

[0187] S04. The one-click blockbuster module of the business layer calls the initialization interface of the media middle platform framework layer through the application function layer to initialize the relevant algorithms of the hardware abstraction layer.

[0188] Relevant algorithms refer to the algorithms for the business functions implemented by the business layer. Here, the business function is the one-click blockbuster feature. For example, relevant algorithms may include highlight segmentation algorithms, face detection algorithms, video acceleration algorithms, and image super-resolution algorithms.

[0189] S05. The initialization interface of the media middle platform framework layer sends initialization parameters to the algorithm module of the hardware abstraction layer through the channel interface of the media middle platform framework layer and the corresponding service interface of the application framework layer.

[0190] Among them, the service interface is used to transmit data between the channel interface and the algorithm module. Figure 7 etc.) The application framework layer is not shown.

[0191] S06. The algorithm module of the hardware abstraction layer uses an initialization algorithm to initialize each algorithm.

[0192] S07, the algorithm module of the hardware abstraction layer sends an initialization success message to the channel interface of the media middleware framework layer through the service interface of the application framework layer.

[0193] S08, the channel interface of the media middleware framework layer calls the capability query interface of the hardware abstraction layer through the service interface of the application framework layer to query the algorithm capability supported by the algorithm module.

[0194] The algorithm capability supported by the algorithm module can be the chip analysis speed of the chip (such as an image signal processor) corresponding to the algorithm module. The chip analysis speeds of different chips are different. For an electronic device that has been put on the market, the chip in the electronic device is fixed, and therefore the chip analysis speed corresponding to the electronic device is fixed. Based on this, the algorithm capability of the algorithm module in the electronic device is fixed.

[0195] In an embodiment, the algorithm capability can be a multiple. The larger the multiple, the stronger the algorithm capability, and the faster the analysis speed of the image.

[0196] For example, taking a material video with a time length of 60 seconds as an example. If the multiple is 1, the analysis time of the material video is 60 seconds. If the multiple is 2, the analysis time of the material video is 30 seconds.

[0197] S09, the capability query interface of the hardware abstraction layer sends the algorithm capability to the initialization interface of the media middleware framework layer through the service interface of the application framework layer, the channel interface of the media middleware framework layer.

[0198] In an embodiment, the capability query interface can also send other related performance parameters of various algorithms to the initialization interface through the service interface and the channel interface.

[0199] S10, the initialization interface of the media middleware framework layer sends an initialization success message to the one-key movie module of the service layer through the application function layer.

[0200] The initialization success message can include the algorithm capability of the algorithm module in the hardware abstraction layer.

[0201] It should be noted that after S10, the following S11-S13 can be performed for each material video. Hereinafter, taking a material video as video 1 as an example for introduction.

[0202] S11, the one-key movie module of the service layer sends a first query message to the analysis performance query interface of the media middleware framework layer through the application function layer.

[0203] The first query message includes the file descriptor of a material video and the algorithm capability of the algorithm module. Hereinafter, taking a material video as video 1 as an example for introduction.

[0204] S12, the analysis performance query interface of the media middle station framework layer obtains the estimated analysis time length of the video 1 according to the file descriptor of the video 1.

[0205] The introduction of the estimated analysis time length can refer to the related introduction in S403.

[0206] S13, the analysis performance query interface of the media middle station framework layer sends the estimated analysis time length of the video 1 to the one-key highlight module of the service layer through the application function layer.

[0207] S14, the one-key highlight module of the service layer obtains the analysis parameters according to the estimated analysis time length of all materials.

[0208] The all materials can include all material videos; or the all materials can include all material pictures; or the all materials can include all material videos and all material pictures.

[0209] The analysis parameters can include an analysis total time length, an analysis total time length upper limit suggestion value, a maximum time length of a highlight segment, a minimum time length of a highlight segment, a total highlight segment suggestion time length, a highlight segment suggestion time length, whether to force each material video to output a highlight segment, a selected highlight segment, and the like.

[0210] The whether to force each material video to output a highlight segment is by default yes. That is, in the embodiment, a highlight segment needs to be output for each material video. Of course, in other embodiments, whether to force each material video to output a highlight segment can be set according to actual conditions.

[0211] The introduction of the analysis total time length and the analysis total time length upper limit suggestion value can refer to the related introduction in S403.

[0212] The maximum time length of the highlight segment represents an expected maximum time length of a highlight segment. The minimum time length of the highlight segment represents an expected minimum time length of a highlight segment. The maximum time length of the highlight segment and the minimum time length of the highlight segment can be preset values.

[0213] The total highlight segment suggestion time length represents a suggestion value of the sum of the time lengths of the highlight segments of all material videos. The total highlight segment suggestion time length can be determined according to the number of material videos, the maximum time length of the highlight segment, and the minimum time length of the highlight segment.

[0214] The highlight segment suggestion time length represents a suggestion value of the time length of a highlight segment of a material video. The highlight segment suggestion time length can be determined according to the maximum time length of the highlight segment and the minimum time length of the highlight segment.

[0215] S15, the one-key large piece module of the service layer sends the file descriptors and analysis parameters of all materials to the policy monitoring module through the image highlight segment analysis interface of the application function layer and the media middle platform framework layer.

[0216] After the policy monitoring module of the media middle platform framework layer obtains the file descriptors and analysis parameters of all materials (such as material pictures and / or material videos), in an embodiment, as shown in Figure 8 the electronic device can continue to perform the following S16-S21.

[0217] Figure 8 Fig. 5 shows a flowchart of a video decoding method provided by an embodiment of the present application. In combination with Figure 7 as shown in Figure 8 after S15, the method can include:

[0218] S16, the policy monitoring module of the media middle platform framework layer obtains the to-be-analyzed material from all materials according to the total analysis time and the upper limit of the total analysis time.

[0219] In this step, reference can be made to the related description in S403 above.

[0220] Generally, since the analysis process of a picture is relatively simple and time-consuming is short, when the to-be-analyzed material includes to-be-analyzed material pictures and to-be-analyzed material videos, the to-be-analyzed material pictures are preferentially processed. For each to-be-analyzed material picture, the following S17-S21 can be performed. Hereinafter, taking a to-be-analyzed material picture as picture 1 as an example for introduction.

[0221] S17, the policy monitoring module of the media middle platform framework layer sends a first indication message to the channel interface of the media middle platform framework layer.

[0222] In the first indication message, the file descriptor of picture 1 can be included.

[0223] S18, the channel interface of the media middle platform framework layer obtains picture 1 according to the file descriptor of picture 1, and performs decoding, resolution reduction, format conversion and other processing on picture 1, and stores the processed picture 1.

[0224] Generally, in order to save the analysis time of the electronic device on the picture, before the picture is analyzed, the electronic device can perform resolution reduction processing on the picture.

[0225] For example, first, the electronic device decodes the picture 1 to obtain the frame data address of the picture 1. Then, the electronic device converts the decoded picture 1 from the first format to the second format. Then, the electronic device reduces the resolution of the picture 1 converted from the first format to the second format from the first resolution to the second resolution. Then, the electronic device converts the picture 1 reduced in resolution from the second format to the first format. Finally, the electronic device stores the picture 1 converted from the second format to the first format again.

[0226] It should be noted that, since the resolution reduction algorithm only supports the second format, the picture 1 needs to be converted from the first format to the second format first, and then the resolution of the picture 1 is reduced, and then the picture 1 reduced in resolution is converted to the original first format and stored in the memory.

[0227] Of course, in some embodiments, if the resolution reduction algorithm can support the first format, the electronic device does not need to perform format conversion on the picture 1, but can directly reduce the resolution after decoding. For this, the embodiments of the present application do not make any limitation.

[0228] S19, the channel interface of the media station framework layer sends the frame data address of the picture 1 to the algorithm module of the hardware abstraction layer through the service interface of the application framework layer.

[0229] S20, the algorithm module of the hardware abstraction layer obtains the picture 1 according to the frame data address of the picture 1, and performs image analysis on the picture 1 to obtain an analysis result of the picture 1.

[0230] Wherein, after the algorithm module obtains the picture 1, the algorithm module can score the picture 1 according to the image color, image texture feature, image quality and edge change rate value of the picture 1, etc. to obtain the analysis result. Wherein, the analysis result can include the image score, or the analysis result can include the image score and the scoring result.

[0231] For example, when the image score of the picture 1 is 0-19, the scoring result of the picture 1 is “low”. When the image score of the picture 1 is 20-49, the scoring result of the picture 1 is “medium”. When the image score of the picture 1 is 50-79, the scoring result of the picture 1 is “higher”. When the image score of the picture 1 is 80-100, the scoring result of the picture 1 is “high”.

[0232] In an embodiment, if the image score of a to-be-analyzed material picture is greater than or equal to 50, or the scoring result is “higher” or “high”, the electronic device can determine the to-be-analyzed material picture as a highlight picture.

[0233] S21, the algorithm module of the hardware abstraction layer sends the analysis result of the picture 1 to the policy monitoring module of the media middleware framework layer through the channel interface of the application framework layer and the media middleware framework layer.

[0234] After obtaining the analysis result of the picture 1, if there are other material pictures to be analyzed (such as picture 2), the electronic device can continue to perform S17-S21 to obtain the analysis result of all the material pictures to be analyzed.

[0235] After obtaining the analysis result of all the material pictures to be analyzed, if the material to be analyzed includes material videos to be analyzed, the electronic device can perform S22-S38 to obtain the analysis result of all the material videos to be analyzed. If the material to be analyzed does not include material videos to be analyzed, the electronic device directly outputs the first video composed of all the highlight pictures. Hereinafter, taking one of the material videos to be analyzed (such as video 1) as an example, the image analysis process of the video 1 is introduced.

[0236] Figure 9 Fig. 6 shows a flowchart of a video decoding method provided by an embodiment of the present application. In combination with the above-mentioned S21, the method can include: Figure 8 Figure 9 As shown in the above-mentioned S21, the method can include:

[0237] S22, the policy monitoring module of the media middleware framework layer determines a first number of first image frames of each material video to be analyzed.

[0238] The first image frame refers to an image frame to be analyzed in a material video to be analyzed in the overview analysis stage.

[0239] The first number can be greater than a preset minimum number (such as 5 frames), and the first number of one material video to be analyzed is less than the maximum number of image frames of the material video to be analyzed. The first number can be obtained after being allocated according to the first total analysis number of the first image frames of all the material videos to be analyzed.

[0240] ​In an embodiment, in this step: first, the strategy monitoring module determines the maximum number and the basic number of the first image frames in each video to be analyzed according to the video length and the algorithm capability of each video to be analyzed. The maximum number is greater than the basic number. Then, the strategy monitoring module can determine the first total number of the first image frames of all the videos to be analyzed according to the upper limit of the total analysis time. Then, the strategy monitoring module evenly allocates the first total number of the first image frames to each video to be analyzed. If there is still a remaining number after the even allocation, the remaining number can be preferentially allocated to the video to be analyzed with a longer length until the first total number of the first image frames is allocated.

[0241] The process of determining the number of the first image frames is introduced as follows:

[0242] If the length of the video to be analyzed is less than or equal to the first threshold value, the number of the first image frames is 1.

[0243] If the length of the video to be analyzed is greater than the first threshold value and less than or equal to the second threshold value, the number of the first image frames is: the length of the video to be analyzed divided by S and then rounded up plus m. S is a positive integer. For example, S is 5 seconds and the first threshold value is 3 seconds.

[0244] If the length of the video to be analyzed is greater than the second threshold value and less than or equal to the third threshold value, the number of the first image frames is: the second threshold value divided by S and then rounded up, the difference between the length of the video to be analyzed and the second threshold value divided by K and then rounded up, and m. K is a positive integer, the third threshold value is greater than the second threshold value, and the second threshold value is greater than the first threshold value. For example, K is 10 seconds, the second threshold value is 30 seconds, and the third threshold value is 90 seconds.

[0245] It should be noted that if the length of the video to be analyzed is longer, more threshold values can be set, such as a fourth threshold value, a fifth threshold value, etc. In the process of determining the maximum number and the basic number of the first image frames according to the above method, the threshold values corresponding to the maximum number are less than or equal to the threshold values corresponding to the basic number.

[0246] The process of determining the first total number of the first image frames is introduced as follows:

[0247] If the first multiple (such as 0.3) of the upper limit of the total analysis time is less than the sum of the processing time of the first image frames with the basic number in all the videos to be analyzed, the first total number of the first image frames is the sum of the basic number of the first image frames in all the videos to be analyzed.

[0248] If the first multiplier of the recommended upper limit of the total analysis time is greater than the sum of the processing times of the maximum number of first image frames in all the material videos to be analyzed, then the first total analysis number of first image frames is the sum of the maximum number of first image frames in all the material videos to be analyzed.

[0249] If the first multiplier of the recommended upper limit of the total analysis duration is greater than the sum of the processing times for the minimum number of first image frames in all the source videos to be analyzed, and less than the sum of the processing times for the maximum number of first image frames in all the source videos to be analyzed, then the first total number of first image frames to be analyzed is calculated as follows: the first multiplier of the recommended upper limit of the total analysis duration divided by T, rounded to the nearest integer. T is the time it takes to process one image frame.

[0250] If the recommended upper limit of the total analysis time is less than the sum of the processing times of the basic number of first image frames in all the material videos to be analyzed, then the first total analysis number of the first image frames is: the recommended upper limit of the total analysis time divided by T and rounded up.

[0251] It should be noted that after determining the first number of first image frames in each material video to be analyzed, the following S23 to S38 may be executed for each material video to be analyzed.

[0252] S23. The policy monitoring module of the media middle platform framework layer sends the file descriptor of video 1, video 1, and the position of the first image frame in video 1 to the channel interface of the media middle platform framework layer.

[0253] The first first image frame may be a first image frame at any position in the video 1 .

[0254] S24. The channel interface of the media middle platform framework layer creates a target channel with preset functions.

[0255] The preset function refers to: sequentially performing decoding, first format conversion, resolution reduction, second format conversion, etc. Alternatively, the preset function refers to: sequentially performing decoding, resolution reduction, etc.

[0256] S25. The channel interface of the media middle platform framework layer creates a media extractor corresponding to the file descriptor of video 1 in the target channel, and extracts the video parameters of video 1 through the media extractor.

[0257] The video parameters of video 1 may include: a first encoding format and a first media parameter.

[0258] In an embodiment, before the query decoder, the channel interface can obtain the video parameters of all the to-be-analyzed material videos, and then arrange the to-be-analyzed material videos with the same video parameters together based on the video parameters of all the to-be-analyzed material videos. Then, the following S26 and subsequent steps are executed. Specifically, the arrangement method can refer to the related description in S501 and S502.

[0259] S26, the channel interface of the media middleware framework layer sends a second query message to the decoder cache pool of the media middleware framework layer.

[0260] The second query message includes the video parameters of video 1. The second query message is used to instruct the decoder cache pool to query whether there is a first decoder in the working state and with the same decoding parameters as the video parameters of video 1 in the decoder cache pool.

[0261] S27, the decoder cache pool of the media middleware framework layer queries whether there is a first decoder in the working state and with the same decoding parameters as the video parameters of video 1.

[0262] The above step can refer to the related description in S406.

[0263] It should be noted that the first decoder will return to the decoder cache pool after executing the current decoding task in the channel interface.

[0264] S28, if there is a first decoder, the decoder cache pool of the media middleware framework layer returns the first decoder to the channel interface of the media middleware framework layer.

[0265] S29, the channel interface of the media middleware framework layer directly decodes the first first image frame through the first decoder according to the position of the first first image frame in the target channel.

[0266] After the decoder decodes the first first image frame, the frame data address of the first first image frame can be obtained. After the channel interface decodes the first first image frame in the target channel, the channel interface can also perform format conversion, resolution reduction, etc. on the first first image frame, and store the processed first first image frame.

[0267] The above step of decoding the first first image frame through the first decoder can refer to the related description of decoding the first first image frame through the first decoder in S407.

[0268] S30, if there is no first decoder, the decoder cache pool of the media middleware framework layer queries whether there is a second decoder.

[0269] The second decoder is in an idle state, and the decoding format of the second decoder is the same as the first decoding format. The step can refer to the related description in S408.

[0270] S31, if the second decoder exists, the decoder cache pool of the media middleware framework layer returns the second decoder to the channel interface of the media middleware framework layer.

[0271] S32, the channel interface of the media middleware framework layer sets the media parameters of the second decoder as the first media parameters of video 1 in the target channel, and then enables the second decoder to decode the first image frame.

[0272] The step can refer to the related description in S407.

[0273] S33, if the second decoder does not exist, the decoder cache pool of the media middleware framework layer creates a third decoder with the first decoding format of video 1.

[0274] The process of creating the third decoder in the step can refer to the related description in S410.

[0275] S34, the decoder cache pool of the media middleware framework layer sends the third decoder to the channel interface of the media middleware framework layer.

[0276] S35, the channel interface of the media middleware framework layer sets the media parameters of the third decoder as the first media parameters of video 1 in the target channel, and then enables the third decoder to decode the first image frame.

[0277] S36, the channel interface of the media middleware framework layer sends the frame data address of the first image frame to the algorithm module of the hardware abstraction layer through the service interface of the application framework layer.

[0278] S37, the algorithm module of the hardware abstraction layer obtains the first image frame from video 1 according to the frame data address of the first image frame, and performs image analysis on the first image frame based on the high light fragment algorithm to obtain the analysis result of the first image frame.

[0279] The image analysis process of the first image frame in the step can refer to the related content of the image analysis of picture 1 in S20, which will not be described here. Similarly, the analysis result of the first image frame can refer to the analysis result of picture 1 in S20.

[0280] S38, the algorithm module of the hardware abstraction layer sends the analysis result of the first image frame to the policy monitoring module of the media middleware framework layer through the service interface of the application framework layer and the channel interface of the media middleware framework layer.

[0281] It should be noted that for one to-be-analyzed material video: after the policy monitoring module obtains the analysis result of the first first image frame, if there is a second first image frame, the above S23-S38 is continuously executed until the number of analyzed first image frames is the first number.

[0282] In some embodiments, as shown in Figure 10 If there are multiple decoders in the working state in the target channel, after S27, the method can include:

[0283] S271, if there is no first decoder, the decoder cache pool of the media middleware framework layer queries whether there is a fifth decoder in the working state and the encoding format is the same as the first encoding format of video 1, but the media parameter is different from the first media parameter of video 1.

[0284] S272, if there is a fifth decoder, the decoder cache pool of the media middleware framework layer returns the fifth decoder to the channel interface of the media middleware framework layer.

[0285] S273, the channel interface of the media middleware framework layer enables the fifth decoder to decode the first first image frame after the fifth decoder is in the idle state and the media parameter of the fifth decoder is set to the first media parameter of video 1 in the target channel.

[0286] Wherein, this step can refer to the related introduction in the above S602. If there is no fifth decoder, continue to execute the above S30. Otherwise, after S273, directly execute S36.

[0287] In an embodiment, the channel interface will release the target channel after processing all to-be-analyzed material videos.

[0288] After obtaining the analysis result of all first image frames in video 1, in some embodiments, referring to Figure 11 , the electronic device can continue to execute the following S39-S42.

[0289] Figure 11 Fig. 8 shows a flow diagram of a video decoding method provided by an embodiment of the application. In combination with Figure 9 , as shown in Figure 11 After the above S38, the method can include:

[0290] S39, the policy monitoring module of the media middleware framework layer determines the target region of video 1 based on the analysis result of the first image frame in video 1.

[0291] The policy monitoring module can determine the region in which a plurality of first image frames before and after the first target image frame within a first preset time length as the target region of the to-be-analyzed material video. The first target image frame is the first image frame with the highest image score in all first image frames in the to-be-analyzed material video. One to-be-analyzed material video can include at least one target region.

[0292] It should be noted that the first preset time length is related to the actual time length of the to-be-analyzed material video, the highlight segment suggestion time length, the first quantity, and the like.

[0293] S40, the policy monitoring module of the media middle station framework layer determines the second quantity of second image frames that need to be analyzed in the target region of video 1.

[0294] The second quantity refers to the number of second image frames that need to be analyzed in the target region of each to-be-analyzed material video within the remaining analysis time length.

[0295] In an embodiment, first, the policy monitoring module determines the second total analysis quantity of second image frames that need to be analyzed in the target region of all to-be-analyzed material videos. Then, the policy monitoring module determines the second quantity of second image frames in the target region of each to-be-analyzed material video according to the time length of the target region of each to-be-analyzed material video and the score of the target region.

[0296] For example, the policy monitoring module can determine the second total analysis quantity according to the ratio of the remaining analysis time length to the processing time length of one image frame, but is not limited thereto.

[0297] For example, the policy monitoring module can determine the second quantity of second image frames in each target region according to the ratio of the number of all image frames contained in each target region to the total number of image frames of all target regions.

[0298] S41, the policy monitoring module of the media middle station framework layer, the channel interface, the decoder cache pool, the service interface of the application program framework layer, and the algorithm module of the hardware abstraction layer cooperate with each other to obtain the analysis result of each second image frame in the target region of video 1.

[0299] It should be noted that this step can refer to the related description in S23-S38 described above, and the embodiments of the present application will not be described herein.

[0300] S42, the policy monitoring module of the media middle station framework layer determines the highlight segment of video 1 based on the analysis result of each second image frame in the target region of video 1.

[0301] The policy monitoring module determines the area of the plurality of second image frames within the second preset time period before and after the second target image frame as the highlight segment. The second target image frame is the second image frame with the highest image score among all second image frames of the video 1.

[0302] It should be noted that the second preset time period is related to the actual time length of the target area, the highlight segment suggestion time length, the second quantity, etc.

[0303] In the case where the to-be-analyzed materials include to-be-analyzed material videos and to-be-analyzed material pictures, after the highlight segments of all to-be-analyzed material videos are obtained, in some embodiments, the electronic device can continue to perform the following S43-S52. Figure 12

[0304] Figure 12 Fig. 9 shows a flowchart of a video decoding method according to an embodiment of the present application. In combination with Figure 11 As shown in Fig. 9, after the above S42, the method can include: Figure 12

[0305] S43, the policy monitoring module of the media middleware framework layer sends the analysis results of all to-be-analyzed materials to the application function layer through the image highlight segment analysis interface of the media middleware framework layer.

[0306] The analysis results of the to-be-analyzed materials can include the position of the highlight picture and the position of the highlight segment.

[0307] S44, the application function layer performs editing and screening on all to-be-analyzed materials according to the analysis results of all to-be-analyzed materials, to obtain all highlight pictures and highlight segments.

[0308] S45, the application function layer calls the theme summary interface of the media middleware framework to request to obtain the theme type.

[0309] The theme summary interface includes a theme algorithm.

[0310] S46, the theme summary interface of the media middleware framework obtains the theme type.

[0311] S47, the theme summary interface of the media middleware framework sends the theme type to the application function layer.

[0312] S48, the application function layer obtains the theme template corresponding to the theme type.

[0313] The application function layer includes a template set, and the template set includes a plurality of templates. After the application function layer obtains the theme type, the application function layer obtains the template corresponding to the theme type from the template set as the theme template.

[0314] ​​S49. The application function layer sends the theme template, all highlight clips, and all highlight images to the basic capability layer.

[0315] S50: The basic capability layer stitches all highlight clips and all highlight images, and applies a theme template to obtain a first finished video.

[0316] Among them, this step can refer to the relevant introduction in the above S413, and the embodiment of this application will not be repeated here.

[0317] S51. The basic capability layer sends a second instruction message for playing the completed video to the business layer.

[0318] S52: The business layer plays the first completed video.

[0319] like Figure 13 As shown, an embodiment of the present application also provides a chip system. The chip system 1300 includes at least one processor 1301 and at least one interface circuit 1302. The at least one processor 1301 and the at least one interface circuit 1302 can be interconnected via lines. The processor 1301 is used to support the electronic device to implement the various steps in the above method embodiment, and the at least one interface circuit 1302 can be used to receive signals from other devices (such as a memory) or send signals to other devices (such as a communication interface). The chip system may include a chip and may also include other discrete devices.

[0320] An embodiment of the present application further provides a computer storage medium, which includes instructions. When the instructions are executed on the above-mentioned electronic device, the electronic device executes each step in the above-mentioned method embodiment.

[0321] An embodiment of the present application further provides a computer program product comprising instructions, which, when executed on the electronic device, causes the electronic device to execute each step in the method embodiment.

[0322] Regarding the technical effects of the chip system, computer storage medium, and computer program product, refer to the technical effects of the previous method embodiments.

[0323] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0324] Those skilled in the art can clearly understand that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0325] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and module can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0326] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between devices or modules, which can be electrical, mechanical or other forms.

[0327] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical modules, that is, they can be located in one device, or can be distributed to multiple devices. According to actual needs, part or all of the modules can be selected to achieve the purpose of the embodiment.

[0328] In addition, each functional module in each embodiment of the present application can be integrated in one device, or each module can exist physically alone, or two or more modules can be integrated in one device.

[0329] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by using a software program, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer storage medium or transmitted from one computer storage medium to another computer storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer storage medium can be any available medium that can be accessed by a computer or data storage device including one or more servers, data centers, etc. integrated with the medium. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0330] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A video decoding method, characterized in that: Applied to electronic equipment, the method includes: Obtaining video parameters of the material video to be analyzed; wherein the video parameters include a first encoding format and a first media parameter of the video; If a first decoder is in operation and has decoding parameters identical to the video parameters, the first decoder is used to decode the material video to be analyzed; wherein the decoding parameters are identical to the video parameters, including: a second encoding format in the decoding parameters is identical to the first encoding format, and a second media parameter in the decoding parameters is identical to the first media parameter; If the first decoder does not exist, query whether there is a second decoder in an idle state and having the same encoding format as the first encoding format; If the second decoder exists, after setting the media parameters of the second decoder to the first media parameters, the second decoder is enabled to decode the material video to be analyzed.

2. The method according to claim 1, characterized in that The method further comprises: If the second decoder does not exist, a third decoder whose encoding format is the first encoding format is created, and after the media parameters of the third decoder are set to the first media parameters, the third decoder is enabled to decode the material video to be analyzed.

3. The method according to claim 2, characterized in that The step of creating a third decoder whose encoding format is the first encoding format includes: querying whether the current number of decoders in the decoder cache pool of the electronic device is equal to N; wherein N is the maximum number of decoders in the decoder cache pool, and N is a positive integer greater than or equal to 2; If the current number is equal to N, after releasing the fourth decoder, create the third decoder in the decoder cache pool; wherein the fourth decoder is the decoder with the longest idle time in the decoder cache pool; If the current number is less than N, the third decoder is created.

4. The method according to any one of claims 1 to 3, characterized in that After obtaining the video parameters of the material video to be analyzed, the method further includes: If there is a fifth decoder in a working state, and the encoding format is the same as the first encoding format, but the media parameters are different from the first media parameters, then after the fifth decoder is in an idle state, the media parameters of the fifth decoder are set to the first media parameters, and then the fifth decoder is enabled to decode the material video to be analyzed.

5. The method according to any one of claims 1 to 3, characterized in that The electronic device includes a plurality of material videos to be analyzed; before decoding the material videos to be analyzed using a first decoder that is in a working state and has decoding parameters identical to the video parameters, the method further includes: Dividing a plurality of to-be-analyzed material videos having the same first encoding format into a group to obtain at least one video group; wherein each video group includes at least one video subgroup, and the media parameters of the to-be-analyzed material videos in each video subgroup are the same; The electronic device decodes the material video to be analyzed in each video group in turn; for each video group, if the first decoder exists, the first decoder is used to decode the material video to be analyzed in the video group; for each video group, if the second decoder exists, the media parameters of the second decoder are set to the first media parameters, and then the second decoder is enabled to decode the material video to be analyzed in the video group.

6. The method according to any one of claims 1 to 3, characterized in that Before obtaining the video parameters of the material video to be analyzed, the method further includes: obtaining the material video to be analyzed and a file descriptor of the material video to be analyzed; wherein one file descriptor of the material video to be analyzed is used to uniquely identify the material video to be analyzed; The step of obtaining the video parameters of the material video to be analyzed includes: Creating a media extractor corresponding to the file descriptor of the material video to be analyzed; wherein the media extractor is used to separate video data and audio data in the video, and extract video parameters of the video from the video data; Using the media extractor to obtain video data of the material video to be analyzed from the material video to be analyzed; The media extractor is used to extract video parameters of the material video to be analyzed from the video data.

7. The method according to any one of claims 1 to 3, characterized in that Decoding the to-be-analyzed material video using a decoder in the electronic device includes: Decoding each first image frame of a first number of first image frames in the material video to be analyzed using a decoder in the electronic device; The decoder in the electronic device includes the first decoder and / or the second decoder; the first number of first image frames are evenly distributed at various positions of the material video to be analyzed; or, the first number of first image frames are I frames in the material video to be analyzed.

8. The method according to claim 7, characterized in that After decoding each of the first number of first image frames in the material video to be analyzed using the decoder in the electronic device, the method further includes: Obtaining an image score for each first image frame; Determining an area where a plurality of consecutive first image frames within a second preset time period before and after a first target image frame are located as a target area of ​​the material video to be analyzed; wherein the first target image frame is the first image frame with the highest image score among all first image frames in the material video to be analyzed; acquiring a second number of second image frames from the target area; decoding, using a decoder in the electronic device, each second image frame in a second number of second image frames in the target area; The second number of image frames are evenly distributed at various positions of the target area of ​​the material video to be analyzed; or the second number of image frames are I frames in the target area of ​​the material video to be analyzed.

9. An electronic device, characterized in that: The electronic device comprises one or more processors and a memory coupled to the processor; the memory stores computer program code, which includes instructions; when the instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The method comprises instructions, which, when executed on an electronic device, cause the electronic device to execute the method according to any one of claims 1 to 8.

11. A computer program product, characterized in that The method comprises instructions, which, when executed on an electronic device, cause the electronic device to execute the method according to any one of claims 1 to 8.

Images