Electronic device and multimedia data generating method thereof

Abstract

The present application relates to an electronic device and a method for generating multimedia data thereof. The multimedia data generating method comprises: displaying a shooting preview screen of a dynamic photo; generating and storing a first feature detection result of preview video data; detecting a user's shooting instruction, generating and storing a first feature identifier of a first dynamic photo, wherein the first dynamic photo comprises preview video data, a first preview screen where a shooting instruction is detected, and collected video data collected after the shooting instruction is detected. Through the above method, feature detection is performed on the video in the captured dynamic photo and tags are added to store the correspondence between the dynamic photo and the feature detection result, so that the shooting content corresponding to the dynamic photo can be displayed more concisely through tags. In the process of making short videos, dynamic photos can also be classified or sorted according to tags and displayed, so that users can more quickly select appropriate dynamic photos based on tags, and the user experience is better.

Description

Technical Field

[0001] The present application relates to the technical field of terminal equipment, and in particular to an electronic device and a method for generating multimedia data thereof. Background Art

[0002] Dynamic photos not only include photos taken when the user presses the camera button, but also include videos taken within a period of time before and after the user presses the camera button. This also enables dynamic photos to record richer shooting content, and therefore, they are used more and more widely.

[0003] Since dynamic photos include videos, users can use multiple dynamic photos to generate short videos. Short videos can include, for example, video blogs (Video Log, Vlog), etc., by splicing multiple dynamic photos together to generate a Vlog. If there are too many dynamic photos in the album, users need to search and confirm the shooting content of the dynamic photos used to make the Vlog one by one, which results in a poor user experience. Summary of the invention

[0004] The present application provides an electronic device and a method for generating multimedia data thereof.

[0005] In a first aspect, an embodiment of the present application provides a method for generating multimedia data, which is applied to an electronic device with a shooting function, and the method includes:

[0006] Display the shooting preview screen of the dynamic photo;

[0007] Generate and store a first feature detection result of the preview video data;

[0008] Detecting a shooting instruction of a user, generating and storing a first feature identifier of a first dynamic photo, wherein the first dynamic photo includes preview video data, a first preview screen after the shooting instruction is detected, and collected video data collected after the shooting instruction is detected, and

[0009] The first feature identification includes a first feature detection result, a second feature detection result of the collected video data, and a third feature detection result of the first preview screen.

[0010] In the present application, the electronic device here may also be referred to as a terminal device, which is a variety of mobile terminals with a shooting function, such as a mobile phone. The shooting preview screen here can be a preview screen in which the user points the mobile phone at the shooting object for framing after the user turns on the dynamic photo mode of the mobile phone and before the user presses the photo button. The electronic device can cache and update the preview video data. If the user clicks the photo button of the camera application of the electronic device, the preview video data can be used to record the change process of the shooting object before the user clicks the photo button of the camera application. The first feature detection result here may include the feature detection result obtained by the electronic device performing feature detection on the shooting content corresponding to each video frame included in the preview video data, including scene detection results, human body detection results, face detection results, action evaluation results, etc.

[0011] For example, the shooting instruction here can be an instruction generated at the shooting time point when the user clicks the shooting button of the camera application of the electronic device. The first dynamic photo includes preview video data, a first preview screen where the shooting instruction is detected, and collected video data collected after the shooting instruction is detected.

[0012] It can be understood that the first feature identification includes the first feature detection result of the preview video data, the second feature detection result of the collected video data, and the third feature detection result of the first preview screen. The second feature detection result may include the feature detection result obtained by the electronic device performing feature detection on the shooting content corresponding to each video frame included in the collected video data. The third feature detection result may include the feature detection result obtained by the electronic device performing feature detection on the shooting content corresponding to the first preview screen.

[0013] In a possible implementation of the first aspect above, the first feature detection result, the second feature detection result and the third feature detection result include: at least one of: a scene detection result, a human body detection result, a face detection result, and an action evaluation result.

[0014] In a possible implementation of the first aspect, generating and storing a first feature detection result of the preview video data includes:

[0015] In the process of generating and storing preview video data, feature detection is performed on each video frame included in the preview video data to obtain image features of each video frame. The first feature detection result includes image features obtained after merging and deduplicating the image features of each video frame.

[0016] In a possible implementation of the first aspect, detecting a shooting instruction of a user and generating and storing a first feature identifier of a first dynamic photo includes:

[0017] After detecting the user's shooting instruction, feature detection is performed frame by frame on the video frames included in the preview video data to obtain a first feature detection result.

[0018] In a possible implementation of the first aspect above, the content of the first feature identification includes: at least one of: sports, travel, scenery, and childishness.

[0019] In the present application, the first characteristic identifier here can be a mark of each dynamic photo, and each dynamic photo can include at least one mark, such as: childishness, portrait, scenery, etc.

[0020] In a possible implementation of the first aspect above, the first feature identifier is displayed on the first dynamic photo.

[0021] In the present application, the first feature identifier here can be displayed in a local area on the first dynamic photo.

[0022] In a possible implementation of the first aspect, generating and storing a first feature identifier of the first dynamic photo includes:

[0023] The first feature detection result, the second feature detection result and the third feature detection result are combined and deduplicated to obtain a first feature identifier.

[0024] In a possible implementation of the first aspect, the present invention further includes:

[0025] In the storage display interface of the dynamic photo, the first dynamic photo and the stored second dynamic photo are arranged adjacent to each other, wherein the second characteristic identifier of the second dynamic photo is the same as the first characteristic identifier of the first dynamic photo.

[0026] In the present application, the storage display interface of the dynamic photo here can be the application interface of the gallery or album application, that is, the creation interface in Figure 2(c). In some embodiments, the storage display interface of the dynamic photo can also be the application interface for making dynamic photos in the gallery or album application displayed on the screen of the electronic device after the user clicks the "One-click to create a film" (One-click blockbuster) control in the application interface of the gallery or album application.

[0027] In a possible implementation of the first aspect, the present invention further includes:

[0028] In the storage and display interface of dynamic photos, the first dynamic photo is located before the third dynamic photo, wherein the first feature identifier of the first dynamic photo belongs to the feature identifier in the sorting condition, and the third feature identifier of the third dynamic photo does not belong to the feature identifier in the sorting condition.

[0029] In the present application, the album can classify and sort the dynamic photos according to the tags and present them to the user, that is, the album can display the priority sorting results of the dynamic photos. Before the user clicks the "One-click Movie" (One-click Blockbuster) control, that is, in the dynamic photo option corresponding to the "One-click Movie" control, the user is given a default selection and recommendation of a part of the dynamic photos and checked. In some embodiments, in addition to recommending dynamic photos, a part of other photos or videos may also be included. Dynamic photos, other photos or videos will be sorted uniformly (for example, sorted by shooting time, etc.), and the album can select dynamic photos, other photos or videos used to generate Vlog by default and recommend them to users (checked).

[0030] In a possible implementation of the first aspect, the present invention further includes:

[0031] The user's short video generation operation is detected, and the dynamic photos selected by the user are generated into a short video.

[0032] In this application, the short video generation operation here allows users to add or remove selected dynamic photos, other photos or videos. After selecting, click the "One-click Video" control to generate a Vlog video.

[0033] In a second aspect, the present application provides an electronic device, including:

[0034] a memory for storing instructions to be executed by one or more processors of the electronic device, and

[0035] The processor is one of the processors of the electronic device, and is used to execute the multimedia data generation of the first aspect.

[0036] In a third aspect, the present application provides a computer program product, comprising: a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium contains a computer program code for executing the multimedia data generation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1(a) to Figure 1(e) A schematic diagram of a scene in which an electronic device displays a dynamic photo provided by an embodiment of the present application;

[0038] Figure 2(a) to Figure 2(d) A schematic diagram of a scenario in which an electronic device selects dynamic photos to generate a short video provided by an embodiment of the present application;

[0039] Figure 3(a) to Figure 3(g) A schematic diagram of a scene of a mobile phone displaying dynamic photos provided in an embodiment of the present application;

[0040] Figure 4 A schematic diagram of a process for selecting dynamic photos to generate short videos by an electronic device provided in an embodiment of the present application;

[0041] Figure 5 A flowchart for implementing the generation of short videos for each functional module in the system architecture of the electronic device provided in the embodiment of the present application;

[0042] Figure 6 An interactive schematic diagram of selecting dynamic photos to generate short videos for an electronic device provided in an embodiment of the present application;

[0043] FIG. 7( a ) and FIG. 7( b ) are schematic diagrams of a process of selecting dynamic photos to generate short videos by an electronic device provided in an embodiment of the present application;

[0044] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0045] Fig. 9 A software structure block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and in detail below with reference to the accompanying drawings.

[0047] It can be understood that the technical method of the present application is applicable to various electronic devices with shooting functions, which can also be called terminal devices, for example, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA) and other mobile terminals.

[0048] The following describes an embodiment of the present application by taking a mobile phone as an example of an electronic device.

[0049] Figure 1(a) to Figure 1(e) A schematic diagram of a scene in which a mobile phone 100 displays dynamic photos is introduced. As shown in FIG1(a), the screen of the mobile phone 100 displays a dynamic photo 101 saved in the album, and the dynamic photo 101 includes a photographed person A and a photographed person B. The mobile phone 100 receives a user's click operation on the dynamic photo 101, and in response to the operation, as shown in FIG1(b), the mobile phone 100 displays the dynamic photo 101 in full screen. The mobile phone 100 receives a user's click operation on the dynamic photo 101, and in response to the operation, as shown in FIG1(b), the mobile phone 100 displays the dynamic photo 101 in full screen. Figure 1(c) to Figure 1(e) , the mobile phone 100 plays the video 102 corresponding to the dynamic photo 101, for example, a video with a duration of 3 seconds. FIG1(c) shows that the screen of the mobile phone 100 first displays the photographed person A, such as Figure 1(d) and1(e) As shown, the screen of the mobile phone 100 then displays the photographed person B and the photographed person A side by side. After the photo is taken, the photographed person A leaves.

[0050] In some embodiments, the user wants to create a Vlog by selecting dynamic photos from the album. As shown in FIG2(a), the user opens the album of the mobile phone 100, and the application interface of the album displays multiple photos and a creation control 2001. The mobile phone 100 receives a click operation of the user on the creation control 2001. In response to the operation, as shown in FIG2(b), the mobile phone 100 displays the creation interface. The creation interface displays multiple photos and a free creation control 2002. The mobile phone 100 receives a click operation of the user on the free creation control 2002. In response to the operation, as shown in FIG2(c), the mobile phone 100 displays multiple photos at a glance through the creation interface. The creation interface here can be an application interface for generating dynamic photos in the album. The mobile phone 100 receives a click operation of the user on multiple photos. In response to the operation, as shown in FIG2(d), the mobile phone 100 displays multiple photos that the user has selected through the creation interface and a start creation control 2003. When the mobile phone 100 receives a click operation of the user on the start creation control 2003, in response to the operation, the Vlog is created. It can be understood that the mobile phone 100 can save the Vlog file in the video folder.

[0051] As mentioned above, in the process of users selecting dynamic photos from the album to make a Vlog, due to the large number of dynamic photos, users need to click to open and confirm the shooting content of the dynamic photos one by one, and screen out suitable dynamic photos to make a Vlog, which makes the steps of making a Vlog cumbersome and time-consuming, and the user experience is not good.

[0052] In order to solve the above problems, an embodiment of the present application provides a method for generating multimedia data. In the process of shooting dynamic photos, after marking the dynamic photos, the dynamic photos are added to the corresponding types for display, that is, in the process of generating Vlogs from dynamic photos, the dynamic photos are displayed in categories, and can be sorted according to the default sorting rules or the sorting rules set by the user for user selection. For example, feature detection (also called semantic detection) is performed on the video data in the dynamic photos (including the video data cached before the user presses the photo button and the video data collected after the user presses the photo button), that is, feature detection is performed on the shooting content corresponding to each video frame included in the video data, and the feature detection results are stored, including scene detection results, human body detection results, face detection results, action evaluation results, etc. A mark corresponding to the feature detection result is added to the video data (each video frame), so that the dynamic photos including the video generated based on the video data also have a mark, and the shooting content corresponding to the dynamic photos is displayed through the mark.

[0053] In some embodiments, when a user opens an album (also referred to as a gallery) and selects a dynamic photo to generate a Vlog, the application interface for generating dynamic photos in the album is entered. The album can classify and sort the dynamic photos according to the tags and present them to the user, that is, the album can display the priority sorting results of the dynamic photos. Before the user clicks the "One-click Movie" (One-click Blockbuster) control, that is, in the dynamic photo option corresponding to the "One-click Movie" control, the user is given a default selection and recommendation of a part of the dynamic photos and checked. In some embodiments, in addition to recommending dynamic photos, a part of other photos or videos may also be included. Dynamic photos, other photos or videos will be sorted uniformly (for example, sorted by shooting time, etc.), and the album can select dynamic photos, other photos or videos for generating Vlog by default, and recommend them to the user (checked). On this basis, the user can add or subtract the selected dynamic photos, other photos or videos by himself. After selecting, click the "One-click Movie" control to generate a Vlog video.

[0054] In some embodiments, the user can click the "One-click to Video" control to directly use the default dynamic photos, other photos or videos to generate a Vlog video.

[0055] In some embodiments, the tags here may include sports, such as: playing football, playing basketball, etc., traveling: taking a train, taking a plane, etc., and scenery: seaside, forest, etc.

[0056] In addition, it can be understood that when marking dynamic photos, if the video frame corresponding to the video data included in the dynamic photo has multiple tags, the generated dynamic photos can also have multiple tags accordingly. For example, the user interface shown in Figure 3 (c) can be the creation interface of the album displayed after the mobile phone 100 receives the user's click operation on the one-key film control 3002 as shown in Figure 3 (b), and the creation interface displays multiple dynamic photos at a glance, wherein each dynamic photo can include at least one tag, such as: childishness, portrait, scenery, etc. Exemplarily, the album of the mobile phone 100 can also classify and sort the dynamic photos according to the tags of the dynamic photos, for example: arranging the dynamic photos in descending order according to the number of tags and arranging dynamic photos with the same tags adjacent to each other.

[0057] In some embodiments, for dynamic photos, the tags in the video data whose number is less than the frame number threshold can also be removed to make the tagging of dynamic photos more accurate. For example, if the video data included in the dynamic photo has 100 frames, the tags corresponding to the video frames whose number is less than 10 can be removed.

[0058] It can be seen that through the above multimedia data generation method, in the process of shooting dynamic photos, feature detection is performed on the video included in the dynamic photos, and the corresponding relationship between the dynamic photos and the feature detection results is stored by adding tags, so that the shooting content corresponding to the dynamic photos can be displayed more concisely through tags. In the process of users selecting dynamic photos to make Vlogs, dynamic photos can also be classified or sorted according to tags and then displayed, which is convenient for users to select appropriate dynamic photos more quickly based on tags, simplifying the steps of making Vlogs, and providing a good user experience.

[0059] In order to more clearly and in detail introduce the multimedia data generation method provided by the embodiment of the present application, Figure 4 First, a flowchart of a method for generating multimedia data by a terminal device according to an embodiment of the present application is introduced. Figure 4 The method shown can be implemented by a processor of a terminal device executing related instructions. The multimedia data generation method can include the following steps.

[0060] S401: Displaying a shooting preview screen of a dynamic photo.

[0061] For example, the user can open the camera application of the terminal device and turn on the dynamic photo shooting mode of the camera application. After the user turns on the dynamic photo shooting mode of the camera application of the terminal device, the user can aim the camera of the terminal device at the subject to be photographed. The application interface of the camera application can display a shooting preview screen of the subject.

[0062] S402: Generate and store a first feature detection result of the preview video data.

[0063] Exemplarily, the preview video data here may be a video of a first time period, and during the framing process, the terminal device may cache and update the preview video data. If the user clicks a photo button of a camera application of the terminal device, the preview video data may be used to record the change process of the photographed object before the user clicks the photo button of the camera application.

[0064] The first feature detection result may include a feature detection result obtained by the terminal device performing feature detection on the shooting content corresponding to each video frame included in the preview video data, including a scene detection result, a human body detection result, a face detection result, an action evaluation result, and the like.

[0065] In some embodiments, the terminal device may perform feature detection on the captured content corresponding to each video frame of the acquired preview video data in real time, and store the corresponding relationship between the obtained feature detection results and the video frames. In another embodiment, the terminal device may also cache the preview video data first, until it detects that the user clicks the photo button of the camera application of the terminal device, obtain the cached preview video data and perform feature detection on the captured content corresponding to each video frame of the preview video data, and obtain the feature detection results corresponding to the preview video data. It can be understood that the preview video data may include multiple video frames, and accordingly, the preview video data will also have multiple feature detection results.

[0066] S403: Detecting a shooting instruction from the user, generating and storing a first feature identifier of the first dynamic photo.

[0067] For example, the shooting instruction here can be an instruction generated at the shooting time point when the user clicks the shooting button of the camera application of the terminal device. The first dynamic photo includes preview video data, a first preview screen where the shooting instruction is detected, and collected video data collected after the shooting instruction is detected.

[0068] It can be understood that after the user clicks the camera button of the camera application, the camera application of the terminal device not only captures the first preview screen of the object when the user clicks the camera button of the camera application, but also needs to obtain the collected video data after the user clicks the camera button of the camera application. The collected video data here can have a second time period, and the duration of the second time period can be the same as or different from the duration of the first time period of the preview video data. The collected video data can be used to record the change process of the object after the user clicks the camera button of the camera application.

[0069] The first feature identifier includes a first feature detection result of the preview video data, a second feature detection result of the collected video data, and a third feature detection result of the first preview screen. The second feature detection result may include a feature detection result obtained by the terminal device performing feature detection on the shooting content corresponding to each video frame included in the collected video data. The third feature detection result may include a feature detection result obtained by the terminal device performing feature detection on the shooting content corresponding to the first preview screen. The terminal device generates the preview video data, the collected video data, and the first preview screen as a dynamic photo and saves it in the photo album of the terminal device. The dynamic photo includes the first feature identifier. In some embodiments, the first feature identifier in the photo album of the terminal device may be represented by a tag of the dynamic photo.

[0070] After introducing the method flow of the multimedia data generation method, the following introduces the system architecture of the terminal device implementing the multimedia data generation method in the embodiment of the present application. Figure 5 , Figure 5The system architecture diagram for taking dynamic photos by a terminal device provided in an embodiment of the present application is exemplarily shown.

[0071] like Figure 5 As shown, the system architecture of the terminal device includes a hardware part and an operating system 500, wherein the operating system 500 is composed of an application layer, a system framework layer, and a hardware abstraction layer. The hardware part includes: a camera sensor 501, an image signal processor 502, and an image optimization module 503. The hardware abstraction layer of the operating system 500 includes: a video processing module 504, a semantic algorithm module 505, and an image processing module 506. The system framework layer includes: an electronic anti-shake module 507, a buffer pool 508, and the application layer includes: a camera application 509.

[0072] The photo sensor 501 is used to collect data of the photographed object, which may include preview data and video data.

[0073] The image signal processor 502 (Image Front End, IFE) is used to perform white balance correction, color correction and other processing on the data. The image data and video data are sent to the image optimization module 503, the video processing module 504 and the semantic algorithm module 505 respectively. The video data sent to the video processing module 504 here can be divided into preview data and video data, wherein the preview data is used for the application interface of the camera application to display the preview screen, and the video processing module 504 can perform skin beautification, anti-shake and other processing on the preview screen. The video data is used to cache the video for generating dynamic photos. The video data (Tiny stream) sent to the semantic algorithm module 505 is used by the semantic algorithm module 505 to perform semantic detection (video scene detection), and tags are added to the generated video according to the semantic detection results. When generating a short video (Vlog), you can choose to judge the use of tags.

[0074] The image optimization module 503 (Bayer Processing Segment, BPS) is used to perform bad pixel removal, phase focusing, demosaicing, down sampling, HDR processing and Bayer hybrid noise reduction processing on the image data.

[0075] The video processing module 504 is used to perform video optimization processing such as anti-shake, exposure, and mirroring on the video data.

[0076] The semantic algorithm module 505 (perception engine) is used to perform semantic detection on each video frame of the video data, that is, feature detection, and save the semantic detection results such as human body, face, action evaluation, etc. in the video frame.

[0077] The image processing module 506 is used to perform image processing tasks such as cropping, noise reduction, color processing, and detail enhancement on image data.

[0078] The electronic image stabilization module 507 (Electric Image Stabilization, EIS) is used to detect the amplitude of the terminal device's shaking through the acceleration sensor and gyroscope of the terminal device, and dynamically adjust the shutter, exposure and imaging to correct the video data and preview data to ensure clarity. The electronic image stabilization module 507 can send the preview data to the preview display module 5091 of the camera application 509 for previewing the shooting picture. The electronic image stabilization module 507 can also send the video data to the buffer pool 508.

[0079] Cache pool 508 is used to sequentially store video frames corresponding to video data. The video frames here can constitute a video of a certain length, for example: a 1.5s video (45 video frames). After cache pool 508 is full of 45 video frames, the video frames can be continuously updated to ensure that the latest video frames are stored in cache pool 508.

[0080] The preview display module 5091 of the camera application 509 is used to preview the shooting screen and display the shooting object. After receiving the user's shooting command, the video and image synthesis module 5092 of the camera application 509 can obtain the image (photo) taken when the shooting command is received through the image processing module 506, obtain the video data saved before receiving the shooting command and the video data collected after receiving the shooting command through the cache pool 508, obtain the semantic detection result corresponding to the video data through the semantic algorithm module 505, synthesize it into a dynamic photo, and save it in the album of the terminal device 500. The dynamic photo includes the mark corresponding to the semantic detection result.

[0081] In passing Figure 4 and Figure 5 After introducing the multimedia data generation method of the embodiment of the present application and the system architecture of the terminal device of the embodiment of the present application, the following is a Figure 6 The method flow of implementing the multimedia data generation method by each functional module in the system architecture of the terminal device is further described. The multimedia data generation method may include the following steps.

[0082] S601: The user turns on the camera.

[0083] Exemplarily, the opening of the camera here may be that the device in the terminal opens the camera application 601. The terminal device may receive an operation of the user clicking the icon of the camera application, and in response to the operation, the terminal device may execute an instruction to open the camera application 601.

[0084] S602: The user turns on the dynamic photo shooting mode.

[0085] For example, the camera application 601 may receive an operation of a user clicking on a dynamic photo shooting capability control, and in response to the operation, the camera application 601 turns on a dynamic photo shooting mode. It is understood that after the camera application 601 turns on the dynamic photo shooting mode, the application interface of the camera application 601 needs to present a preview of the object being shot at the current moment in real time, and in order to generate a dynamic photo, it is also necessary to obtain video data of a first time period from the current moment in real time.

[0086] S603: Camera HAL6031 returns to the preview screen.

[0087] For example, the camera HAL 6031 here can be a (Hardware Abstraction Layer) camera hardware abstraction. The preview stream here can be a video frame of the object being photographed at the current moment acquired by the camera HAL 6031 through the camera, which is returned to the camera application 601 as a preview screen in the form of a video stream, and the application interface of the camera application 601 can present a preview screen of the object being photographed at the current moment in real time.

[0088] S604: The camera HAL6031 caches the video stream into a cache pool.

[0089] Exemplarily, the buffer pool here can be a memory space (QueueBuffer) configured in the memory of the terminal device, which is used to cache the video stream. In some embodiments, the framework layer 602 can configure a buffer pool for HAL6031, and the buffer pool can also be called ION buffer. The camera HAL6031 can cache a video stream with a preset duration as preview video data, for example: 45 frames of video frames, that is, a video stream with a duration of 1.5s. The video stream here can be video data of the first time period before the current moment, and the camera HAL6031 also needs to continuously refresh the video stream in the buffer pool. The video frames corresponding to the video stream here can also be called cached frames.

[0090] S605: The semantic algorithm module 6032 obtains and caches the semantic detection result.

[0091] Exemplarily, the semantic detection result here may include feature detection results obtained by performing feature detection on the shooting content corresponding to each video frame of the video stream, including scene detection results, human body detection results, face detection results, and action evaluation results, which are used to add tags corresponding to the semantic detection results in the video frames, so that the dynamic photos generated according to the video frames also have tags, which is convenient for users to use dynamic photos to generate video blogs (Vlogs). In some embodiments, the semantic algorithm module 6032 can obtain semantic detection results from the video frames frame by frame, that is, when the camera HAL6031 caches a video frame of preview video data, the semantic algorithm module 6032 performs feature detection on the video frame to obtain feature detection results and caches them. For example: the camera HAL6031 caches the video streams corresponding to 90 frames of video frames in sequence, and the semantic algorithm module 6032 can cache the semantic detection results corresponding to 90 frames of video frames, and each video frame can correspond to a semantic detection result.

[0092] It can be seen that the semantic algorithm module 6032 performs feature detection on the video frame in real time, and can obtain the semantic detection result at the same time when the camera HAL 6031 caches the preview video data, thereby saving the time spent by the semantic algorithm module 6032 in performing feature detection.

[0093] In another embodiment, after the camera application 601 issues a command to take a photo, the semantic algorithm module 6032 may perform feature detection on each video frame of the preview video data cached by the camera HAL 6031 to obtain feature detection results and cache them. For example, if the camera HAL 6031 caches the video streams corresponding to 90 frames of video frames in sequence, the semantic algorithm module 6032 may perform feature detection on the cached 90 frames of video frames to obtain semantic detection results for the 90 frames of video frames.

[0094] It can be seen that if the camera application 601 does not issue a photo taking instruction for a long time, that is, the user uses the mobile phone to frame for a long time, the preview video data cached by the camera HAL 6031 will be constantly updated, resulting in the semantic detection result obtained by the semantic algorithm module 6032 being updated. After the camera application 601 issues a photo taking instruction, the semantic algorithm module 6032 performs feature detection on each video frame of the preview video data cached by the camera HAL 6031, which can save more power consumption.

[0095] S606: The user clicks to take a photo, and the camera application 601 sends a command to take a photo.

[0096] For example, the user clicking to take a photo here may be an operation in which the user clicks a photo button of the camera application 601 of the terminal device. In response to the operation, the camera application 601 may send a photo instruction to the camera HAL 6031. It is understood that the camera application 601 may record and save the time when the user clicks the photo button of the camera application 601 of the terminal device as the photo taking time point.

[0097] S607: Camera HAL6031 returns the target photo.

[0098] Exemplarily, the target photo here may be the target photo of the subject when the user clicks a photo button of the camera application 601 of the terminal device.

[0099] S608: The camera application 601 sends an instruction to obtain a video stream.

[0100] Exemplarily, the camera application 601 sends an instruction to obtain a video frame corresponding to a video stream to the camera HAL 6031 through the framework layer 602 .

[0101] For example, the camera application 601 can issue a command to obtain a video stream to the HAL 6031 through the framework layer 602 while issuing a command to take a photo, so as to obtain a video stream for generating a dynamic photo. It can be understood that the video stream here can include preview video data and acquisition video data.

[0102] S609 : The framework layer 602 returns the video stream to the camera application 601 .

[0103] Exemplarily, the framework layer 602 can perform video optimization processing on the video frame, and after processing such as video encoding, return a video stream (preview video data and collected video data) to the camera application 601, and the video frame in the video stream includes a corresponding frame number and a timestamp. The camera application 601 can save the video frames in sequence according to the frame number and timestamp corresponding to the video frame. In some embodiments, the camera application 601 can receive and cache the encoded video frames with a duration of 3s. The cache form of the video frame here can be an AVC (Advanced Video Coding) cache queue.

[0104] S609: The camera application 601 sends the photo taking time point and detection interval to the semantic algorithm module 6032.

[0105] Exemplarily, the photographing time point here can be used for the camera application 601 to send the semantic detection result to the semantic algorithm module 6032, so as to obtain the semantic detection result corresponding to the video of the preset length before and after the photographing time point. The interval here is used to represent the time period of the video determined according to the photographing time point. For example: the duration of the video can be 3s, then the interval can be an interval formed by 1.5s before and after the photographing time point. Exemplarily, the semantic algorithm module 6032 obtains the semantic detection result corresponding to the photographing time point from the cached semantic detection results according to the photographing time point.

[0106] S611 : The semantic algorithm module 6032 returns the semantic detection result to the camera application 601 .

[0107] Exemplarily, after the semantic algorithm module 6032 obtains the semantic detection result corresponding to the photographing time point, the semantic detection result is returned to the camera application 601. It can be understood that the semantic detection result here includes the semantic detection result corresponding to the preview video data and the collected video data.

[0108] S612: The camera application 601 synthesizes a dynamic photo.

[0109] Exemplarily, the camera application 601 can obtain the cached video frames from the AVC cache queue and synthesize them into a video file, such as an MP4 file. The camera application 601 can synthesize the target photo, the MP4 file, and the semantic detection result into a dynamic photo and save it to the album of the terminal device. The semantic detection result of the dynamic photo in the album of the terminal device can be represented by a mark on the dynamic photo, such as sports, scenery, childishness, etc.

[0110] In passing Figure 6 After introducing the method flow of implementing the multimedia data generation method by each functional module in the system architecture of the terminal device, the following is a schematic diagram of a process of selecting a dynamic photo to generate a short video through Figure 7 (a) and Figure 7 (b). The process of generating a short video can be executed by the processor of the terminal device, including the following steps. In Figure 7, the terminal device here can be a mobile phone.

[0111] S701: Open the album to display dynamic photos.

[0112] For example, the photo album here may also include a photo gallery of the mobile phone 100, and the operation of opening the photo album here may be as follows: Figure 2(a) to Figure 2(c)As shown, the mobile phone 100 receives the user's operation of opening the album of the mobile phone 100, and the application interface of the album displays multiple photos and the creation control 2001. The mobile phone 100 receives the user's click operation on the creation control 2001, and in response to the operation, the mobile phone 100 displays the creation interface. The creation interface displays multiple photos and the free creation control 2002. The mobile phone 100 receives the user's click operation on the free creation control 2002, and in response to the operation, the mobile phone 100 displays multiple dynamic photos at a glance through the creation interface.

[0113] S702: Detect whether the number of dynamic photos in the album exceeds a quantity threshold.

[0114] For example, the mobile phone 100 can detect the number of dynamic photos currently taken in the album. If the number exceeds the number threshold, it means that there are too many dynamic photos in the album. In the process of selecting dynamic photos to generate Vlog, the user needs to slide the application interface of the album to browse and select dynamic photos, then step S703 is executed to judge and recommend the dynamic photos in the album. Otherwise, step S704 is executed to determine the priority and classification of dynamic photos according to the tags of dynamic photos.

[0115] S703: judging and recommending dynamic photos in the album.

[0116] Exemplarily, step S703 may be composed of the following steps, as shown in FIG. 7( b ).

[0117] S703a: Determine the dynamic photos taken within a preset time period.

[0118] Exemplarily, the preset time period here may be pre-set, for example, the preset time period may be within the past three days. If the shooting time of the dynamic photo is earlier than the preset time period, the dynamic photo may be excluded, that is, the dynamic photo may be arranged at the back.

[0119] S703b: Determine the dynamic photos whose shooting time is a holiday or a preset holiday.

[0120] Exemplarily, the preset holiday here may be pre-set, for example, the preset holiday may be a wedding anniversary, birthday, etc. If the shooting time of the dynamic photo belongs to a holiday or a preset holiday, the dynamic photo may be retained, that is, the dynamic photo may be arranged in the front.

[0121] S703c: Collect information about the shooting location of the dynamic photo.

[0122] Exemplarily, the shooting location here may be the location information obtained when the user takes a dynamic photo.

[0123] S703d: De-duplication according to the tags of the dynamic photos.

[0124] For example, if there are multiple dynamic photos with the same tag, at least one dynamic photo can be retained to avoid the same dynamic photos being arranged adjacent to each other and occupying the application interface of the album.

[0125] S704: Determine the priority and classification of the dynamic photos according to the tags of the dynamic photos.

[0126] Exemplarily, the tag here is determined by the feature detection result of the video data (preview video data and acquisition video data) included in the dynamic photo when the mobile phone 100 generates the dynamic photo, that is, the semantic detection result, and the tag may include: childishness, travel, nature, sports, etc. It can be seen that a dynamic photo can include at least one tag, and a dynamic photo including multiple tags indicates that the dynamic photo includes multiple shooting contents.

[0127] In some embodiments, the classification here can be based on the tags of the dynamic photos, that is, the dynamic photos with the same tags are arranged adjacently and continuously. The priority of the dynamic photos may include: shooting time priority, shooting location priority, tag number priority, etc. Taking the shooting time priority as an example, the album of the mobile phone 100 can arrange the dynamic photos in descending order according to the shooting time, that is, the most recently shot dynamic photos are arranged at the front of the album.

[0128] S705: Determine the priority of generating Vlog recommendations based on the determined priorities and categories of the dynamic photos.

[0129] Exemplarily, the priority of the Vlog recommendation here can be pre-set, and the priority can be determined according to the priority and / or classification. For example, according to the shooting time priority of the dynamic photos, the priority of the Vlog recommendation corresponding to the dynamic photos marked as motion is set to the highest, that is, the dynamic photos taken most recently and marked as motion are arranged at the front of the album.

[0130] It can be understood that determining the priority of generating Vlog recommendations may also include various other judgment criteria.

[0131] S706: Prompt the user to generate a control for generating a Vlog through the photo album.

[0132] Exemplarily, the mobile phone 100 can prompt the user to generate a Vlog with one click through the generation control by highlighting or animating a local area on the application interface of the album (for example, below or above the application interface of the album, etc.). For example, as shown in FIG3(a), the mobile phone 100 receives the operation of the user opening the album of the mobile phone 100, and the application interface of the album displays multiple photos and the creation control 3001. The mobile phone 100 receives the user's click operation on the creation control 3001, and in response to the operation, as shown in FIG3(b), the mobile phone 100 displays the creation interface. The creation interface displays multiple photos and a one-key film control 3002. The mobile phone 100 receives the user's click operation on the one-key film control 3002, and in response to the operation, as shown in FIG3(c), the mobile phone 100 displays multiple recommended dynamic photos, images or videos at a glance through the creation interface. In some embodiments, the one-key film control 3002 can also be identified by the one-key blockbuster control 3002.

[0133] S707: Displaying a preset number of dynamic photos, images or videos for generating a Vlog and recommending them to the user according to the default recommendation rule.

[0134] Exemplarily, the default recommendation rules here may be the display and recommendation strategies of dynamic photos that the user has set in advance in the album. The default recommendation rules may be determined based on the shooting time, shooting location, tags, and priority of the dynamic photos. For example: the user may set the default recommendation rules to include: recommending dynamic photos that are shot recently and marked as childish. The preset recommended number here may be the number of dynamic photos for generating a Vlog at a time, for example: the preset recommended number may be 3. It will be understood that the numerical values ​​here are exemplary, and the embodiments of the present application do not impose any limitations. For example, the default recommendation rules may be landscapes and portraits; the preset recommended number may be 3, and the album of the mobile phone 100 may display preset dynamic photos recommended to the user. The user may directly generate a Vlog or adjust the recommended dynamic photos.

[0135] In some embodiments, the album of the mobile phone 100 may include a portion of other photos or videos in addition to recommending dynamic photos. The album of the mobile phone 100 may prioritize dynamic photos, other photos (images) or videos, and select dynamic photos, other photos or videos for generating Vlog by default, and recommend them to the user (checked). As shown in FIG3(c), the album of the mobile phone 100 recommends multiple dynamic photos, images and videos. The album of the mobile phone 100 also checks 3 dynamic photos by default. The local area 3003 on the application interface of the album prompts the user that the dynamic photos have been checked, and the user can directly generate Vlog. The user can also decide to increase or decrease the recommended dynamic photos, other photos or videos, or keep them unchanged. For example: increase or decrease dynamic photos, change the order of dynamic photos, etc. As shown in FIG3(d), the user removes the dynamic photos that are checked by default in the album of the mobile phone 100 and selects image 3005. At this time, the local area 3003 on the application interface of the album prompts that the dynamic photos that have been checked have changed, and image 3005 has been added.

[0136] S708: Receive the dynamic photo, video or image selected by the user.

[0137] For example, in addition to the dynamic photos displayed on the album application interface that can be used to generate Vlog materials, the user can also select other videos or images, that is, the user can add videos or images by himself. For example, referring to FIG3 (e), the mobile phone 100 detects the user's click operation on the directory selection control 3006. In response to the operation, as shown in FIG3 (f), the mobile phone 100 displays the video application interface, which includes multiple videos, and the user can select at least one video as the material of the dynamic photo.

[0138] S709: Determine whether the number of selected materials exceeds the upper limit and give a prompt.

[0139] Exemplarily, the upper limit of the number here may also be preset, for example: the upper limit of the number may be 5. If the number of materials selected by the user exceeds the upper limit of the number, the mobile phone 100 may prompt the user in the local area 3003 on the application interface of the album.

[0140] S710: Filter based on the priority of dynamic photos, videos, and images.

[0141] For example, the priority here can be arranged in the order of dynamic photos to videos to images, that is, dynamic photos have the highest priority, videos have the second highest priority, and images have the lowest priority. If the number of materials selected by the user exceeds the upper limit, the mobile phone 100 can retain the dynamic photos selected by the user and prioritize images and videos.

[0142] S711: Detecting the user's operation on the generation control, prompting the audio and Vlog template for generating the Vlog.

[0143] Exemplarily, continuing to refer to FIG3(d), the mobile phone 100 detects a click operation performed by the user on the one-key video control 3004, and as shown in FIG3(g), the mobile phone 100 may open a prompt interface for displaying the audio and Vlog template for generating the Vlog.

[0144] S712: Detecting the user's operation on the confirmation control, generating and saving a Vlog.

[0145] Exemplarily, with continued reference to FIG. 3( g ), the mobile phone 100 detects a click operation performed by the user on the confirmation control 3007 , and the mobile phone 100 generates a Vlog and can save the Vlog in a video directory of the mobile phone 100 .

[0146] Figure 8 800 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Figure 8 As shown, the electronic device (such as a mobile phone) may include: a processor 810, an external memory interface 820, an internal memory 821, a universal serial bus (USB) interface 830, a charging management module 840, a power management module 841, a battery 842, an antenna 1, an antenna 2, a mobile communication module 850, a wireless communication module 860, an audio module 870, a speaker 870A, a receiver 870B, a microphone 870C, an earphone interface 870D, a sensor module 880, a button 890, a motor 891, an indicator 892, a camera 893, a display screen 894, and a subscriber identification module (SIM) card interface 895, etc.

[0147] Among them, the above-mentioned sensor module 880 may include sensors such as a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor and an ambient light sensor.

[0148] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0149] The processor 810 may include one or more processing units, for example, the processor 810 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0150] The processor 810 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 810 is a cache memory. The memory may store instructions or data that the processor 810 has just used or cyclically used. If the processor 810 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 810, and thus improves the efficiency of the system.

[0151] In some embodiments, the processor 810 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0152] It is understandable that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation of the electronic device. In other embodiments, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0153] The external memory interface 820 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 810 through the external memory interface 820 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0154] The internal memory 821 may be used to store computer executable program codes, which may include instructions. The processor 810 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 821. For example, in an embodiment of the present application, the processor 810 may execute instructions stored in the internal memory 821, and the internal memory 821 may include a program storage area and a data storage area.

[0155] The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 821 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0156] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0157] Fig. 9 It is a software structure block diagram of the terminal device according to an embodiment of the present invention.

[0158] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer (APK), application framework layer (FWK), hardware abstraction layer (HAL) and kernel layer.

[0159] like Fig. 9 As shown, the application layer may include applications such as camera applications, etc. It is understood that the application here may be an application or service installed on the terminal device, or an application or service not installed on the terminal device retrieved by the quick service center.

[0160] The application layer includes a camera application, which includes a preview display module 901 and a dynamic photo management module 902. The preview display module 901 is used to preview the shooting screen and display the shooting object. The dynamic photo management module 902 is used to synthesize the shot target photo, MP4 file, semantic detection result and audio data into a dynamic photo.

[0161] The application framework layer includes a storage management module 903, which is used to allocate memory space for the camera HAL 904, that is, to configure a buffer pool for caching video data.

[0162] The hardware abstraction layer includes a camera HAL 904 , a semantic algorithm module 905 , an image processing front end 906 , an image processing engine 907 and an image processing module 908 .

[0163] The camera HAL 904 is used to control the camera of the terminal device to take photos or videos, cache video frames, and perform video optimization processing on the video frames. The semantic algorithm module 905 is used to detect semantic detection results such as human body, face, action evaluation, etc. from the video frames obtained by the camera of the terminal device.

[0164] The image processing front end 906 (Image Front End, IFE) is used to perform effect processing on the image frame, for example: performing some color correction, down sampling, de-mosaicing and statistical 3A data processing on the image frame.

[0165] Image processing engine 907 (IPE): used for performing image processing tasks such as cropping, noise reduction, color processing, and detail enhancement on image frames.

[0166] The image processing module 908 (Bayer Processing Segment, BPS) is used to perform bad pixel removal, phase focusing, demosaicing, downsampling, HDR processing and Bayer hybrid noise reduction processing on the image frame.

[0167] It should be understood that although the terms "first", "second", etc. may be used herein to describe various features, these features should not be limited by these terms. These terms are used only to distinguish and should not be understood as indicating or implying relative importance. For example, a first feature may be referred to as a second feature, and similarly a second feature may be referred to as a first feature without departing from the scope of the exemplary embodiments.

[0168] In addition, various operations will be described as multiple separate operations in a manner that is most helpful for understanding the illustrative embodiments; however, the order of description should not be interpreted as implying that these operations must rely on the order of description, and many of the operations can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can also be rearranged. When the described operations are completed, the process can be terminated, but there can also be additional operations not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0169] References in the specification to "one embodiment," "an embodiment," "an illustrative embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or property, but each embodiment may or may not necessarily include the particular feature, structure, or property. Moreover, these phrases are not necessarily directed to the same embodiment. In addition, when particular features are described in conjunction with a specific embodiment, the knowledge of those skilled in the art can affect the combination of these features with other embodiments, whether or not these embodiments are explicitly described.

[0170] Unless the context dictates otherwise, the terms "comprising," "having," and "including" are synonymous. The phrase "A / B" means "A or B." The phrase "A and / or B" means "(A), (B), or (A and B)."

[0171] As used herein, the term "module" may refer to, be part of, or include: a memory (shared, dedicated, or group) for running one or more software or firmware programs, an application-specific integrated circuit (ASIC), an electronic circuit and / or processor (shared, dedicated, or group), a combinational logic circuit, and / or other suitable components that provide the described functionality.

[0172] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order is not required. Instead, in some embodiments, these features may be described in a manner and / or order different from that shown in the illustrative drawings. In addition, the structural or method features included in a specific figure do not mean that all embodiments need to include such features. In some embodiments, these features may not be included, or these features may be combined with other features.

[0173] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the use of the technical solution of the present application is not limited to the various applications mentioned in the embodiments of the present patent. Various structures and variations can be easily implemented with reference to the technical solution of the present application to achieve the various beneficial effects mentioned herein. Various changes made within the knowledge of ordinary technicians in the field without departing from the purpose of the present application should all fall within the scope of the patent application.

Claims

1. A method for generating multimedia data, characterized in that: Applied to an electronic device with a photographing function, the method comprises: Display the shooting preview screen of the dynamic photo; Generate and store a first feature detection result of the preview video data; detecting a shooting instruction of a user, generating and storing a first dynamic photo and a first feature identifier of the first dynamic photo, wherein the first dynamic photo includes the preview video data, a first preview screen where the shooting instruction is detected, and collected video data collected after the shooting instruction is detected, and The first feature identifier includes the first feature detection result, the second feature detection result of the collected video data, and the third feature detection result of the first preview screen; The first dynamic photo and the first characteristic identifier of the first dynamic photo are stored in an album; Detecting an opening operation of the photo album, opening the photo album to display a plurality of dynamic photos; Determining priorities and classifications of the multiple dynamic photos according to the feature identifiers of the multiple dynamic photos; Determining a recommendation priority of generating short videos from the multiple dynamic photos according to the priorities and classifications of the multiple dynamic photos; Displaying a preset recommended number of dynamic photos in the order corresponding to the recommendation priority of generating short videos from the dynamic photos; Determining a dynamic photo selected by a user from the preset recommended number of dynamic photos; Generate short videos based on the dynamic photos selected by the user.

2. The method according to claim 1, characterized in that: The first feature detection result, the second feature detection result and the third feature detection result include: at least one of a scene detection result, a human body detection result, a face detection result and an action evaluation result.

3. The method according to claim 2, characterized in that The generating and storing the first feature detection result of the preview video data comprises: In the process of generating and storing the preview video data, feature detection is performed on each video frame included in the preview video data to obtain image features of each video frame, and the first feature detection result includes image features obtained after merging and deduplicating the image features of each video frame.

4. The method according to claim 2, characterized in that: The detecting of the shooting instruction of the user and generating and storing the first feature identifier of the first dynamic photo includes: After detecting the user's shooting instruction, feature detection is performed frame by frame on the video frames included in the preview video data to obtain the first feature detection result.

5. The method according to claim 2, characterized in that: The content of the first feature identification includes at least one of sports, travel, scenery, and childishness.

6. The method according to claim 5, characterized in that The first feature identifier is displayed on the first dynamic photo.

7. The method according to claim 2, characterized in that: The generating and storing the first feature identifier of the first dynamic photo includes: The first feature detection result, the second feature detection result, and the third feature detection result are combined and deduplicated to obtain the first feature identifier.

8. The method according to claim 1, characterized in that Also includes: In the storage display interface of the dynamic photo, the first dynamic photo is arranged adjacent to the stored second dynamic photo, wherein the second characteristic identifier of the second dynamic photo is the same as the first characteristic identifier of the first dynamic photo.

9. The method according to claim 8, characterized in that Also includes: In the storage and display interface of dynamic photos, the first dynamic photo is located before the third dynamic photo, wherein the first feature identifier of the first dynamic photo belongs to the feature identifier in the sorting condition, and the third feature identifier of the third dynamic photo does not belong to the feature identifier in the sorting condition.

10. The method according to claim 9, characterized in that Also includes: The user's short video generation operation is detected, and the dynamic photos selected by the user are generated into a short video.

11. An electronic device, characterized in that: include: a memory for storing instructions to be executed by one or more processors of the electronic device, and The processor is one of the processors of the electronic device, and is used to execute the multimedia data generation method according to any one of claims 1 to 10.

12. A computer program product, characterized in that include: A non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium contains a computer program code for executing the multimedia data generating method according to any one of claims 1 to 10.

Images