Media data generation method and electronic device

Abstract

The present application relates to the technical field of interaction. Provided in the embodiments are a media data generation method and an electronic device, which solve the problem of low degree of intelligence in creating media data. A specific solution comprises: displaying a first interface, the first interface displaying a preview stream acquired by a camera in real time, and the first interface further displaying a first material; in response to a sliding operation of a user with respect to the first material, displaying a second interface, the second interface displaying the preview stream acquired by the camera in real time, and the second interface further displaying a second material; in response to a first operation of the user instructing photographing, photographing a third material; and, in response to a second operation, displaying the second interface, the second interface displaying first spliced data consisting of the first material and the third material.

Description

Media data generation method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on October 27, 2023, with application number 202311418278.6 and invention name “A video production method and electronic equipment based on large models”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of terminal technology, and in particular to a method for generating media data and an electronic device. Background Art

[0003] With the advancement of technology, electronic devices (such as mobile phones, tablets, etc.) will be equipped with cameras to meet users' needs to record and share their lives anytime and anywhere.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a media data generation method and electronic device, which automatically creates media data containing reference materials and shooting materials after guiding the user to shoot materials according to reference materials, thereby reducing the difficulty of creating high-quality media data and improving the intelligence of the device.

[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, embodiments of the present application provide a method for generating media data, applied to an electronic device. The electronic device displays a first interface. The first interface is a shooting interface. The first interface displays a preview stream captured in real time by a camera and a first source material. The first source material may be a photo, video, or animated image recommended to the user for shooting reference.

[0008] The electronic device displays a second interface in response to a user sliding operation relative to the first material. The sliding operation relative to the first material may be an operation in which the sliding track is located in the display area of ​​the first material. The second interface is also an interface for shooting. The second interface displays a preview stream captured in real time by the camera and the second material. That is, compared to the first interface, the second interface mainly changes the display content of the reference material area, while other areas still display the preview stream captured in real time by the camera or other controls of the camera interface.

[0009] While the second interface is displayed, the electronic device captures a third clip in response to a first user operation instructing the user to capture the clip. In response to a second operation instructing the user to end the capture, the electronic device displays a third interface. The third interface displays first stitched data composed of the second and third clips.

[0010] In the above embodiment, during the shooting process, recommended reference materials can be displayed on the shooting interface, making it easier for users to reference these materials when shooting, reducing the difficulty of capturing high-quality footage. Furthermore, users can manually switch between the recommended reference materials in the shooting interface, enhancing shooting flexibility. Subsequently, stitching data containing the reference materials and the currently captured material is automatically generated, reducing the difficulty of creating high-quality media data and improving the intelligence of the device.

[0011] In one possible implementation, while the first interface is displayed, the electronic device responds to a user's instruction to shoot and shoots a new material corresponding to the first material. At this time, the third interface also displays spliced ​​data consisting of the first material and the new material corresponding to the first material.

[0012] In some embodiments, the first material and the second material are materials that meet one or more of the following first conditions: the shooting location is the same as the current location of the electronic device, or the distance is less than a preset distance threshold; the time interval between the shooting time and the current time point exceeds a preset time threshold.

[0013] In the above embodiment, by recommending reference materials that meet the first condition, users are guided to shoot materials that meet the theme of revisiting old places. This reduces the difficulty of producing media data for the theme of revisiting old places, improves user satisfaction with the produced spliced ​​data, and reduces the rework rate.

[0014] In some embodiments, if the electronic device stores material that meets a first condition, a first card is displayed. The first condition includes a combination of one or more of the following: the shooting location is the same as the current location of the electronic device, or the distance is less than a preset distance threshold; the time interval between the shooting time and the current time point exceeds a preset time threshold. The electronic device can detect a user operation on the first card. After detecting the user operation on the first card, the first interface is displayed.

[0015] In the above embodiment, the electronic device can promptly identify the opportunity to shoot materials that meet the theme of revisiting old places, promptly recommend users to shoot, and trigger creation, thereby improving the timeliness of the recommendation.

[0016] Exemplarily, in response to a user operating a first card, the electronic device displays an interactive window. The interactive window includes a first control and a first display bar, wherein the first display bar includes the first material and the second material. In response to the user clicking the first control, a first interface is displayed.

[0017] In some embodiments, after capturing the third material and before displaying the third interface, the electronic device displays a fourth interface in response to a third operation. The fourth interface is also an interface for capturing, and displays a preview stream captured in real time by the camera. The fourth interface also displays a fourth material, which is different from the second material. In response to the user's first operation instructing capture, a fifth material is captured. After capturing the fifth material, the third interface also includes second stitched data, which is composed of the fourth and fifth materials.

[0018] In the above embodiment, when the electronic device recommends multiple reference materials to the user, after the user takes a photo according to one reference material, the next reference material can be recommended to facilitate the user to refer to the next reference material and continue taking photos, thereby obtaining more usable materials and improving the human-computer interaction efficiency of shooting.

[0019] In some embodiments, in response to a fourth operation on the third interface, first media data is displayed, where the first media data is a video including the first spliced ​​data and the second spliced ​​data.

[0020] In some embodiments, the similarity evaluation value between the third material and the second material is higher than the similarity evaluation value between the third material and the fourth material, and the similarity evaluation value between the fifth material and the fourth material is higher than the similarity evaluation value between the fifth material and the second material. The similarity evaluation values ​​quantify the degree of similarity between the image content of different materials. The more similar the poses of the characters in the different materials are, the higher the corresponding similarity evaluation value. This ensures the image quality of the stitched data.

[0021] In some embodiments, after the interactive window is displayed, the user's first voice command is detected, and the sixth material is newly displayed in the first display bar, and the sixth material is also a material that meets the first condition; or, the user's second voice command is detected, and the fourth material in the first display bar is canceled; or, the user's third voice command is detected, and the fourth material in the first display bar is replaced with the sixth material.

[0022] In the above embodiment, before the user instructs to shoot, he or she can preview the reference materials recommended by the electronic device and can also adjust the reference materials through voice commands, thereby improving the human-computer interaction efficiency of the electronic device.

[0023] In some embodiments, after the interactive window is displayed, after the user's operation on the first display bar is detected, a fifth interface is displayed, the fifth interface including the first material, the second material, the fourth material and the sixth material. In the fifth interface, a selected mark is displayed relative to the first material, the second material and the fourth material, and an unselected mark is displayed relative to the sixth material, and the sixth material is also a material that meets the first condition; a selection preview box is displayed on the fifth interface, and the selection preview box includes thumbnails of the first material, the second material and the fourth material; in response to the user's operation on the sixth material, the unselected mark displayed relative to the sixth material is changed to the selected mark; in the selection preview box, a thumbnail of the sixth material is newly displayed; in response to the user's confirmation operation in the selection preview box, the fifth interface is canceled, and the interactive window is displayed again, and the sixth material is newly displayed on the first display bar of the interactive window.

[0024] In other embodiments, after the interactive window is displayed, after the user's operation on the first display bar is detected, a fifth interface is displayed, the fifth interface including the first material, the second material, and the fourth material, and in the fifth interface, a selection mark is displayed relative to the first material, the second material, and the fourth material; a selection preview box is displayed on the fifth interface, the selection preview box including thumbnails of the first material, the second material, and the fourth material; in response to the user's operation on the thumbnail of the fourth material in the selection preview box, the selection mark displayed relative to the fourth material is changed to the unselected mark; in the selection preview box, the thumbnail of the fourth material is canceled; in response to the user's confirmation operation in the selection preview box, the fifth interface is canceled, the interactive window is displayed again, and the fourth material is canceled from the first display bar of the interactive window.

[0025] In the above embodiment, the user can manually adjust the recommended reference material so that the user can subsequently shoot satisfactory material.

[0026] In some embodiments, the aesthetic scores of the first material, the second material, and the fourth material are higher than the aesthetic scores of other materials in the electronic device that meet the first condition, thereby ensuring the quality of the recommended reference material.

[0027] In some embodiments, after capturing the third material, the electronic device may display a sixth interface, which is a preview interface of the third material; the sixth interface includes a second control; and displaying the third interface in response to a second operation includes displaying the third interface in response to a user operation on the second control. This allows the user to terminate the capture process early even if they haven't referenced all reference materials, improving the flexibility of capture.

[0028] In some embodiments, after shooting the third material, the electronic device displays a sixth interface, which is a preview interface of the third material; the sixth interface includes a third control; in response to the user's operation of the third control, a fourth interface is displayed, and the fourth interface displays a preview stream collected in real time by the camera. The fourth interface also displays the fourth material, which is different from the second material.

[0029] In a second aspect, an embodiment of the present application provides a method for generating media data, which is applied to an electronic device, the electronic device including a camera, the method including: displaying a first card when a material satisfying a first condition is stored in the electronic device, wherein the first condition includes a combination of one or more of the following: the shooting position is the same as the current position of the electronic device, or the distance is less than a preset distance threshold; the time interval between the shooting time and the current time point exceeds a preset time threshold; after detecting a user operation on the first card, displaying an interactive window; the interactive window including a first control and a first display bar, the first display bar including a first material and a second material, the first material and the second material being materials satisfying the first condition; in response to the user The operation of the first control displays the seventh interface, which displays a preview stream captured by the camera in real time; in response to the first operation of the user instructing to shoot, the seventh material is shot; in response to the second operation, the interactive window is displayed again, and a second display bar is newly displayed on the interactive window, and the seventh material is displayed on the second display bar; in response to the user's fifth operation, the third spliced ​​data is newly displayed on the interactive window; the third spliced ​​data is composed of the seventh material and the second material, and the similarity evaluation value between the seventh material and the second material is higher than the similarity evaluation value between the seventh material and the first material; wherein the above-mentioned similarity evaluation value is a quantitative value of the similarity of the picture content between different materials, and the more similar the character postures in different materials are, the higher the corresponding similarity evaluation value is.

[0030] In some embodiments, the interactive window also includes a fourth control, which is displayed relative to the first display bar. The method also includes: in response to the user's operation of the fourth control, displaying second media data on the interactive window, wherein the second media data includes the first material and the second material.

[0031] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, the memory being used to store code instructions; the processor being used to run the code instructions so that the electronic device executes the method described in the first aspect, the second aspect, and any one of their implementations.

[0032] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed, the computer executes the method described in the first aspect, the second aspect and any one of their implementations.

[0033] In a fifth aspect, a computer program product comprises a computer program, which, when executed, enables a computer to execute the method as described in the first aspect, the sixth aspect and any one of their implementations.

[0034] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] FIG1 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0036] FIG2 is a flow chart of a method for generating media data provided in an embodiment of the present application;

[0037] FIG3 is a diagram illustrating an example of a scenario for recommending target materials for shooting related to the theme of revisiting an old place, provided by an embodiment of the present application;

[0038] FIG4A is an example diagram of a scene in which different reference materials are switched during the process of shooting a target material, provided by an embodiment of the present application;

[0039] FIG4B is an example diagram of a scene in which different shooting modes are switched during the shooting of a target material according to an embodiment of the present application;

[0040] FIG5 is one of the example diagrams of a scene for shooting and saving target material provided by an embodiment of the present application:

[0041] FIG6A is a second example diagram of a scene for shooting and saving target material provided by an embodiment of the present application:

[0042] FIG6B is one of the exemplary diagrams of the method of splicing videos provided in an embodiment of the present application;

[0043] FIG6C is a second example diagram of a method for stitching videos provided in an embodiment of the present application;

[0044] FIG7 is a diagram illustrating an example of a scenario in which reference materials are added through voice commands according to an embodiment of the present application;

[0045] FIG8 is a diagram illustrating an example of a scenario in which reference materials are reduced through voice commands according to an embodiment of the present application;

[0046] FIG9 is one of the example diagrams of a scenario in which a user manually reduces reference materials according to an embodiment of the present application;

[0047] FIG10 is a second example diagram of a scenario in which a user manually reduces reference materials according to an embodiment of the present application;

[0048] FIG11 is a diagram illustrating an example of a scenario in which a user manually adds reference materials according to an embodiment of the present application;

[0049] FIG12 is a third example diagram of a scene for shooting and saving target material provided by an embodiment of the present application:

[0050] FIG13 is one of the example diagrams of a scenario for creating media data provided in an embodiment of the present application;

[0051] FIG14 is a second example diagram of a scenario for creating media data according to an embodiment of the present application;

[0052] FIG15 is an example diagram of a scene for creating a puzzle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0054] At the same time, in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner to facilitate understanding.

[0055] In the technical solutions disclosed in this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0056] With the advancement of terminal technology, users are increasingly using electronic devices. To meet users' needs to record and share their lives anytime, anywhere, most electronic devices are equipped with cameras. This allows users to capture footage, such as videos and photos, anytime, anywhere. Furthermore, electronic devices can respond to user actions and generate corresponding media data based on the captured footage.

[0057] Most users lack experience in creating media data, and often have difficulty figuring out when and what footage to shoot, as well as how to create media data based on the footage they have shot. This results in low usage of the media data creation functions provided by electronic devices.

[0058] To address the aforementioned issues, embodiments of the present application provide a method for generating media data that can be applied to electronic devices including cameras. After enabling this method, in response to identifying a target scene, the electronic device can guide the user to capture one or more target materials. The target scene is suitable for capturing materials that conform to a preset theme. Media data that conforms to the preset theme is then generated based on the captured one or more target materials.

[0059] In the above embodiment, the electronic device can reduce the difficulty of creating media data and improve the user's satisfaction with the produced media data by guiding the user to shoot materials at the right time and produce media data that conforms to the preset theme. This further improves the utilization rate of the media data creation function and increases the intelligence level of the electronic device.

[0060] For example, the preset themes may include various types of themes, such as a revisiting theme, a party theme, a check-in theme at a popular place, etc. Different preset themes may correspond to different target scenarios.

[0061] It is understandable that the electronic device can distinguish different target scenes through characteristics of the time dimension and / or characteristics of the space dimension.

[0062] For example, if it is detected that the electronic device is located in a place (eg, a tourist spot) where material has been shot before, the electronic device determines that the current scene is a target scene that meets (matches) the theme of revisiting an old place.

[0063] For another example, when it is detected that the system time has reached the party time, or when it is detected that the electronic device is located at a place for holding parties (eg, a restaurant), the electronic device determines that the current scene is a target scene that matches the party theme.

[0064] For example, when the electronic device detects that it is located at a popular spot, the electronic device determines that the current scene is a target scene that meets the theme of checking in at a popular spot. The popular spot can be a location where many people have taken photos and checked in, or a location where popular footage has been filmed.

[0065] For example, the electronic device can count the number of users who have filmed footage at the current location, and when the number of users exceeds a preset threshold, the current location is determined to be a popular online destination. Alternatively, the electronic device can obtain traffic information (number of visits, number of likes, or number of shares / comments, etc.) of media data with the same or similar shooting address as the current location (the distance is less than a preset distance threshold), and when the traffic information of any media data meets a preset condition (for example, the number of visits is greater than a preset visit threshold), the current location is determined to be a popular online destination.

[0066] It should be noted that the media data includes videos or images. The embodiment of the present application does not limit the type of media data generated by the electronic device. In addition, the above-mentioned material can be understood as photos or videos that constitute the media data.

[0067] For example, if photos A, B, and video C are collected and edited to produce video D, then photos A, B, and C are all sources, while video D is the generated media data. For another example, if photos E are collected and edited to produce video F, then photos E is the source material, and video F is the generated media data.

[0068] The above-mentioned electronic device may also be referred to as a terminal, terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), etc.

[0069] For example, the electronic device may be a mobile phone, tablet computer, wearable device, smart screen, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), or other electronic device with a display and camera; it may also be a vehicle-mounted device with a display and camera, such as a car computer or vehicle-mounted computer; it may also be some IoT devices with a display and camera, such as smart watches and smart bracelets. The embodiments of the present application do not impose any restrictions on the product form of the electronic device.

[0070] Next, the hardware structure and software architecture of the electronic device provided in the embodiments of the present application are introduced.

[0071] FIG1 shows a schematic diagram of the hardware structure of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, a display 194, an audio module 170, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a touch sensor 180K, etc.; and the audio module 170 may include a speaker 170A, a receiver 170B, a microphone 170C, an earphone jack 170D, etc.

[0072] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0073] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0074] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0075] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0076] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), and the like.

[0077] In some embodiments, the processor 110 and the camera 193 communicate via a CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via a DSI interface to implement the display function of the electronic device 100.

[0078] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0079] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0080] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0081] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0082] The pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor 180A can be provided on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a device comprising at least two parallel plates having a conductive material. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the display screen 194, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 180A.

[0083] In some embodiments, touch operations applied to the same touch location but with different touch operation intensities may correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than a first pressure threshold is applied to a short message application icon, an instruction to view short messages is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold is applied to a short message application icon, an instruction to create a new short message is executed.

[0084] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0085] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0086] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0087] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0088] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0089] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0090] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0091] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0092] For example, the electronic device 100 may obtain a wake-up word through the microphone 170C, and in response to obtaining the wake-up word, the electronic device 100 activates the voice assistant function.

[0093] Among them, the voice assistant function of the electronic device can be understood as the electronic device 100 collecting the user's voice through the microphone, analyzing and recognizing the user's voice, and executing the instructions corresponding to the user's voice, so that the user can control the electronic device 100 through voice.

[0094] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 , and antenna 2 is coupled to wireless communication module 160 , so that electronic device 100 can communicate with other devices through wireless communication technology and network.

[0095] For example, the electronic device 100 may obtain materials collected by other devices through the antenna 1 and the mobile communication module 150 , or through the antenna 2 and the wireless communication module 160 , and generate corresponding media data by the electronic device 100 .

[0096] The following takes the electronic device being a mobile phone, which has the hardware structure shown in FIG. 1 as an example, and describes the method provided in the embodiment of the present application in conjunction with the accompanying drawings.

[0097] For example, as shown in FIG2 , the method may include the following steps:

[0098] S101 , in response to the current scene being consistent with a preset theme, the mobile phone guides the user to shoot one or more target materials.

[0099] The preset themes may include a revisiting theme, a party theme, a check-in theme at a popular place, etc. In the following embodiments, description will be made by taking the revisiting theme as an example.

[0100] In some embodiments, the mobile phone can use a pre-set artificial intelligence (AI) model to identify whether the current scene is consistent with a preset theme. For example, the mobile phone can use the AI ​​model to determine whether the current scene is a target scene that is consistent with the "revisiting old places" theme based on the current location information collected in real time by the mobile phone and historical materials stored in the mobile phone. This can also be referred to as the current scene being consistent with the "revisiting old places" theme.

[0101] The stored historical materials may be videos, images and other data shot by the local device, or videos, images and other data from other devices. The above historical materials may correspond to basic information. The above basic information includes the shooting time and shooting address (such as latitude and longitude information or place name). When the shooting address of at least one historical material is the same as or close to the current location information collected (the distance is less than the preset distance threshold), it is determined that the current scene meets the theme of revisiting the old place.

[0102] For example, on January 1, 2023, the mobile phone responds to a user operation and takes and stores photo A at scenic spot A in city A. Correspondingly, the basic information corresponding to photo A includes the shooting time "January 1, 2023" and the shooting address "scenic spot A in city A".

[0103] On January 3, 2024, in response to the phone being located in City A or Scenic Spot A in City A, the phone searches through the historical footage stored in the phone. If Photo A is found, it is determined that Photo A was taken at or near the current location. Based on this information, the AI ​​model can determine that the current scene aligns with the theme of revisiting an old place.

[0104] For example, the mobile phone can use an artificial intelligence model to identify that the current scene is a target scene that matches the party theme based on the current location information collected by the mobile phone in real time and / or pre-configured itinerary information, such as the current scene is consistent with the party theme.

[0105] For example, the pre-configured itinerary in the phone indicates that you will attend Xiao Ming's birthday party at 2:00 PM on January 3, 2024. When the system time reaches 2:00 PM on January 3, 2024, the AI ​​model in the phone can recognize that the current scene matches the party theme.

[0106] For example, a user enters Restaurant A with their phone. After the phone collects their current location information, the AI ​​model uses this information to identify Restaurant A and searches for the restaurant's tag. If the restaurant's tag contains party-related content, the AI ​​model in the phone can recognize that the current scene aligns with the party theme.

[0107] As another example, a mobile phone can use an AI model to access a cloud database to query the number of users who have filmed footage at or near the current location. If the number of users exceeds a preset threshold, the AI ​​model can identify the current scene as a target scene that matches the theme of a popular check-in spot. This can also be referred to as the current scene being consistent with the theme of a popular check-in spot.

[0108] Alternatively, the phone can use an AI model to access a cloud database and search for media data that has been publicly released by the user and has the same or similar shooting address as the current location. If the traffic information (number of visits, likes, or reposts / comments) of the found media data meets preset conditions (for example, the number of visits is greater than a preset visit threshold, the number of likes exceeds a preset like threshold, or the number of reposts / comments is greater than a preset number), the AI ​​model can identify that the current scene matches the theme of the popular check-in location.

[0109] The above is only an example. In other embodiments, the artificial intelligence model can also evaluate whether the current scenario conforms to the preset theme according to other rules. The embodiments of the present application do not specifically limit this.

[0110] In some embodiments, if the current scene matches the theme of revisiting an old place, the phone can display a function entry for taking a photo and checking in. The display form of the function entry can be a card, notification, floating capsule, or status capsule, etc., which is not specifically limited in this embodiment of the application. The function entry can be displayed on the phone's desktop, the negative first screen, the notification bar, etc.

[0111] For example, the above-mentioned function entry is used to make the photo-taking and check-in function available to the user. In response to the user's operation of the above-mentioned function entry, the mobile phone can display guidance information. The guidance information can guide the user to use the mobile phone to shoot target materials that meet the theme of revisiting the old place and take a photo and check in at the current location.

[0112] As shown in FIG3 , while the mobile phone is displaying the main interface 301 (desktop), in response to the current scene being consistent with the theme of revisiting old places, a function card 302, such as the first card, is displayed on the main interface 301. In some embodiments, the function card 302 displays content related to the theme of revisiting old places, such as a place name 303, reference materials 304, and a theme name 305.

[0113] Illustratively, the place name 303 displayed on the function card 302 is the place name corresponding to the current location information.

[0114] As another example, the reference material 304 comes from historical materials that meet condition 1.

[0115] It can be understood that the reference material is a sample material recommended by the mobile phone to the user for reference. As a sample, the reference material can be used to guide the user to shoot target material that meets the theme of revisiting an old place.

[0116] In addition, the above-mentioned condition 1 may be a pre-configured condition related to the theme of revisiting an old place, which is called the first condition. For example, the above-mentioned condition 1 may include that the shooting address is the same as or close to the current location information (the distance is less than a preset distance threshold). For another example, the above-mentioned condition 1 may include a combination of one or more of the following: the shooting address is the same as or close to the current location information, the shooting time is not on the same day (the time interval exceeds the preset time threshold), the person appearing in the material is not the owner of the camera himself (for example, a celebrity), etc.

[0117] It is understandable that historical materials that meet the theme of revisiting an old place can be preliminarily screened out through condition 1. In this way, from the historical materials that meet condition 1, the mobile phone can select reference materials related to the theme of revisiting an old place.

[0118] As an implementation method, the reference material can be historical materials that meet condition 1 and are selected according to aesthetic scores.

[0119] It is understood that the above-mentioned aesthetic score can be quantitative information about the aesthetic quality of the image data. For example, the above-mentioned aesthetic score can be an artificial intelligence module that comprehensively evaluates the score corresponding to the historical material from dimensions such as underlying aesthetic features (color, hue, brightness, contrast, etc.) and photographic aesthetic features (spatial composition features, foreground and background features, etc.). As an implementation method, after obtaining any material, the mobile phone can use the artificial intelligence model to determine the aesthetic score corresponding to the material and associate the aesthetic score with the material.

[0120] In some embodiments, after the mobile phone finds historical materials that meet condition 1, it can sort the historical materials that meet condition 1 in descending order based on their aesthetic scores. If the number of historical materials ranked in the top N positions is equal to N, the historical materials ranked in the top N positions are used as reference materials. If the number of historical materials ranked in the top N positions is greater than N, the historical materials ranked in the top N positions can be deduplicated. After deduplication, historical materials that primarily feature people and those with high aesthetic scores are prioritized as reference materials.

[0121] The deduplication process described above can be a means of removing duplicate material. For example, if historical material A and historical material B both meet condition 1, the similarity between them is greater than or equal to a preset similarity threshold, and the aesthetic score of historical material A is greater than that of historical material B, then after deduplication, only historical material A will be retained. It is understood that the similarity described above is a quantitative measure of the degree of visual similarity between the materials, which can be referred to as a similarity assessment value.

[0122] In other embodiments, after the mobile phone finds historical materials that meet condition 1, it can first remove duplicates from the historical materials that meet condition 1. Then, the removed duplicates are sorted in descending order according to their aesthetic scores. If the number of historical materials ranked in the top N positions is equal to N, the historical materials ranked in the top N positions are used as reference materials. If the number of historical materials ranked in the top N positions is greater than N, historical materials that primarily feature people and historical materials with high aesthetic scores are preferentially selected as reference materials.

[0123] In some examples, when N is equal to 1, reference material 304 is the material with the highest aesthetic score among the historical materials that meet condition 1, and the mobile phone can display this reference material 304 on function card 302. When N is greater than 1, for example, N is equal to 3, reference material 304 includes the materials ranked in the top three aesthetic scores among the historical materials, and the mobile phone can display these three reference materials 304 in a carousel on function card 302.

[0124] In a possible embodiment, when the mobile phone selects three reference materials, only the reference material with the highest aesthetic score can be displayed on the function card 302. In addition, in response to the user sliding operation on the function card 302, other reference materials can be switched to be displayed on the function card 302.

[0125] As another example, the theme name 305 displayed on the function card 302 can be the name of a preset theme that matches the current scene, that is, the characters "Revisiting an Old Place". It is understood that while the phone is displaying other user interfaces (the negative first screen, notification center, lock screen, etc.), in response to the current scene matching the theme of "Revisiting an Old Place", a function entry indicating taking a photo and checking in can also be displayed on other user interfaces to ensure that the photo and check-in function can reach the user in a timely manner.

[0126] In addition, the function card 302 also includes a control 306. In response to the user's operation on the control 306, such as a click operation, the mobile phone can display an interactive window 307, and the user can operate the mobile phone to shoot the target material according to the prompt information in the interactive window 307.

[0127] As shown in FIG3 , the interactive window 307 includes a shooting prompt 308 , reference materials 309 , a check-in control 310 , a voice input control 311 , and a keyboard input control 312 .

[0128] In some embodiments, the interactive window 307 may be in the form of a dialogue, displaying guidance reminder information from the mobile phone (e.g., shooting prompt 308) and instruction content from the user (not shown in FIG3 , refer to the recognition result 702 in the subsequent embodiments). Correspondingly, the dialogue control 313 of the interactive window 307 is selected.

[0129] In some embodiments, the shooting prompt 308 is used to recommend the user to shoot at the current location, for example, "select the check-in photos you took here three years ago, and use the same posture to take a photo and check in~".

[0130] The reference material 309 includes photos 309-1, 309-2, and 309-3, which are displayed in the first display column. In some embodiments, the photos 309-1, 309-2, and 309-3 are only examples of reference materials selected by the mobile phone, and may include more or fewer reference materials, or reference materials of other themes.

[0131] In some embodiments, both the voice input control 311 and the keyboard input control 312 are used to assist the user in issuing commands to the mobile phone. The difference is that the user can issue voice commands to the mobile phone through the voice input control 311, and the mobile phone can obtain the voice commands by recognizing the content of the user's voice input. The user clicks the keyboard input control 312, and the mobile phone can display a keyboard on the interface. In response to the user's keyboard operation, the mobile phone can obtain text commands.

[0132] Take the example of a user issuing a command through the voice input control 311. In response to the user clicking the voice input control 311, the mobile phone can wake up the voice assistant function. Of course, if the user speaks a pre-configured keyword (a pre-set phrase, such as "hello"), the mobile phone can also wake up the voice assistant function. The embodiment of the present application does not specifically limit the method for waking up the voice assistant function.

[0133] After the voice assistant function is activated, users can instruct the phone to perform certain functions through voice commands.

[0134] For example, a mobile phone can detect the audio signal being transmitted in the environment. After detecting the audio signal (including the voice command spoken by the user), the mobile phone recognizes and analyzes the audio signal to obtain corresponding recognition results and analysis results.

[0135] The recognition result may be the text content of the voice command, and the mobile phone may display the recognition result in the interactive window 307 to prompt the user of the command issued to the mobile phone. The analysis result includes the function indicated by the semantics of the voice command, and the mobile phone may execute the function indicated by the voice command according to the analysis result.

[0136] For example, a user might say the voice command "take a photo and check in" to their phone. The phone can activate the voice assistant function, which then collects audio signals, such as those containing the phrase "take a photo and check in." The phone can then use an AI model to identify and analyze the collected audio signals.

[0137] Based on the above audio signal, the recognition result of the voice command spoken by the user can be obtained as the text "take a photo and check in". Based on the above audio signal, the semantics that can be parsed are associated with the shooting function. In addition, combined with the fact that the reference material recommended to the user last time is related to the "revisit the old place theme", the mobile phone can determine that the parsing result is a voice command indicating the shooting function corresponding to the "revisit the old place theme". Correspondingly, the mobile phone can start the shooting function corresponding to the "revisit the old place theme". After enabling the shooting function corresponding to the "revisit the old place theme", the mobile phone can guide the user to shoot the target material that meets the "revisit the old place theme". For example, after starting the shooting function corresponding to the "revisit the old place theme", the mobile phone can display the shooting interface 314. For specific details, please refer to the description in the subsequent embodiments.

[0138] It is understood that other preset themes may also have corresponding dedicated shooting functions. Dedicated shooting functions can also guide users to shoot target materials suitable for other preset themes. When the mobile phone activates the dedicated functions of other preset themes, it can also display the shooting interface corresponding to other preset themes.

[0139] Let's take the example of a user issuing an instruction through the keyboard input control 312. In response to the user's operation on the keyboard input control 312, the mobile phone can input an instruction in text form, referred to as a text instruction. For example, the user enters the text "take a photo and check in", and the mobile phone can determine that the user has issued the text instruction "take a photo and check in". Then, the mobile phone parses the above text instruction and obtains the corresponding parsing result. For example, it is determined that the semantics of the above text instruction is related to the shooting function. In addition, combined with the fact that the reference material recommended to the user most recently is related to the "revisit the old place theme", the mobile phone can determine that the parsing result is that the text instruction indicates the shooting function corresponding to the "revisit the old place theme". Correspondingly, the mobile phone can activate the shooting function corresponding to the "revisit the old place theme" and display the shooting interface 314.

[0140] The above is only an example of a user issuing a "photo and check-in" command to the mobile phone through the voice input control 311 and the keyboard input control 312. The user can also issue other commands to the mobile phone through the voice input control 311 and the keyboard input control 312, such as commands for adding or deleting reference materials, commands for generating media data, etc. For details, please refer to the subsequent embodiments and will not be repeated here.

[0141] In some embodiments, the check-in control 310 is an interface for launching the photography function corresponding to the "Revisiting Old Places" theme, and may also be referred to as a first control. As shown in FIG3 , in response to a user clicking the check-in control 310 , the mobile phone may launch the photography function corresponding to the "Revisiting Old Places" theme and display a photography interface 314 .

[0142] The mobile phone can display the shooting interface 314 in response to the user clicking the check-in control 310, in response to detecting the voice command "check-in and take a photo", or in response to receiving the text command "check-in and take a photo".

[0143] As shown in Figure 3, the shooting interface 314 includes a capture preview area 315 and a shooting operation area 316. The capture preview area 315 is used to present the preview stream captured by the camera in real time, wherein the preview stream consists of image frames continuously captured by the camera.

[0144] As shown in FIG3 , a reference material is displayed on the capture preview area 315. The reference material is displayed on the capture preview area 315 in the form of a picture-in-picture or a small window. The position of the reference material on the capture preview area 315 is not specifically limited. It can be the upper area as shown in the figure, or it can be located in a corner area, etc. When the reference material includes multiple, for example, including photo 309-1, photo 309-2 and photo 309-3, one of the photos 309-1, 309-2 and 309-3 can be displayed on the capture preview area 315. As shown in FIG3 , photo 309-1 is displayed. Among them, the shooting interface 314 displaying photo 309-1 (such as referred to as the first material) can be called the first interface.

[0145] In addition, the mobile phone displays navigation points 319 relative to the photo 309-1, wherein the number of navigation points 319 is the same as the number of reference materials, and each navigation point 319 corresponds to a reference material. As shown in FIG3 , when the photo 309-1 is displayed in the capture preview area 315, the navigation point corresponding to the photo 309-1 is in a selected state.

[0146] In some embodiments, in response to a user clicking on a reference material in the capture preview area 315, the mobile phone may launch a gallery application and display the clicked reference material in an application interface provided by the gallery application. In some embodiments, in response to a user clicking on a reference material in the capture preview area 315, the mobile phone may display a new mask and display the clicked reference material on the mask.

[0147] In addition, as shown in FIG4A , in response to a user sliding a photo 309-1 on the capture preview area 315, the mobile phone can switch to displaying photo 309-2 on the capture preview area 315. Correspondingly, the navigation point corresponding to photo 309-1 is deselected, and the navigation point corresponding to photo 309-2 is selected. The shooting interface 314 displaying photo 309-2 (e.g., the second material) can be referred to as the second interface.

[0148] The shooting operation area 316 displays a control 317 for indicating shooting and configuration items 318 for indicating shooting modes. Configuration items 318 include a configuration item 318-1 for indicating portrait mode, a configuration item 318-2 for indicating photo mode, and a configuration item 318-3 for indicating video mode. It is understood that different shooting modes may support different cameras and require different camera parameters. Furthermore, the above shooting mode types are merely examples and may include other shooting modes, which are not specifically limited here.

[0149] In some embodiments, if it is predetermined that all materials related to the theme of revisiting an old place are photos. Correspondingly, the selected reference materials are also photos. In this example, after the mobile phone activates the shooting function related to the theme of revisiting an old place, in the displayed shooting interface 314, the configuration item 318-1 indicating the portrait mode is selected, and the mobile phone can capture the preview stream in portrait mode. Alternatively, after the mobile phone activates the shooting function related to the theme of revisiting an old place, in the displayed shooting interface 314, the configuration item 318-2 indicating the photo mode is selected, and the mobile phone captures the preview stream in the photo mode. This embodiment of the application does not specifically limit this.

[0150] For example, when the shooting function related to the theme of revisiting an old place is activated, the portrait mode is enabled by default. Correspondingly, the above configuration item 318-1 is in a selected state. When the portrait mode is enabled, the mobile phone displays the preview stream collected in real time according to the portrait mode in the collection preview area 315. It can be understood that the default activation of the portrait mode is only an example. In other embodiments, the photo mode can also be enabled by default, and this is not specifically limited.

[0151] For another example, when the reference material contains the largest number of photos mainly of people, when starting a shooting function related to the theme of revisiting an old place, the portrait mode is enabled, and accordingly, the configuration item 318 - 1 is in a selected state.

[0152] For example, if the reference material contains the largest number of photos of objects or scenery, and the shooting function related to the theme of revisiting old places is activated, the photo mode is enabled, and accordingly, the configuration item 318-2 is selected. When the photo mode is enabled, the mobile phone displays the preview stream collected in real time according to the photo mode in the collection preview area 315.

[0153] For another example, if the reference material displayed on the shooting interface 314 is a photo mainly of people, the configuration item 318-1 is selected. If the reference material displayed on the shooting interface 314 is a photo mainly of objects or scenery, the configuration item 318-2 is selected.

[0154] As an implementation, before displaying the shooting interface 314, the mobile phone may determine the order in which the reference materials are recommended to the user (e.g., referred to as the recommendation order). For example, the mobile phone may arrange the recommendation order of the reference materials from near to far based on the distance between the shooting address of the reference material and the current location. For another example, the mobile phone may arrange the recommendation order of the reference materials from far to near based on the shooting time of the reference material. For another example, the mobile phone may arrange the recommendation order of the reference materials from high to low based on the aesthetic score of the reference material.

[0155] After the shooting interface 314 is displayed, reference material a is displayed on the shooting interface 314. Reference material a is the first reference material in the recommended order. If reference material a is a photo primarily of a person, configuration item 318-1 is selected. If reference material a is a photo primarily of a person, an object, or a scene, configuration item 318-2 is selected.

[0156] In addition, the user can also manually switch the shooting mode. As shown in FIG4B , when the configuration item 318 - 1 in the shooting interface 314 is selected, the mobile phone can switch the portrait mode to the photo mode in response to the user clicking the configuration item 318 - 2 .

[0157] It is understandable that different modes may support different cameras and / or require different camera parameters. The process of switching from portrait mode to photo mode may include changing the enabled camera and / or changing the camera parameters, etc. The specific details can be referred to in the relevant art and will not be described here.

[0158] As shown in Figure 4B, after switching from portrait mode to photo mode, the phone continues to display shooting interface 314. In shooting interface 314, configuration item 318-1 is set to unchecked, and configuration item 318-2 is set to checked. In shooting interface 314, capture preview area 315 displays a preview stream captured in photo mode. In addition, the reference material (e.g., photo 309-1) is still displayed floating above capture preview area 315.

[0159] If it is pre-determined that the materials related to the revisiting theme are all photos, as shown in Figure 4B, the configuration item 318-1 in the shooting interface 314 is in the selected state. In response to the user clicking on the configuration item 318-3, the shooting function corresponding to the revisiting theme is triggered to exit, the shooting of the target materials related to the revisiting theme is stopped, and the recording function provided by the camera application is enabled. In addition, in response to the user clicking on the configuration item 318-3, the mobile phone can also display the recording preview interface 401. The above-mentioned recording preview interface 401 is the application interface corresponding to the recording function. The preview stream collected according to the recording mode is displayed in the recording preview interface 401. In addition, the reference material is not displayed in the recording preview interface 401.

[0160] It is understandable that after exiting the shooting function corresponding to the revisiting theme, in response to the user's operation on the video preview interface 401, the mobile phone can shoot regular materials. For details, please refer to the shooting function provided by the camera application in the related art.

[0161] In addition, in a possible embodiment, when the mobile phone switches the reference material displayed on the shooting interface 314, the shooting mode can also change according to the type of the switched reference material. The types of the reference material can include photos mainly of people and photos mainly of objects or scenery.

[0162] After the reference material displayed on the shooting interface 314 switches from a photo primarily featuring people to a photo primarily featuring objects or scenery, the shooting mode can be switched from portrait mode to photo mode, and accordingly, configuration item 318-2 is set to the selected state. Similarly, after the reference material displayed on the shooting interface 314 switches from a photo primarily featuring objects or scenery to a photo primarily featuring people, the shooting mode can be switched from photo mode to portrait mode. Correspondingly, configuration item 318-1 is set to the selected state.

[0163] Of course, if the type of material related to the revisiting theme is not predefined, the type of reference material may also include video. After the mobile phone activates the shooting function related to the revisiting theme, in the displayed shooting interface 314, one of configuration items 318-1, configuration item 318-2, or configuration item 318-3 is selected, that is, the mobile phone can capture the preview stream in portrait mode, photo mode, or video mode. For specific examples, please refer to the aforementioned embodiments and will not be described in detail here. In addition, in response to the user switching the shooting mode, the mobile phone does not trigger the exit of the shooting function corresponding to the revisiting theme.

[0164] In some embodiments, when the mobile phone has not exited the shooting function corresponding to the revisiting theme, the displayed shooting interface includes reference materials. The user can refer to the reference materials displayed on the shooting interface 314, adjust the framing screen, the posture of the shooting subject, and operate the mobile phone to collect the target material.

[0165] When the enabled shooting mode is portrait mode or photo mode, as shown in FIG3 , the mobile phone can capture a frame of image in response to the user clicking control 317, which serves as the target material corresponding to the first reference material. Clicking control 317 can be referred to as the first operation. The mobile phone then displays a material preview interface 320 , which displays the target material being captured, for example, a photo 321.

[0166] As can be understood, when the shooting mode is video recording mode, the process of capturing the target material can be similar to the process of recording a regular video. After detecting the user's instruction to stop recording, it is determined that the target material corresponding to the first reference material has been captured. The mobile phone then displays a material preview interface 320, which displays the target material captured this time.

[0167] In addition, when the number of reference materials is N (the value of N is greater than 1), after collecting the target material corresponding to the Mth reference material, if M is a positive integer less than N, the displayed material preview interface includes a reshoot control 322, a save control 323 and a completion control 324.

[0168] For example, if the number of reference materials is 3, after the target material (photo 321) corresponding to the first reference material is acquired, as shown in FIG3 , the displayed material preview interface 320 includes a retake control 322, a save control 323, and a completion control 324. For another example, if the number of reference materials is 3, after the target material corresponding to the second reference material (e.g., the third material) is acquired, as shown in FIG5 , the displayed material preview interface 501 also includes a retake control 322, a save control 323, and a completion control 324.

[0169] As shown in Figure 3, while the material preview interface 320 is displayed, in response to the user clicking the retake control 322, the shooting interface 314 can be displayed again. The reference material displayed on the shooting interface 314 remains unchanged, for example, it is still photo 309-1, prompting the user to re-capture the target material corresponding to the first reference material. In addition, the shooting mode enabled by the mobile phone remains unchanged.

[0170] Then, after the mobile phone responds to the user operation and re-collects the target material corresponding to the first reference material, the material preview interface 320 can be displayed again. The target material corresponding to the re-collected first reference material can be displayed on the material preview interface 320. Similarly, the re-displayed material preview interface 320 also includes a retake control 322, a save control 323 and a completion control 324.

[0171] 5 , while the material preview interface 320 is displayed, in response to the user clicking the save control 323, the first target material is saved, and the capture interface 502 is displayed. For example, the first target material (photo 321) can be saved to the gallery. Accordingly, a thumbnail of photo 321 is displayed in the capture interface 502.

[0172] The shooting interface 502 is similar to the shooting interface 314 in that both have a capture preview area and a shooting operation area, and both include a control 317 for instructing shooting in the shooting operation area.

[0173] The difference between the above-mentioned shooting interface 502 and the shooting interface 314 is that the reference materials displayed thereon are different. For example, the reference material displayed on the shooting interface 314 is the reference material with the recommended order being the Mth, and the reference material displayed on the shooting interface 502 is the reference material with the recommended order being the M+1th.

[0174] In the examples of Figures 3 and 5, the value of M is 1. As shown in Figures 3 and 5, photo 309-1 is displayed on the shooting interface 314, and photo 309-2 is displayed on the shooting interface 502. In addition, on the shooting interface 314, among the navigation points 319 displayed relative to the reference material, the navigation point corresponding to photo 309-1 is selected, and on the shooting interface 502, among the navigation points 319 displayed relative to the reference material, the navigation point corresponding to photo 309-2 is selected. In a possible embodiment, in response to a left swipe operation by the user on photo 309-2, photo 309-3 can be switched to display. In response to a right swipe operation by the user on photo 309-2, photo 309-1 can be switched to display again. In this way, in response to user operations, multiple corresponding target materials can be collected with reference to the same reference material.

[0175] In some examples, when the phone switches to displaying shooting interface 502, the shooting mode remains unchanged. For example, after the phone captures and saves the first target material in portrait mode, it continues to capture the preview stream in portrait mode in response to displaying shooting interface 502. Correspondingly, configuration item 318-1 in shooting interface 502 remains selected.

[0176] In other examples, when the mobile phone switches to display the shooting interface 502, the enabled shooting mode may change with the type of reference material displayed on the shooting interface 502. For example, if the reference material displayed on the shooting interface 502 is a photo mainly of a person, the portrait mode is enabled to capture the preview stream, and accordingly, the configuration item 318-1 in the shooting interface 502 is selected. If the reference material displayed on the shooting interface 502 is a photo mainly of an object or a scene, the shooting mode is enabled to capture the preview stream, and accordingly, the configuration item 318-2 in the shooting interface 502 is selected. If the reference material displayed on the shooting interface 502 is a video, the video recording mode is enabled to capture the preview stream, and accordingly, the configuration item 318-3 in the shooting interface 502 is selected.

[0177] Of course, after the shooting interface 502 is displayed, the user can manually switch the shooting mode. For details, please refer to switching the shooting mode on the shooting interface 314, which will not be described here.

[0178] For example, when configuration item 318-1 is selected in shooting interface 502, the phone captures a frame of imagery in response to the user clicking control 317 on shooting interface 502. This frame of imagery serves as the target material corresponding to the second reference material. Subsequently, the phone displays material preview interface 501, which displays the target material being captured, such as photo 503. This material preview interface 501 can also be referred to as the sixth interface.

[0179] As described in the above embodiment, the number of reference materials is 3. After the target material corresponding to the second reference material is collected, the material preview interface 501 also displays a retake control 322, a save control 323 (third control) and a completion control 324 (second control).

[0180] As shown in Figure 5 , while the material preview interface 501 is displayed, in response to the user clicking the save control 323, a third operation is performed, and the target material (photo 503) corresponding to the second reference material (photo 309-2) is stored in the gallery. Furthermore, in response to the user clicking the save control 323, a capture interface 504 is displayed, a fourth interface. Capture interface 504 includes a thumbnail 506 of photo 503. In response to the user clicking thumbnail 506, an application interface provided by the gallery application is displayed, which includes photo 503.

[0181] The shooting interface 504 is similar to the shooting interface 502 in that both have a capture preview area and a shooting operation area, and both include a control 317 for instructing shooting in the shooting operation area.

[0182] The shooting interface 504 is different from the shooting interface 502 in that the reference materials displayed thereon are different. In addition, the reference materials displayed on the shooting interface 504 are also different from the reference materials displayed on the shooting interface 314 .

[0183] For example, the reference material displayed on the shooting interface 314 is the reference material with the Mth order of recommendation, the reference material displayed on the shooting interface 502 is the reference material with the M+1th order of recommendation, such as the second material, and the reference material displayed on the shooting interface 504 is the reference material with the M+2th order of recommendation, such as the fourth material. In some examples, when the mobile phone switches to display the shooting interface 504, the shooting mode remains unchanged. For example, after the mobile phone uses portrait mode to capture and save the third target material, in response to displaying the shooting interface 504, it continues to capture the preview stream in portrait mode. Correspondingly, the configuration item 318-1 in the shooting interface 504 remains selected.

[0184] In other examples, when the mobile phone switches to display the shooting interface 504 , the enabled shooting mode may change according to the type of the reference material (eg, photo 309 - 3 ) displayed on the shooting interface 504 .

[0185] In some embodiments, the number of target materials that the user is guided to capture is the same as the number of reference materials. For example, when the number of reference materials is N (where N is greater than or equal to 1), after capturing the corresponding target material with reference to N different reference materials 309, the material preview interface for displaying the target material includes a retake control 322 and a completion control 324.

[0186] For example, as shown in FIG6A , while shooting interface 504 is displayed, in response to a user clicking control 317 on shooting interface 504, the mobile phone captures a frame of imagery as the target material corresponding to the third reference material, referred to as the fifth material. Subsequently, the mobile phone displays a material preview interface 601, which displays the target material being captured, such as photo 602. If the number of reference materials is three and the target material corresponding to the third reference material has already been captured, as shown in FIG6A , material preview interface 601 displays a retake control 322 and a completion control 324.

[0187] In some embodiments, as shown in Figure 6A, in response to the user clicking the completion control 324, the mobile phone may display the main interface 301. Clicking the completion control 322 may be referred to as the second operation. An interactive window 307 may be displayed on the main interface 301, with a new prompt 603 added to the interactive window 307, such as the text "Checked in." The main interface 301 displaying the interactive window 307 may be referred to as the third interface.

[0188] In some scenarios, users can select reference materials for reference by sliding relative to the reference materials in the shooting interface. After shooting, they can end the shooting by clicking the completion control in the material preview interface.

[0189] In addition, a new dialog box 604 is added to interactive window 307. This dialog box 604 displays the collected target materials. In some embodiments, the number of target materials displayed in dialog box 604 is the same as the number of target materials actually collected. If dialog box 604 cannot display all target materials, some target materials may be hidden. As shown in FIG6A , after the third target material has been collected, dialog box 604 may display photo 321, photo 503, and photo 602.

[0190] It is understood that when the mobile phone displays the material preview interface 320, that is, when the mobile phone has only collected the first target material, in response to the user clicking the completion control 324, the main interface 301 including the interactive window 307 may also be displayed. In this scenario, the newly added dialog box 604 in the interactive window 307 only displays the photo 321 (the first target material). When the mobile phone displays the material preview interface 501, that is, when the mobile phone has collected the second target material, in response to the user clicking the completion control 324, the main interface 301 including the interactive window 307 may also be displayed. In this scenario, the dialog box 604 in the interactive window 307 displays both the photo 321 (the first target material) and the photo 503 (the second target material).

[0191] S102: The mobile phone generates media data that conforms to the theme of revisiting an old place based on the target material and the reference material.

[0192] In some embodiments, after completing the acquisition of the target material, the mobile phone can generate one or more splicing data based on the target material and the reference material, wherein the splicing data can be data obtained by splicing one target material with one reference material.

[0193] For example, after the mobile phone detects that the user clicks the completion control 324, it can determine the associated materials corresponding to each target material in the reference material. The mobile phone can generate corresponding splicing data based on the target material and the corresponding associated materials, such as first splicing data generated based on the second material (reference material) and the third material (target material).

[0194] As an implementation method, the associated material can be the reference material that the AI ​​model assesses as having the highest similarity to the target material. Specifically, if the target material is a photo primarily of a person, the higher the similarity between the pose of the person in the target material and the pose of the person in the reference material, the higher the similarity between the target material and the reference material assessed by the phone.

[0195] As another implementation, the aforementioned associated material can also be a reference material displayed when guiding the user to capture the target material. For example, as shown in Figure 3, when photo 309-1 is displayed on capture interface 314, the phone captures photo 321 in response to a user's operation. In this scenario, photo 309-1 can be considered an associated material for photo 321. This embodiment of the present application does not specifically limit the method for determining associated materials, and other methods may also be used.

[0196] After determining the associated materials of each target material, the mobile phone generates corresponding stitching data in the following manner: the mobile phone stitches each target material with the corresponding associated material in a left-right, top-bottom, or picture-in-picture arrangement to obtain stitching data corresponding to each target material.

[0197] It can be understood that when the target material and the associated material are both images, the stitched data is a stitched image, for example, the stitched photo 606 - 1 in FIG6A .

[0198] It can be understood that when both the target material and the associated material are videos, the above-mentioned spliced ​​data can be a spliced ​​video.

[0199] The aforementioned spliced ​​video can be video data generated by splicing the target material and the associated material in the time dimension. For example, as shown in FIG6B , a transition effect is used to connect the target material and the associated material to generate a corresponding spliced ​​video. In the generated spliced ​​video, the playback time of the video frames from the target material is earlier than (or later than) the playback time of the video frames from the associated material.

[0200] The aforementioned stitched video can also be video data generated by stitching the target material and the associated material together in the picture dimension. For example, as shown in FIG6C , the stitched video is generated by stitching the video frames from the target material and the associated material together frame by frame. Video frames 616 of the stitched video include video frames 617 from the target material and video frames 618 from the associated material.

[0201] When one of the target material and the associated material is a photo and the other is a video, the stitched data may also be a stitched video. For example, if the target material is a video and the associated material is a photo, the video frame of the stitched video is an image obtained by stitching the video frame of the target material and the corresponding photo of the associated material.

[0202] In some embodiments, after the mobile phone obtains the splicing data, the displayed interactive window 307 further includes a preview box 605 , in which the splicing data 606 is displayed.

[0203] As shown in Figure 6A, after the mobile phone collects three target materials, for example, after collecting photos 321, 503, and 602, it generates three corresponding stitching data 606, for example, a stitching photo 606-1 containing photo 321, a stitching photo 606-2 containing photo 503 (such as first stitching data), and a stitching photo 606-3 containing photo 602 (such as second stitching data). The mobile phone can display the stitching data 606 in response to the user clicking on any stitching data 606. If the stitching data 606 is a stitching video, in response to the user clicking on the stitching video, a playback window is displayed for playing the stitching video. If the stitching data 606 is a stitching image, in response to the user clicking on the stitching image, a mask is newly displayed to display an enlarged stitching image on the layer.

[0204] In some embodiments, the preview box 605 also includes a video generation control 607. Furthermore, a prompt 608 is displayed on the interactive window 307, guiding the user to click the video generation control 607, thereby triggering the mobile phone to generate media data based on the spliced ​​data. In other embodiments, the interactive window 307 also includes a save control 614. In response to the user clicking control 614, the spliced ​​data 606 in the preview box 605 can be stored in the gallery.

[0205] For example, the mobile phone can process and splice data according to a video generation template corresponding to the theme of revisiting an old place, and generate media data that conforms to the theme of revisiting an old place. The video generation template includes template data for producing a video, such as music to be configured, special effects to be added, etc.

[0206] As shown in Figure 6A, in response to a user clicking on the Generate Video control 607, the mobile phone generates media data 609 based on stitched photos 606-1, 606-2, and 606-3. The media data 609 may be video data composed of stitched photos 606-1, 606-2, and 606-3. Furthermore, the media data 609 may also be configured with music, special effects, and / or transitions. Clicking on the Generate Video control 607 may be referred to as the fourth operation. The media data 609 may be referred to as the first media data.

[0207] After the media data 609 is generated, the media data 609 can be newly displayed on the interactive window 307. The mobile phone can respond to the user's operation on the media data 609 and store, share, delete, play or comment on the media data 609.

[0208] In some embodiments, a smart text control 610 , a template change control 611 , a music change control 612 , and a duration adjustment control 613 are displayed below the media data 609 .

[0209] For example, in response to a user's operation on the smart text control 610, the phone can add text effects to the video frame of media data 609. In response to a user's operation on the template change control 611, the phone can regenerate new media data based on the reference material and the replaced video generation template. In response to a user's operation on the music change control 612, the phone can change the music in media data 609. In response to a user's operation on the music change control 613, the phone can also change the playback duration of media data 609.

[0210] The above embodiment introduces a process of guiding a user to use a mobile phone to shoot a target material based on a reference material recommended by a mobile phone, and then generating media data according to the target material.

[0211] In some embodiments, the mobile phone can also manage reference materials in response to user instructions.

[0212] Taking the example of a user managing reference materials through voice commands (or text commands), in other examples, the user may also manage reference materials through manual operations, which is not limited in this application.

[0213] Scenario of adding reference materials through voice commands: As shown in Figure 7, the mobile phone displays an interactive window 307, which includes a dialog box 701. The dialog box 701 displays reference materials 309 (such as photos 309-1, 309-2, and 309-3). In response to the user long pressing the voice input control 311, or in response to detecting that the user speaks a pre-configured keyword, the mobile phone can wake up the voice assistant function. After waking up the voice assistant function, the mobile phone can detect whether the audio signal propagating in the environment contains the user's voice command.

[0214] For example, a user might speak the voice command "Add three photos of the beach" into a mobile phone, which is referred to as the first voice command. Accordingly, the mobile phone might detect an audio signal containing the voice command "Add three photos of the beach." Upon detecting the voice command, the mobile phone recognizes and parses the voice command, obtaining recognition and parsing results for the voice command.

[0215] As shown in FIG. 7 , the recognition result obtained by the mobile phone is the text “add three photos of the seaside”. Correspondingly, the mobile phone may display the recognition result 702 of the voice command on the interactive window 307 .

[0216] As shown in Figure 7, the phone's parsing result indicates a voice command instructing the editing function. After initiating the editing function, the phone, based on the key semantic terms "three photos," "beach," and "photo" parsed from the voice command, selected three photos with beach backgrounds as new reference materials from the historical materials that met condition 1 and were not already selected as reference materials.

[0217] In response to selecting the three newly added reference materials, the mobile phone may display a prompt 703 on interactive window 307, indicating that the three newly added reference materials have been added. Furthermore, the mobile phone may display a dialog box 704 on interactive window 307. Dialog box 704 also displays reference materials 309, which include photo 309-1, photo 309-2, photo 309-3, photo 309-4, photo 309-5, and photo 309-6.

[0218] As can be understood, in interactive window 307, various types of information (e.g., prompts, dialog boxes, etc.) are arranged from top to bottom in the order in which they appear. Both dialog boxes 701 and 704 are used to display reference material 309. Dialog box 704 is arranged below dialog box 701, indicating that dialog box 704 appears later. Compared to dialog box 701, dialog box 704 adds photos 309-4, 309-5, and 309-6, indicating that three more photos have been added to reference material 309 determined by the mobile phone. This newly added material can be referred to as the sixth material.

[0219] Scenario of deleting reference materials through voice commands: As shown in Figure 8, the mobile phone displays an interactive window 307, wherein the interactive window 307 includes a dialog box 704. In response to the user long pressing the voice input control 311, or in response to detecting a keyword used to wake up the voice assistant function, the mobile phone can wake up the voice assistant function again. Afterwards, if the user can say the voice command "delete a photo" to the mobile phone, it is called the second voice command. Correspondingly, the mobile phone can detect an audio signal containing the voice command "delete a photo". In response to detecting the above voice command, the mobile phone recognizes and parses the voice command to obtain the recognition result and parsing result of the voice command.

[0220] As shown in FIG. 8 , the recognition result obtained by the mobile phone is the text “delete a photo”. Correspondingly, the mobile phone may display the recognition result 801 of the voice command on the interactive window 307 .

[0221] As shown in FIG8 , the parsed result obtained by the mobile phone is a voice command indicating the material editing function. After the material editing function is activated, the mobile phone selects to delete a reference material from the current reference material 309 based on the key semantic words "delete," "one," and "photo" parsed from the voice command, where the type of the selected reference material is a photo.

[0222] For example, the mobile phone can select reference materials to be deleted from the reference materials (e.g., photo 309-1, photo 309-2, photo 309-3, photo 309-4, photo 309-5, photo 309-6) in dialog box 704 according to their aesthetic scores. The reference materials to be deleted are photos, and their aesthetic scores are lower than a preset value, or lower than other reference materials (photo-type materials).

[0223] As shown in Figure 8, after deleting the reference material, the mobile phone may display a prompt message 802 on interactive window 307 to remind the user that a reference material has been deleted. In addition, the mobile phone may display a dialog box 803 on interactive window 307. Dialog box 803 also displays the reference materials. The reference materials in dialog box 803 include photos 309-1, 309-2, 309-3, 309-4, and 309-5, but do not include the deleted photo 309-6. Dialog box 803 is displayed below dialog box 704, indicating that the latest reference material does not include photo 309-6.

[0224] The user can also directly indicate which photo to delete, and the phone can delete the photos with the corresponding sequence number according to the sorting sequence number.

[0225] In addition, the user can also use voice commands to instruct the phone to replace the reference material with other historical materials that meet condition 1. For example, from other historical materials that meet condition 1, select R materials, where R is the number of current reference materials. For example, in Figure 8, when dialog box 803 is displayed, the number R is 5. Then, the current reference material is deleted, and the selected R reference materials are used as the new reference material. In other embodiments, the user can also use a third voice command to update some reference materials, which is not specifically limited in this embodiment of the present application.

[0226] In other embodiments, users can also manage reference materials through text commands. For example, in response to a user clicking keyboard input control 312, the mobile phone receives a text command entered by the user. In response to the text command "Add three beach photos," the mobile phone selects three photos with beach backgrounds from the historical materials that meet condition 1 as new reference materials. The implementation details are shown in FIG7 and are not further described here.

[0227] In some other embodiments, the user may also manually manage reference materials, for example.

[0228] As shown in Figure 9, the mobile phone displays interactive window 307. Interactive window 307 includes dialog box 701, with control 901 displayed relative to dialog box 701. In response to the user clicking control 901, the mobile phone may display interface 902. Interface 902 includes the reference materials displayed in dialog box 701, such as photo 309-1, photo 309-2, and photo 309-3.

[0229] If dialog box 701 cannot display N reference materials, interface 902 also includes reference materials not displayed in dialog box 701. Furthermore, if dialog box 701 cannot display all reference materials, reference materials that primarily feature people can be prioritized in dialog box 701, and reference materials with high aesthetic scores can be prioritized.

[0230] In other embodiments, the control 901 may not be displayed in the above-mentioned interactive window 307 . In this scenario, the mobile phone may also display the interface 902 in response to the user clicking on the blank area in the dialog box 701 .

[0231] In addition, in the interface 902, the above-mentioned photos 309-1, 309-2 and 309-3 are in the selected state, and icons indicating selection are displayed on the photos 309-1, 309-2 and 309-3. For example, the icon 903 displayed relative to the photo 309-1 indicates that the photo 309-1 is in the selected state.

[0232] In some embodiments, the mobile phone may initiate a process of deleting the reference material specified by the user in response to the user's operation on the interface 902 .

[0233] A method of deleting a reference material: In response to clicking on any reference material in the interface 902 , for example, clicking on the photo 309 - 1 , the mobile phone may set the reference material (eg, photo 309 - 1 ) to an unselected state.

[0234] For example, as shown in FIG9 , in response to the user clicking on the photo 309 - 1 , the mobile phone may cancel the mark 903 and display a mark 904 indicating that the photo 309 - 1 is not selected.

[0235] Afterwards, in response to the user clicking the confirmation control 905, the mobile phone again displays the main interface 301 including the interactive window 307. In the interactive window 307 that is displayed again, the dialog box 701 includes the photo 309-2 and the photo 309-3, thereby achieving the deletion of the reference material.

[0236] In this scenario, in response to the user clicking on control 906, the user can be guided to operate the mobile phone to shoot the target material based on 309-2 and photo 309-3, and then generate media data based on the target material. The specific implementation process can be referred to the aforementioned embodiment and will not be repeated here.

[0237] Another way to delete reference materials: As shown in Figure 10, while interface 902 is displayed, in response to the user clicking on the add control 1001, the mobile phone may display interface 1002, which is referred to as the fifth interface. Interface 1002 is a grid browsing page that displays all historical materials that meet condition 1 (e.g., the first material, the second material, the fourth material, and the sixth material) for user selection. It is understood that the historical materials that meet condition 1 also include photos 309-1, 309-2, and 309-3.

[0238] It is understandable that the above-mentioned photos 309-1, 309-2 and 309-3 are current reference materials, and are marked as selected in the interface 1002. In the interface 1002, other materials (such as the sixth material) are marked as unselected.

[0239] In addition, the phone displays a window 1003, such as a selection preview box, above interface 1002. Window 1003 displays the current reference material, for example, photo 309-1, photo 309-2, and photo 309-3. Window 1003 displays thumbnails representing each of these photos. Within window 1003, a control indicating deletion is displayed for each reference material. For example, control 1004 is displayed for photo 309-1 in window 1003.

[0240] In some embodiments, the mobile phone can delete the photo 309 - 1 from the reference material in response to the user clicking the control 1004 . Correspondingly, the mobile phone can set the photo 309 - 1 in the interface 1002 to an unselected state and delete the photo 309 - 1 in the window 1003 .

[0241] In response to the user clicking the control 1005 on the window 1003, the mobile phone again displays the main interface 301 including the interactive window 307. In the interactive window 307 that is displayed again, the dialog box 701 includes the photo 309-2 and the photo 309-3. The reference material is deleted.

[0242] In other embodiments, after the mobile phone displays the interface 1002 , the mobile phone may initiate a process of adding reference materials in response to a user operation on the interface 1002 .

[0243] As shown in Figure 11, the mobile phone displays interface 1002. Interface 1002 includes a material 1101 marked as unselected. Window 1003 is displayed on interface 1002. In response to a user clicking on material 1101, material 1101 in interface 1002 is selected. Furthermore, the mobile phone may also display a new material 1101 in window 1003, indicating that material 1101 has been selected as a reference material.

[0244] Afterwards, in response to the user clicking the control 1005 on the window 1003, the mobile phone again displays the main interface 301 including the interactive window 307. In the interactive window 307 that is displayed again, the dialog box 701 includes the photo 309-2, the photo 309-3 and the material 1101. This realizes the addition of a new reference material.

[0245] In the example shown in FIG11 , the mobile phone can respond to user operations and select historical materials that meet condition 1 as reference materials. In other embodiments, the mobile phone can also respond to user operations and select new reference materials from all materials (photos or videos) stored in the mobile phone.

[0246] For example, as shown in FIG11 , the interface 1002 includes a control 1102. In response to the user clicking the control 1102, the mobile phone can display all stored materials for the user to select. In this way, the mobile phone can respond to the user's operation and select a newly added reference material from all stored materials.

[0247] In another embodiment, as shown in FIG12 , while the mobile phone displays the main interface 301 , in response to the current scene being consistent with the theme of revisiting an old place, a function card 1201 is displayed on the main interface 301 . The above function card 1201 is similar to the function card 302 and will not be described in detail here.

[0248] In response to the user clicking on the function card 1201, the interaction window 1202 is displayed.

[0249] Exemplarily, the interactive window 1202 includes prompt information 1203, and the prompt information 1203 is used to remind the user to view photos and / or videos that have been taken at this location.

[0250] For example, interactive window 1202 also includes dialog box 1204, which is used to display reference material 309. For example, photos 309-1, 309-2, and 309-3. Reference material 309 may be selected from historical materials that meet condition 1. For details, refer to the previous embodiment and are not further elaborated here.

[0251] In addition, the reference material 309 displayed in the dialog box 1204 may satisfy one or more of the following conditions:

[0252] (1) The aesthetic score is higher than other materials, where other materials refer to historical materials that meet condition 1 but are not selected as reference materials.

[0253] (2) The similarity between any two reference materials is lower than a preset similarity threshold. It is understood that if the similarity between two reference materials is higher than or equal to the preset similarity threshold, deduplication can be performed based on the aesthetic score. For example, if the similarity between reference material d and reference material e is higher than or equal to the preset similarity threshold, and the aesthetic score of reference material d is higher than that of reference material e, after deduplication based on the aesthetic score, reference material d is displayed in dialog box 1204, while reference material e is not displayed.

[0254] In addition, when the dialog box 1204 cannot display all reference materials, some reference materials may be hidden, wherein the dialog box 1204 prioritizes displaying reference materials with images mainly featuring people.

[0255] When the dialog box 1204 is displayed, the user can view the reference material 309 by clicking the reference material 309 in the dialog box 1204. For example, the mobile phone can display a large image of the photo 309-1 in response to the user clicking the photo 309-1.

[0256] Exemplarily, the interactive window 1202 also includes a control 1205. In response to the user clicking on the control 1205, the mobile phone displays a shooting interface 1206, such as the seventh interface. The above-mentioned shooting interface 1206 can display a preview stream captured in real time. In addition, the shooting interface 1206 can also include a control 1207 for indicating shooting. Taking the mobile phone in portrait mode as an example, in response to the user clicking on the control 1207, a frame of image is captured as the target material (seventh material). In addition, the captured target material can also be stored in the gallery. After the target material is stored in the gallery, a thumbnail 1208 of the target material can be displayed in the shooting interface 1206. In response to the user clicking on the thumbnail 1208, the mobile phone can display an image preview interface containing the target material.

[0257] It is understandable that while the shooting interface 1206 is displayed, the mobile phone can respond to the user's operation, adopt any shooting mode, shoot one or more target materials, and save them into the gallery.

[0258] Afterwards, as shown in FIG13 , in response to the user's instruction to return to the main interface 301 (e.g., referred to as the second operation), the mobile phone can display the main interface 301 again, and the main interface 301 includes an interactive window 1202. Compared with the interactive window 1202 in FIG12 , the interactive window 1202 in FIG13 has a new prompt message 1301 and a dialog box 1302 (second display bar). Among them, the prompt message 1301 is used to prompt the user that the shooting has been completed at the current location. The dialog box 1302 can display the target material collected at the current location, such as a photo 1303 (e.g., the seventh material).

[0259] In some embodiments, before displaying dialog box 1302, the mobile phone may further perform deduplication on the target materials. For example, if the target materials collected at the current location include materials a, b, c, and d, and the similarity between materials b and c exceeds a preset similarity threshold, and the aesthetic score of material b is higher than that of material c, after deduplication, material b is retained and material c is removed, and the mobile phone displays materials a, b, and d in dialog box 1302.

[0260] In addition, when the dialog box 1302 cannot display all target materials, some target materials may be hidden, wherein target materials with characters as the main subject are preferentially displayed on the dialog box 1302 .

[0261] In other embodiments, after the mobile phone shoots the target material in response to the user operation, it can also select one or more materials from the target material and the reference material respectively.

[0262] For example, the mobile phone may select the material with the highest aesthetic score from the reference material as one of the selected materials, such as photo 309 - 1 from the reference material. The mobile phone may select the material with the highest aesthetic score from the target material as one of the selected materials, such as photo 1303 from the target material.

[0263] Correspondingly, the interactive window 1202 also includes a prompt message 1304, which is used to prompt the user that the target material and reference material suitable for splicing have been selected. In addition, the interactive window 1202 also includes a dialog box 1305, which is used to display the selected reference material and target material. For example, the dialog box 1305 displays photo 309-1 and photo 1303. A control 1306 is displayed on the lower side of the dialog box 1305. In response to the user clicking the control 1306 (such as referred to as the fifth operation), the mobile phone generates a puzzle video based on the video generation template corresponding to the theme of revisiting the old place based on photos 1303 and 309-1. If the similarity evaluation value between photo 1303 and photo 309-2 is higher than the similarity evaluation value between photo 309-1, a third spliced ​​data containing photos 1303 and photo 309-2 can be generated.

[0264] After the puzzle video is generated, the mobile phone can display the puzzle video in the interactive window 1202, such as the video 1307 in Figure 13. In addition, the mobile phone can also display functional controls for editing the puzzle data in the interactive window 1202, such as the template change control 1308, the music change control 1309, and the duration change control 1310.

[0265] For example, in response to the user's operation on the template change control 1308, or detecting a voice instruction to change the video template, the mobile phone can reselect a video generation template, and then regenerate a puzzle video based on photo 1303 and photo 309-1 according to the newly selected video generation template.

[0266] For example, when the generated puzzle video is video 1307, the mobile phone can change the video music configured in video 1307 in response to the user's operation on the change music control 1309 or detecting a voice instruction to change the music.

[0267] For example, when the generated puzzle video is video 1307, the mobile phone can extend or shorten the duration of video 1307 in response to the user's operation on the change duration control 1310 or detecting a voice instruction instructing to change the duration.

[0268] In addition, the interactive window 1202 also includes a confirmation control 1311. In response to the user clicking the confirmation control 1311, the generated puzzle video or the edited puzzle video can be saved.

[0269] In another embodiment, as shown in FIG14 , while the mobile phone displays the main interface 301, in response to the current scene being consistent with the theme of revisiting an old place, a function card 1201 is displayed on the main interface 301. In response to the user clicking on the function card 1201, an interactive window 1202 is displayed.

[0270] As shown in FIG14 , interactive window 1202 also includes control 1401 (fourth control). In response to the user clicking control 1401, the mobile phone can generate a video related to the theme of past journeys. It is understandable that the video related to past journeys can be a video generated based on historical material (second media data).

[0271] For example, video 1402 may be generated based on reference materials such as photos 309-1, 309-2, and 309-3, using a video generation template related to the theme of past journeys. Alternatively, the mobile phone may generate video 1402 based on historical materials that meet condition 1, using a video generation template related to the theme of past journeys. After generating video 1402, the mobile phone may display video 1402 in interactive window 1202.

[0272] In another embodiment, while the mobile phone displays the main interface 301, in response to the current scene being consistent with the theme of revisiting an old place, a function card 1201 is displayed on the main interface 301. In response to the user clicking on the function card 1201, an interactive window 1202 is displayed.

[0273] Exemplarily, the interactive window 1202 further includes a dialog box 1204, which is used to display the reference material 309, such as photo 309-1, photo 309-2, and photo 309-3.

[0274] Afterwards, the mobile phone collects one or more target materials in response to the user's operation on control 1205. The process of collecting target materials can be referred to in the previous embodiment and will not be described in detail here. After collecting the target materials, as shown in Figure 15, the mobile phone again interacts with window 1202. Interactive window 1202 includes a dialog box 1302. The dialog box 1302 can display the target materials collected at the current location, such as photos 1303.

[0275] In addition, the interactive window 1202 also includes prompt information 1501, which can recommend that the user can create a puzzle based on the target material and the reference material. In addition, the interactive window 1202 also includes prompt information 1502, which is used to prompt the user to trigger the mobile phone to create a puzzle by saying "Try to create a puzzle."

[0276] In response to detecting the user speaking a voice command "Try to put one picture together", the mobile phone can select one or more materials from the reference materials and the historical materials that meet condition 1 to form a puzzle. For example, photos 309-1, 309-2, and 1503 are all historical materials taken at the current location, and photo 309-1 was taken in spring, photo 309-2 was taken in summer, and photo 1503 was taken in winter. In addition, if the current season is autumn and photo 1303 was taken at the current location, the mobile phone can generate puzzle 1504 based on photos 309-1, 309-2, 1503, and 1303 in response to the voice command "Try to put one picture together", where puzzle 1504 is composed of photos 309-1, 309-2, 1503, and 1303.

[0277] The present application also provides an electronic device. The electronic device can perform the methods described in the preceding embodiments. The electronic device may include a memory and one or more processors. The memory and processor are coupled. The memory is used to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the electronic device can perform the steps described in the preceding embodiments. Of course, the electronic device includes, but is not limited to, the memory and one or more processors described above.

[0278] For example, the electronic device displays a first interface, which may be the shooting interface 314 in the aforementioned embodiment. The first material displayed on the first interface may be the photo 309-1 mentioned in the aforementioned embodiment.

[0279] The present application also provides a chip system that can be applied to the electronic device in the aforementioned embodiment. The chip system includes at least one processor and at least one interface circuit. The processor can be the processor in the aforementioned electronic device. The processor and the interface circuit can be interconnected via a line. The processor can receive and execute computer instructions from the memory of the aforementioned electronic device via the interface circuit. When the computer instructions are executed by the processor, the electronic device can perform the various steps in the aforementioned embodiment. Of course, the chip system can also include other discrete components, which are not specifically limited in the present application.

[0280] In some embodiments, through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0281] The functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0282] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk.

[0283] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for generating media data, characterized in that: Applied to an electronic device, the electronic device includes a camera, and the method includes: Displaying a first interface, wherein the first interface displays a preview stream collected by the camera in real time, and the first interface also displays a first material; In response to a user sliding operation relative to the first material, a second interface is displayed, wherein the second interface displays a preview stream collected by the camera in real time, and the second material is also displayed on the second interface; In response to a first operation of the user instructing shooting, shooting a third material; In response to the second operation, a third interface is displayed, on which the first spliced ​​data composed of the second material and the third material is displayed.

2. The method according to claim 1, characterized in that The first material and the second material are materials that satisfy one or more of the following first conditions: The shooting position is the same as the current position of the electronic device, or the distance is less than a preset distance threshold; The time interval between the shooting time and the current time point exceeds the preset time threshold.

3. The method according to claim 2 or 3, characterized in that: Before displaying the first interface, the method further includes: In the case where the electronic device stores materials satisfying the first condition, a first card is displayed, wherein the first condition includes a combination of one or more of the following: the shooting location is the same as the current location of the electronic device, or the distance is less than a preset distance threshold; the time interval between the shooting time and the current time point exceeds a preset time threshold; An operation of the user on the first card is detected.

4. The method according to claim 3, characterized in that: The method further comprises: In response to the user's operation on the first card, an interactive window is displayed; the interactive window includes a first control and a first display bar, and the first display bar includes the first material and the second material; The displaying the first interface includes: displaying the first interface in response to a user clicking on the first control.

5. The method according to any one of claims 1 to 4, characterized in that: After shooting the third material and before displaying the third interface, the method further includes: In response to the third operation, a fourth interface is displayed, wherein the fourth interface displays a preview stream collected by the camera in real time, and a fourth material is also displayed on the fourth interface, and the fourth material is different from the second material; In response to a first operation of the user instructing shooting, shooting a fifth material; After shooting the fifth material, the third interface further includes second stitching data, and the second stitching data is composed of the fourth material and the fifth material.

6. The method according to claim 5, characterized in that The method further comprises: In response to a fourth operation on the third interface, first media data is displayed, where the first media data is a video including the first spliced ​​data and the second spliced ​​data.

7. The method according to claim 5, characterized in that The similarity evaluation value between the third material and the second material is higher than the similarity evaluation value between the third material and the fourth material; the similarity evaluation value between the fifth material and the fourth material is higher than the similarity evaluation value between the fifth material and the second material; The similarity evaluation value is a quantitative value of the similarity of the contents of the pictures between different materials. The more similar the poses of the characters in the material are, the higher the corresponding similarity evaluation value is.

8. The method according to claim 4, characterized in that After displaying the interactive window, the method further includes: A first voice command of the user is detected, and a sixth material is newly displayed in the first display column, where the sixth material is also a material that meets the first condition; Alternatively, a second voice command of the user is detected, and the fourth material in the first display bar is canceled; Alternatively, a third voice instruction of the user is detected, and the fourth material in the first display column is replaced with the sixth material.

9. The method according to claim 5, characterized in that After displaying the interactive window, the method further includes: After detecting the user's operation on the first display bar, displaying a fifth interface, the fifth interface including the first material, the second material, the fourth material and the sixth material, in which a selected mark is displayed relative to the first material, the second material and the fourth material, and an unselected mark is displayed relative to the sixth material, and the sixth material is also a material that meets the first condition; a selection preview box is displayed on the fifth interface, and the selection preview box includes thumbnails of the first material, the second material and the fourth material; In response to a user's operation on the sixth material, changing the unselected mark displayed relative to the sixth material to the selected mark; In the selection preview frame, a thumbnail of the sixth material is newly displayed; In response to the user's confirmation operation in the selection preview box, the fifth interface is canceled, the interactive window is displayed again, and the sixth material is newly displayed on the first display bar of the interactive window.

10. The method according to claim 5, characterized in that The aesthetic scores of the first material, the second material, and the fourth material are higher than the aesthetic scores of other materials in the electronic device that meet the first condition.

11. The method according to claim 5, characterized in that After displaying the interactive window, the method further includes: After detecting the user's operation on the first display bar, displaying a fifth interface, the fifth interface including the first material, the second material, and the fourth material, and in the fifth interface, selection marks are displayed relative to the first material, the second material, and the fourth material; a selection preview box is displayed on the fifth interface, and the selection preview box includes thumbnails of the first material, the second material, and the fourth material; In response to a user operation on the thumbnail of the fourth material in the selection preview box, changing the selected mark displayed relative to the fourth material to the unselected mark; In the selection preview frame, canceling display of the thumbnail of the fourth material; In response to a confirmation operation by the user in the selection preview box, the fifth interface is canceled, the interactive window is displayed again, and the fourth material is canceled from the first display bar of the interactive window.

12. The method according to any one of claims 1 to 4, characterized in that: After shooting the third material, the method further includes: Displaying a sixth interface, wherein the sixth interface is a preview interface of the third material; the sixth interface includes a second control; Displaying the third interface in response to the second operation includes: displaying the third interface in response to the user's operation on the second control.

13. The method according to any one of claims 1 to 4, characterized in that: After shooting the third material, the method further includes: Displaying a sixth interface, wherein the sixth interface is a preview interface of the third material; the sixth interface includes a third control; In response to the user's operation on the third control, a fourth interface is displayed, wherein the fourth interface displays a preview stream collected by the camera in real time, and a fourth material is also displayed on the fourth interface, and the fourth material is different from the second material.

14. A method for generating media data, characterized in that: Applied to an electronic device, the electronic device includes a camera, and the method includes: In the case where the electronic device stores materials satisfying the first condition, a first card is displayed, wherein the first condition includes a combination of one or more of the following: the shooting location is the same as the current location of the electronic device, or the distance is less than a preset distance threshold; the time interval between the shooting time and the current time point exceeds a preset time threshold; After detecting the user's operation on the first card, an interactive window is displayed; the interactive window includes a first control and a first display bar, the first display bar includes a first material and a second material, and the first material and the second material are materials that meet the first condition; In response to the user's operation on the first control, displaying a seventh interface, wherein the seventh interface displays a preview stream collected by the camera in real time; In response to a first operation of the user instructing shooting, shooting a seventh material; In response to the second operation, the interactive window is displayed again, a second display bar is newly displayed on the interactive window, and the seventh material is displayed on the second display bar; In response to a fifth operation of the user, third spliced ​​data is newly displayed on the interactive window; the third spliced ​​data is composed of a seventh material and a second material, and a similarity evaluation value between the seventh material and the second material is higher than a similarity evaluation value between the seventh material and the first material; The similarity evaluation value is a quantitative value of the similarity between the image contents of different materials. The more similar the postures of the characters in the different materials are, the higher the corresponding similarity evaluation value is.

15. The method according to claim 14, characterized in that The interactive window further includes a fourth control, and the fourth control is displayed relative to the first display bar. The method further includes: In response to the user's operation on the fourth control, second media data is displayed on the interactive window, where the second media data includes the first material and the second material.

16. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 15.

17. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 15 are implemented.

18. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 15 are implemented.

Images