An image processing method, an electronic device, and a storage medium

Abstract

This application provides an image processing method, electronic device, and storage medium, relating to the field of image processing, and solves the problem that the imaging effect of human figures in photos taken by electronic devices is poor due to adverse environmental conditions. The method includes: displaying a first image; if the first image includes a face image, obtaining first feature information of a first person to whom the first face image in the first image belongs; wherein the first feature information includes at least facial feature information of the first person; obtaining a first three-dimensional model matching the first feature information; wherein the first three-dimensional model includes facial three-dimensional features extracted from historical images corresponding to the first person; fusing the facial three-dimensional features in the first three-dimensional model with the first face image in the first image to obtain a second image; and fusing the first image and the second image to obtain a fifth image.

Description

[0001] This application is a divisional application. The original application has the application number 202210682176.4 and the original application date is June 15, 2022. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of image processing for electronic devices, and more particularly to an image processing method, an electronic device, and a storage medium. Background Technology

[0003] Currently, with the development of technology, taking photos has become an essential function of electronic devices (such as mobile phones). In well-lit or suitable environments, smartphones can generally take clear portrait photos. However, in some unfavorable shooting scenarios, such as backlighting or low light, the image quality of portraits taken by users using electronic devices is poor, resulting in a bad user experience. Summary of the Invention

[0004] This application provides an image processing method, an electronic device, and a storage medium, which solves the problem that the imaging effect of human images is poor due to adverse environmental conditions when photographs taken by electronic devices.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] Firstly, this application provides an image processing method applicable to electronic devices. In this method, after acquiring a first photograph, if the first photograph includes a face image, the electronic device acquires first feature information of a first person to whom the first face image belongs. The first feature information includes at least facial feature information of the first person. Then, the electronic device acquires a first three-dimensional model matching the first feature information. The first three-dimensional model represents the three-dimensional features of the first person in a historical photograph corresponding to the first person; the three-dimensional features include at least facial three-dimensional features. Finally, the electronic device fuses the first three-dimensional model with the image of the first person in the first photograph to optimize the image of the first person in the first photograph, obtaining a second photograph.

[0007] Based on the above solution, when an electronic device acquires a photograph, it can obtain a 3D model of the person whose face image corresponds to that photograph. This 3D model is then used to optimize the person image in the photograph, resulting in an optimized photograph. This is because the 3D model represents the 3D features of the person in historical photographs. The detail information of the person in multiple historical photographs will inevitably be more abundant than the detail shown in the photograph currently taken by the electronic device. Therefore, the optimized photograph will have richer detail and a clearer image of the person. Thus, even if the photograph is not clear due to unfavorable shooting conditions, the technical solution provided in this application can optimize the person image to obtain a photograph with better image quality, thereby improving the user experience.

[0008] In one possible implementation of the first aspect, the electronic device acquires the first photograph by: the electronic device receiving a photographing operation; and the electronic device acquiring the first photograph in response to the photographing operation.

[0009] In this way, after the electronic device takes a picture, it can immediately optimize the image of the person in the photo, especially if the photo includes a face, using a 3D model of the person in that face. This optimized image is derived from the 3D model, which represents the 3D features of the person in historical photographs. The detailed information in multiple historical photographs of the person will inevitably be more abundant than the detail in the current photo taken by the phone. Therefore, the optimized photo will have richer detail and a clearer image of the person. This means that even in unfavorable environmental conditions, users can obtain photos with good image quality, thus improving the user experience.

[0010] In another possible implementation of the first aspect, before the electronic device receives the first photo in response to the photo-taking operation, the method further includes: the electronic device responding to the user's triggering operation on the camera application icon, displaying a camera preview interface, and determining in real time whether the camera preview interface includes a face image; wherein the camera preview interface is used to preview the shooting screen of the electronic device's camera; wherein whether the first photo includes a face image is the latest determination result of whether the camera preview interface includes a face image before the electronic device receives the photo-taking operation.

[0011] Based on the above scheme, before taking the first photo, the electronic device determines in real time whether the camera preview interface includes a face image. Subsequently, when the first photo is obtained, it can directly determine whether the first photo contains a face image, and thus decide whether to execute subsequent photo optimization schemes. In this way, because the electronic device does not need to determine whether the photo contains a face image after taking the photo, but can know whether the photo contains a face image when it is taken, it saves the step of determining whether a face image is included in the post-photo optimization process. Therefore, this scheme improves the efficiency of the photo optimization scheme.

[0012] In another possible implementation of the first aspect, the electronic device acquires the first photo by: the electronic device determining the first target photo as the first photo in response to a user's optimization operation on the first target photo in the gallery display interface.

[0013] Based on the above scheme, when a user instructs the electronic device to optimize a target photo in the local image library (i.e., to perform optimization on the target photo), it can first obtain a 3D model of the person to whom the face image in the photo belongs. Then, using this 3D model, the image of the person in the photo is optimized to obtain an optimized photo. This is because the 3D model represents the 3D features of the person in the corresponding historical photos. The detailed information of the person in multiple historical photos will inevitably be more abundant than the detail shown in the photo currently taken by the electronic device. Therefore, the optimized photo will have richer detail and a clearer image of the person. Thus, even if the target photo stored on the electronic device was taken in an unfavorable environment, subsequent optimization by the electronic device can still produce a photo with better image quality, thereby improving the user experience.

[0014] In another possible implementation of the first aspect, before the electronic device acquires the first three-dimensional model matching the first feature information, the method further includes: the electronic device acquiring at least one set of historical photos prior to the current moment; wherein each set of historical photos is associated with a person, and different sets of historical photos are associated with different people; each historical photo in the set of historical photos includes an image of the person associated with the set of historical photos; at least one set of historical photos includes a first set of historical photos associated with the first person; the electronic device determines the three-dimensional features of the person associated with the set of historical photos based on the set of historical photos; wherein the three-dimensional features include at least three-dimensional facial features; the electronic device constructs a three-dimensional model of the person associated with the set of historical photos based on the three-dimensional features of the person associated with the set of historical photos.

[0015] Based on the above solution, electronic devices can pre-build 3D models of the individuals in historical photos, making it convenient to use them when optimizing the photos later. This allows electronic devices to quickly and easily obtain the necessary 3D models when optimizing photos taken or stored, improving the efficiency of photo optimization.

[0016] In another possible implementation of the first aspect, the electronic device determines the three-dimensional features of the person corresponding to the historical photo set based on the historical photo set, including: the electronic device selecting at least one second target photo from the historical photo set based on the imaging quality parameters of each historical photo in the historical photo set; the imaging quality parameters include: resolution, the completeness of the image of the person associated with the historical photo set; the imaging quality parameters of at least one second target photo are higher than the imaging quality parameters of other photos in the historical photo set; the electronic device extracts the three-dimensional features of the person associated with the historical photo set from the second target photo.

[0017] Because historical photographs with poor image quality will inevitably have poor image quality for the people depicted, extracting features from each historical photograph is a waste of the electronic device's computing resources. Therefore, in the technical solution provided in this application, the electronic device can select a second target photograph with higher image quality parameters from the set of historical photographs as the basis for extracting 3D features. In this way, the electronic device can extract the 3D features of the people associated with the set of historical photographs with minimal computing resources, thus saving the electronic device's computing resources.

[0018] In another possible implementation of the first aspect, after the electronic device acquires the first photograph, the method further includes: the electronic device adding the first photograph to a first historical photograph set associated with the first person to update the first historical photograph set and obtain a second historical photograph set; the electronic device redetermining the three-dimensional features of the first person based on the second historical photograph set; and the electronic device reconstructing the three-dimensional model of the first person based on the redetermined three-dimensional features of the first person.

[0019] Based on the above scheme, after optimizing the image of a person in a photograph (e.g., the image of the first person), the electronic device can add the photograph (e.g., the first photograph) as a historical photograph to the historical photograph set associated with that person. Then, the updated historical photograph set associated with that person can be used to obtain an updated 3D model of that person, resulting in better optimization effects when subsequent image optimization of that person is needed.

[0020] In another possible implementation of the first aspect, when the electronic device includes a face image in the first photograph, after obtaining the first feature information of the first person to whom the first face image in the first photograph belongs, the method further includes: when the electronic device does not obtain the first three-dimensional model, constructing a first set of historical photographs associated with the first person using the first photograph; determining the three-dimensional features of the first person based on the first set of historical photographs; and constructing a three-dimensional model of the first person based on the three-dimensional features of the first person.

[0021] In some embodiments, if the electronic device has not previously photographed or stored the first person's 3D features or 3D model, it will be unable to optimize the image of the first person in the first photograph. Therefore, to subsequently optimize the image of the first person in photos taken or stored by the mobile phone, based on the aforementioned technical solution, the electronic device can utilize the current first photograph to establish a set of first historical photos associated with the first person, obtain the 3D features of the first person, and thus obtain a 3D model of the first person. This facilitates the subsequent optimization of the image of the first person by the electronic device, enabling it to easily obtain the 3D model of the first person and improve the user experience.

[0022] In another possible implementation of the first aspect, the electronic device acquires a first three-dimensional model that matches the first feature information, including: the electronic device acquires a first three-dimensional feature that matches the first feature information; the first three-dimensional feature is the three-dimensional feature of the first person in the historical photo corresponding to the first person; the electronic device uses the first three-dimensional feature to construct the first three-dimensional model.

[0023] In practice, directly matching the first feature information with multiple 3D models to find the first 3D model requires first obtaining comparable feature information from the 3D models before matching can be performed, making the matching process cumbersome and complicated. Therefore, to improve matching efficiency and thus image optimization efficiency, based on the above-mentioned solution, in this application, the electronic device can first obtain the first 3D feature matched by the first feature information, and then construct the first 3D model based on the first 3D feature.

[0024] In another possible implementation of the first aspect, when the electronic device includes a face image in the first photograph, before obtaining the first feature information of the first person to whom the first face image in the first photograph belongs, the method further includes: the electronic device obtaining at least one set of historical photographs prior to the current moment; wherein each set of historical photographs is associated with a person, and different sets of historical photographs are associated with different people; each historical photograph in the set of historical photographs contains an image of the person associated with the set of historical photographs; at least one set of historical photographs includes a first set of historical photographs associated with the first person; the electronic device determines the three-dimensional features of the person associated with the set of historical photographs based on the set of historical photographs; wherein the three-dimensional features include at least three-dimensional facial features.

[0025] The aforementioned timely solution allows electronic devices to pre-determine the 3D features of each person associated with a historical photograph. This facilitates the subsequent optimization of the photograph by readily acquiring these initial 3D features and constructing a first 3D model. Consequently, electronic devices can quickly obtain the necessary 3D model when optimizing photographs, significantly improving the efficiency of photo optimization.

[0026] In another possible implementation of the first aspect, after the electronic device acquires the first photograph, the method further includes: the electronic device adding the first photograph to a first historical photograph set associated with the first person to update the first historical photograph set and obtain a second historical photograph set; the electronic device redetermining the three-dimensional features of the first person based on the second historical photograph set.

[0027] Based on the above technical solution, after optimizing the image of a person in a photograph (e.g., the image of the first person), the electronic device can add the photograph as a historical photo to the historical photo set corresponding to that person. This results in the updated 3D features of the person, allowing for the construction of a more accurate 3D model when further optimizing the image of that person in a photograph, thus improving the photo optimization effect.

[0028] In another possible implementation of the first aspect, when the electronic device includes a face image in the first photograph, after obtaining the first feature information of the first person to whom the first face image in the first photograph belongs, the method further includes: when the electronic device does not obtain the first three-dimensional model, constructing a first set of historical photographs associated with the first person using the first photograph; and determining the three-dimensional features of the first person based on the first set of historical photographs.

[0029] Based on the above technical solution, when an electronic device cannot construct a corresponding 3D model because it cannot obtain the 3D features of a person in the first photo (i.e., the first person), it can use the first photo to establish a corresponding set of historical photos, thereby obtaining the 3D features. This allows the electronic device to successfully obtain the 3D features and construct a corresponding 3D model when it needs to optimize the image of that person in a newly obtained photo, thus completing the photo optimization process and improving the user experience.

[0030] In another possible implementation of the first aspect, after the electronic device acquires the first photograph, the method further includes:

[0031] When the first photograph includes a face image, the electronic device acquires second feature information of a second person to whom the second face image belongs; wherein the second person is different from the first person, and the second feature information includes at least the face feature information of the second person; the electronic device acquires a second three-dimensional model that matches the second feature information; wherein the second three-dimensional model is used to characterize the three-dimensional features of the second person in the historical photograph corresponding to the second person; the electronic device fuses the second three-dimensional model with the image of the second person in the first photograph to optimize the image of the second person in the first photograph, thereby obtaining a third photograph.

[0032] Based on the above technical solution, the electronic device can optimize each person image in the first photo using the corresponding 3D model. Therefore, the optimized photo will have richer details and clearer images of the people. This way, even if the target photo stored on the electronic device was taken in an unfavorable environment, subsequent optimization by the electronic device can still produce a photo with better image quality, thus improving the user experience.

[0033] In another possible implementation of the first aspect, the method further includes: the electronic device merging the first photograph and the fourth photograph to obtain a fifth photograph; wherein the fourth photograph is obtained by the electronic device after optimizing the image of the target person in the first photograph; wherein the target person includes the first person and / or the second person.

[0034] In some embodiments, the unoptimized target photo stored by the electronic device is directly captured, so the transition between the image of the person and its background image (i.e., the image other than the person) appears smooth. However, the optimized second photo (or third photo, or a combination of the second and third photos), because the image of the person is optimized, may result in a more noticeable difference between the image of the person and the background image, creating a sense of disjointedness. Therefore, to avoid this disjointedness, based on the aforementioned technical solution, the electronic device can merge the first and fourth photos to obtain a fifth photo. Because the final processed fifth photo is obtained by merging the original first photo and the optimized fourth photo, it not only has a clear and detailed image of the person, but also eliminates the sense of disjointedness between the image of the person and the background image, resulting in a more harmonious overall image. Consequently, the user experience is improved, resulting in a better viewing experience.

[0035] In a second aspect, this application provides an electronic device including a camera, a display screen, a memory, and one or more processors; the camera, display screen, memory, and processors are coupled together; wherein the memory stores computer program code, which includes computer instructions, and when the computer instructions are executed by the processor, the electronic device performs an image processing method as provided in the first aspect and any possible design thereof.

[0036] Thirdly, this application provides a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform an image processing method as provided in the first aspect and any of its possible design embodiments.

[0037] Fourthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the image processing method provided by the first aspect and any possible design of the first aspect. The computer may be the aforementioned electronic device.

[0038] It is understood that the beneficial effects achieved by the electronic device described in the second aspect and any possible design thereof, the computer-readable storage medium described in the third aspect, and the computer program product described in the fourth aspect can be referred to the beneficial effects in the first aspect and any possible design thereof, which will not be repeated here. Attached Figure Description

[0039] Figure 1 A schematic diagram illustrating the implementation environment of an image processing method provided in this application embodiment;

[0040] Figure 2This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0041] Figure 3 This application provides an example of an optimized process for taking photos. Figure 1 ;

[0042] Figure 4 A schematic diagram of a camera preview interface provided in an embodiment of this application;

[0043] Figure 5 This is a schematic diagram of a scene for starting a camera, provided as an embodiment of this application;

[0044] Figure 6 This application provides an example of an optimized process for taking photos. Figure 2 ;

[0045] Figure 7 This application provides an example of an optimized process for taking photos. Figure 3 ;

[0046] Figure 8 This application provides an example of an optimized process for taking photos. Figure 4 ;

[0047] Figure 9 This application provides an example of an optimized process for taking photos. Figure 5 ;

[0048] Figure 10 An optimized photographic illustration provided for an embodiment of this application;

[0049] Figure 11 A schematic diagram illustrating the comparison of a photo before and after optimization, provided as an embodiment of this application;

[0050] Figure 12 This application provides an example of an optimized process for taking photos. Figure 6 ;

[0051] Figure 13 This application provides a schematic diagram comparing photos before and after optimization.

[0052] Figure 14 This application provides an example of an optimized process for taking photos. Figure 7 ;

[0053] Figure 15 An optimized process for storing photos is provided as an embodiment of this application. Figure 1 ;

[0054] Figure 16This application provides a schematic diagram of a scene for determining a first photograph.

[0055] Figure 17 An optimized process for storing photos is provided as an embodiment of this application. Figure 2 ;

[0056] Figure 18 An optimized process for storing photos is provided as an embodiment of this application. Figure 3 ;

[0057] Figure 19 A schematic diagram of an optimized information generation process provided in this application embodiment. Figure 1 ;

[0058] Figure 20 This application provides a schematic diagram of a photo classification scenario in a gallery application.

[0059] Figure 21 A schematic diagram of an optimized information generation process provided in this application embodiment. Figure 2 ;

[0060] Figure 22 A schematic diagram of an optimized information generation process provided in this application embodiment. Figure 3 ;

[0061] Figure 23 This application provides an example of an optimized process for taking photos. Figure 8 ;

[0062] Figure 24 An optimized process for storing photos is provided as an embodiment of this application. Figure 4 ;

[0063] Figure 25 This application provides an example of an optimized process for taking photos. Figure 9 ;

[0064] Figure 26 An optimized process for storing photos is provided as an embodiment of this application. Figure 5 ;

[0065] Figure 27 This application provides an example of an optimized process for taking photos. Figure 10 ;

[0066] Figure 28 This application provides an example of an optimized process for taking photos. Figure 10 one;

[0067] Figure 29 An optimized process for storing photos is provided as an embodiment of this application. Figure 6 ;

[0068] Figure 30 An optimized process for storing photos is provided as an embodiment of this application. Figure 7 ;

[0069] Figure 31 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application. Detailed Implementation

[0070] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that “ / ” means “or,” for example, A / B can mean A or B; “and / or” in the text is merely a description of the relationship between related persons, indicating that three relationships can exist, for example, A and / or B can mean: A alone, A and B simultaneously, and B alone.

[0071] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0072] The terms "first" and "second" in the following embodiments of this application are for descriptive purposes only and should not be construed as implying relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0073] Currently, most electronic devices have camera functions. However, in some unfavorable shooting scenarios, such as low light or backlight, the image quality of portraits in photos taken by users of electronic devices is not high, resulting in a poor user experience.

[0074] To address the aforementioned problems, this application provides a photo optimization method applicable to electronic devices with camera functions. This method can also be referred to as an image processing method. In this method, the electronic device can obtain a three-dimensional model of the image of a person in the photo based on the first feature information, such as facial feature information. This three-dimensional model represents the three-dimensional features of the person in the historical photograph (i.e., a historical photograph containing images of the person corresponding to the image in the photo). Then, the electronic device can fuse the three-dimensional model of the person with the image of the person to optimize the photo, obtaining an optimized photo. In this way, because the portrait in the photo can be optimized using historical photographs, the portrait in the photo becomes clearer and more three-dimensional, improving the user experience.

[0075] A schematic diagram illustrating the possible implementation environments of the technical solutions provided in this application can be found in the embodiments. Figure 1 As shown. In this implementation environment, it may include an electronic device 01 and a server 02. The electronic device 01 and the server 02 communicate via wired or wireless communication. In this embodiment, there is a certain correspondence between the electronic device 01 and the server 02. For example, when the electronic device is a product of Honor Terminal Co., Ltd., the server 02 should be configured by Honor Terminal Co., Ltd. to store the data that the electronic device needs to upload and store, such as photos taken by the electronic device, three-dimensional features determined by the electronic device based on historical photos, or three-dimensional models. The server 02 can also generate data that the electronic device may need later based on the data uploaded and stored by the electronic device. For example, the server 02 can use the photos uploaded by the electronic device to generate three-dimensional features corresponding to a certain person, or further generate a three-dimensional model corresponding to a certain person.

[0076] For example, server 02 in this application can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center; this application does not limit this. In this application, server 02 is mainly used for storing photos taken by electronic devices and / or, based on historical photos, the three-dimensional features determined by the electronic devices, and / or, based on historical photos, the three-dimensional models. When the electronic device executes the technical solution provided in this application, server 02, upon receiving a request from electronic device 01 to obtain data, can send the requested data to electronic device 01.

[0077] For example, the electronic device in this application embodiment may be a mobile phone, tablet computer, wearable device (such as a smartwatch or smart bracelet), ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, navigation device, mobile internet device (MID), or wearable device, etc., that can take pictures. This application embodiment does not impose any special restrictions on the specific form of the electronic device.

[0078] Take mobile phones as an example of electronic devices. Figure 2 A schematic diagram of the structure of the electronic device 100 provided in this application is shown.

[0079] Reference Figure 2 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a display screen 193, a subscriber identification module (SIM) card interface 194, and a camera 195, etc. The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0080] The processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. In this application, the AP and GPU work together to determine whether a photograph in an electronic device includes a face image and extract relevant feature information, such as facial feature information or body posture feature information; furthermore, it can determine three-dimensional features or a three-dimensional model matching the face image.

[0081] Electronic device 100 implements display functions through a GPU, a display screen 193, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 193 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0082] The display screen 193 is used to display images, videos, etc. This display screen may be a touchscreen. In some embodiments, the electronic device 100 may include one or N display screens 193, where N is a positive integer greater than 1.

[0083] Electronic device 100 can implement shooting functions through an ISP, camera 195, video codec, GPU, display 193, and application processor. The ISP is used to process data fed back by the camera 195. The camera 195 is used to capture still images or videos. In some embodiments, electronic device 100 may include one or N cameras 195, where N is a positive integer greater than 1.

[0084] An NPU (Neural Processing Unit) is a neural network (NN) computing processor that, by borrowing from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, rapidly processes input information and can continuously learn on its own. NPUs can enable intelligent cognitive applications in electronic devices, such as screen protector status recognition, image restoration, image recognition, face recognition, speech recognition, and text understanding. In this application, the NPU can generate corresponding 3D models using the 3D features acquired by the electronic device. The collaboration between the GPU and NPU allows for the fusion and optimization of images of people in photographs using the 3D models acquired by the electronic device.

[0085] A controller can be the nerve center and command center of an electronic device. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0086] The processor 110 may also include a memory for storing instructions and data, such as photographs taken by the electronic device, 3D features obtained from the photographs, and even 3D models. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or is reusing. If the processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0087] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0088] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0089] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 110 and can be used to store executable programs (e.g., machine instructions) of the operating system or other running programs, as well as user and application data. The NVM can also store executable programs and user and application data, and can be pre-loaded into the RAM for direct read and write operations by the processor 110.

[0090] A touch sensor, also known as a "touch device," can be located on the display screen 193. The touch sensor and the display screen 193 together form a touchscreen, also called a "touchscreen." The touch sensor detects touch operations applied to or near it. The touch sensor can then transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 193. In other embodiments, the touch sensor may also be located on the surface of the electronic device, in a different position than the display screen 193.

[0091] In this application, the touch sensor can detect the user's operation on the mobile phone display screen, and the processor 110 analyzes and executes the function corresponding to the operation, such as turning off the screen or turning on the screen.

[0092] A pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor may be located on the display screen 193. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to the pressure sensor, the capacitance between the electrodes changes. The electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to the display screen 193, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor. The electronic device 100 may also calculate the touch position based on the detection signal from the pressure sensor. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities may correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS message is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS message is executed.

[0093] Button 190 includes a power button, volume buttons, etc. Users can control the phone to turn on or off by pressing and holding the power button, and users can control the playback volume by pressing the volume buttons.

[0094] Motor 191 can generate vibration alerts. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 194 is used to connect a SIM card.

[0095] Of course, it is understandable that the above... Figure 2 The illustration shown is merely an example when the electronic device is in the form of a mobile phone. If the electronic device is in the form of a tablet, handheld computer, PC, PDA, wearable device (such as a smartwatch, smart bracelet), or other similar device, the structure of the electronic device may include more advanced features. Figure 2 The fewer structures shown can also include more than Figure 2 The structures shown are not limited here.

[0096] Understandably, the photos requiring optimization can be taken by an electronic device using a camera application, or they can be unoptimized photos stored in the user's local gallery. Therefore, in the photo optimization method provided in this application, the electronic device can optimize photos taken directly by the device, or it can optimize photos already stored in the user's gallery. Furthermore, the photo optimization process also requires the use of pre-stored optimization information (3D features or 3D models). Therefore, the photo optimization method provided in this application can include a "photo optimization" process, an "optimization information generation" process, and a "stored photo optimization" process.

[0097] The method flows in the following embodiments can all be implemented in an electronic device with the above-described hardware architecture.

[0098] The following uses a mobile phone as an example to describe the photo optimization process in the photo optimization method provided in this application embodiment. For example... Figure 3 As shown, the optimization process for this captured image may include steps S301-S307:

[0099] S301, The mobile phone receives the user's trigger operation on the camera application icon.

[0100] When a user needs to take a photo with their phone, they can tap the camera app icon on their phone's home screen to trigger the camera to start taking pictures.

[0101] S302. In response to the user's triggering operation on the camera application icon, the phone starts the camera and displays the camera preview interface.

[0102] In practice, to ensure that users can clearly see the content within the camera's shooting area when the camera app is opened, the phone displays the image captured by the camera in real-time on the camera preview screen. In other words, the camera preview screen is used to preview the image captured by the phone's camera.

[0103] Most mobile phones currently have a rear camera and a front camera. The front camera is used to capture the area directly in front of the phone screen, while the rear camera captures the area directly behind the phone. The camera activated by the phone as mentioned in S302 of this application can be a front camera, a rear camera, or both. Which camera is activated depends on the camera application settings or the settings from a previous shooting session; this application does not impose specific limitations on this.

[0104] Reference Figure 4As shown in (a), when the camera activated by the mobile phone is the front-facing camera, the camera preview interface displays the image captured by the front-facing camera, which can be referred to as the foreground image in this application.

[0105] Reference Figure 4 As shown in (b), when the camera activated by the mobile phone is the rear camera, the camera preview interface displays the image captured by the rear camera, which can be referred to as the background image in this application.

[0106] Reference Figure 4 As shown in (c), when the phone's cameras are both the front and rear cameras, the camera preview interface is divided into two areas, a first area 401 and a second area 402, at a certain ratio (e.g., 1:1). The first area 401 displays the foreground image captured by the front camera, while the second area 402 displays the background image captured by the rear camera. Of course, Figure 4 The arrangement of the first and second regions in the camera preview interface shown in (c) is merely an example; in practice, any other feasible arrangement is possible, such as picture-in-picture. This application does not impose any specific limitations on this arrangement.

[0107] For example, if a phone activates its rear camera in response to a user's action on the camera app icon, the phone can display something like this: Figure 5 The desktop 501 shown in (a) includes a camera application icon 502. Specifically, S301 and S302 are implemented as follows: The mobile phone can receive a user's trigger operation (e.g., a click operation) on the camera application icon 502. In response to the trigger operation on the camera application icon 502, the mobile phone can activate its rear camera to take a picture and display the following... Figure 5 The camera preview interface 503 shown in (b) is shown in the middle.

[0108] S303, The mobile phone receives the photo-taking operation performed on the camera preview interface.

[0109] For example, consider a scenario where the phone activates the rear camera in response to a user's interaction with the camera app icon. (See also...) Figure 5 As shown in (b), the camera preview interface 503 may include a photo-taking control 504. This photo-taking control 504 is used to trigger the mobile phone to capture the currently displayed image on the camera preview interface 503 and generate a photo. The photo-taking operation can be a trigger operation for the photo-taking control 504, such as a click operation.

[0110] S304: The mobile phone responds to the user's photo-taking action and acquires the first photo.

[0111] For example, refer to Figure 5As shown in (b), the image content in the first photo can specifically be the screen displayed in the camera preview interface 503 when the user performs the photo-taking operation.

[0112] S305. When the first photo includes a face image, the mobile phone obtains the first feature information of the first person to whom the first face image in the first photo belongs.

[0113] The main purpose of the photo optimization method provided in this application is to optimize the image of people in a photo, and the most important part of a person's image is their face. Therefore, in this application, after the mobile phone responds to the user's photo-taking operation and obtains the first photo, it will only optimize the photo if it is confirmed that the first photo includes a face image, that is, it will execute subsequent steps S306 and S307. Here, the first face image is one of the face images in the first photo.

[0114] The specific method by which the mobile phone determines whether the first photo includes a facial image can be any feasible method. For example, the mobile phone can determine whether the first photo includes a facial image based on whether there is a facial feature in a preset area (which can be determined by the resolution of the first photo) that meets preset relative position conditions. Specifically, if the mobile phone determines that there is a facial feature in a preset area (which can be determined by the resolution of the first photo) that meets preset relative position conditions, then the first photo is determined to include a facial image; if the mobile phone determines that there is no facial feature in a preset area (which can be determined by the resolution of the first photo) that meets preset relative position conditions, then the first photo is determined not to include a facial image.

[0115] For example, a mobile phone can use a pre-trained facial recognition model to identify different objects in a first photo to determine whether the first photo contains a human face. Of course, in practice, any other feasible implementation method can be used.

[0116] The biggest difference between different images of people lies in their facial information; facial features can distinguish different images of people to the greatest extent. Based on this, the first feature information in this application includes at least the facial feature information of the first person. For example, specific features in the facial feature information may include at least one of the following: facial detail information, skin color, and skin texture. Among these, facial detail information may include information such as the relative positions and sizes of facial features.

[0117] Furthermore, to more accurately distinguish different human images and obtain a more accurate 3D model subsequently, the first feature information here can also include the body posture features of the first human figure. Specific features in the body posture features may include, but are not limited to, at least one of the following: limb shape and body proportions.

[0118] Specifically, the mobile phone can extract the first feature information of the first person's image from the first photo in any feasible way.

[0119] In addition, if the mobile phone determines that the first photo does not contain a human face image, it will store the first photo normally and stop executing the technical solution provided in the embodiments of this application.

[0120] In some embodiments, to enable the mobile phone to more quickly determine whether a photo requires image optimization of a person after it has been captured, the phone can detect in real time whether the image in the camera preview interface includes a face before the user takes the photo. This way, when the phone subsequently takes a photo in response to the user's action, it can determine whether the first photo contains an image of a person, and thus decide whether to perform subsequent operations. Based on this, combined with... Figure 3 , refer to Figure 6 As shown, step S302 can be replaced by S302A:

[0121] The S302A mobile phone responds to the user's trigger operation on the camera application icon, displays the camera preview interface, and determines in real time whether the camera preview interface includes a face image.

[0122] For details on how a mobile phone determines whether a face image is included in the camera preview interface, please refer to the aforementioned description of how a mobile phone determines whether a face image is included in the first photo; these details will not be repeated here.

[0123] Whether the first photo includes a human face image is the result of the latest determination of whether the camera preview interface includes a human face image before the electronic device receives the photo-taking operation.

[0124] Based on this, combined Figure 3 , refer to Figure 6 As shown, step S305 can be replaced by S305A:

[0125] S305A: When the mobile phone obtains the target result, it obtains the first feature information of the first person to whom the first face image belongs.

[0126] The target result is the latest determination result of whether the camera preview interface includes a face image before the mobile phone receives the photo-taking operation, and the target result indicates that the camera preview interface includes a face image.

[0127] Specifically, each time the phone determines whether a face image is present in the camera preview, it stores the result in a specific storage area. This result is then retrieved when the phone takes a photo. Furthermore, to conserve storage space, the specific storage area can store only the most recent result. In other words, each time the phone determines whether a face image is present in the camera preview, it replaces the result stored in the specific storage area with the newer one.

[0128] Based on the technical solutions corresponding to S302A and S305A, the mobile phone can determine whether the first photo includes a human face image while responding to the user's photo-taking operation and acquiring the first photo, thus improving the efficiency of photo optimization.

[0129] S306. The mobile phone acquires a first three-dimensional model that matches the first feature information.

[0130] The first 3D model is used to represent the 3D features of the first person in the historical photograph corresponding to the first person's image. These 3D features include at least facial 3D features. All historical photographs corresponding to the first person's image include an image of the first person. To further optimize the first person's posture in the image, the 3D features can also include posture 3D features. This way, when the first 3D model is used to fuse and optimize the first person's image, the details of the first person's face and body in the image can be clearer, resulting in a better optimization effect.

[0131] Specifically, for multiple historical photos, the mobile phone can first use facial recognition technology to identify the faces and related features in each photo. Then, through feature comparison, it can identify faces belonging to the same person. Based on this, multiple historical photos can be identified as corresponding to different individuals. Historical photos belonging to the same person will all include an image of that person. Of course, for historical photos containing multiple individuals, each historical photo can correspond to multiple individuals.

[0132] In one possible approach, the mobile phone can pre-determine and store a first 3D model based on historical photos corresponding to the first person's image stored locally. During step S306, the mobile phone can retrieve this first model from its local storage. This allows for faster retrieval of the first 3D model, improving the efficiency of photo optimization. For details on how to determine the 3D model based on historical photos, please refer to the relevant descriptions in the subsequent optimization information generation process; these will not be elaborated upon here.

[0133] For example, in this case, the process of optimizing the photos taken by the phone can be as follows: Figure 7 As shown, before a user takes a photo, the phone uses historical photos from its gallery to construct 3D models of multiple people and stores them in a corresponding database, such as a 3D portrait database.

[0134] Subsequently, when the user takes a photo, the phone will retrieve the 3D model (e.g., the first 3D model) corresponding to the person in the photo (e.g., the first person) from the 3D model database.

[0135] Then, the phone can use the acquired 3D model to optimize the image of the person in the photo, resulting in the final optimized photo.

[0136] In another possible approach, the mobile phone can obtain the first 3D model from the server. Based on this, combined with... Figure 3 , refer to Figure 8 As shown, S306 may specifically include: S3061A and S3062A:

[0137] S3061A, the mobile phone sends the first request to the server.

[0138] The first request carries first feature information and is used to request a first three-dimensional model that matches the first feature information.

[0139] In this application, the server can be a server associated with the mobile phone. For example, if the mobile phone is manufactured by Honor Terminal Co., Ltd., then the server can be a server configured by Honor Terminal Co., Ltd. for the mobile phone to store the data that the electronic device needs to upload. The server can allocate different storage areas for different mobile phones to store the data uploaded by the corresponding mobile phones. Each storage area can correspond to a unique account and password set by the user. The user can use their own set account and password to log in to the application on the mobile phone for uploading data to the server, and then use the application to upload the data to be uploaded to the storage area configured by the server for the mobile phone.

[0140] S3062A: The mobile phone receives the first response from the server and obtains the first 3D model from the first response.

[0141] In this application, the server may actually store one or more of the following: all historical photos of the first person, the three-dimensional features of the first person determined based on the historical photos (i.e., the first three-dimensional features mentioned in this application), and a first three-dimensional model obtained based on the three-dimensional features of the first person. For details on how to determine the three-dimensional features of the first person based on the historical photos, and how to obtain the first three-dimensional model based on the three-dimensional features of the first person, please refer to the relevant descriptions in the subsequent optimization information generation process; these will not be described in detail here. The specific content stored on the server may be factory-set or user-defined.

[0142] Under normal circumstances, the mobile phone only uploads historical photos (including historical photos corresponding to the first person) to the server. After receiving the historical photos, the server can generate corresponding 3D features or 3D models according to factory settings or user settings and store them for subsequent use by the mobile phone.

[0143] If the server stores all historical photos corresponding to the first person's image, upon receiving a first request from the mobile phone, the server can determine the first person's 3D features based on all the historical photos. Then, the server can determine a first 3D model based on these features. Afterward, the server can send the first response to the mobile phone.

[0144] The specific process of determining all historical photos corresponding to the first person can involve matching the first feature information with the images of the person in all historical photos. Generally, during the matching process, the feature information of the person in the historical photos is first extracted, and then matched with the first feature information. Afterwards, historical photos with a matching degree greater than a certain threshold are identified as the historical photos corresponding to the first person.

[0145] If the server stores the first three-dimensional features corresponding to the first person, upon receiving the first request from the mobile phone, the server will determine the first three-dimensional model based on these features. The server can then send the first response to the mobile phone.

[0146] Specifically, the process of determining the first three-dimensional feature corresponding to the first person can involve matching the first feature information with all three-dimensional features stored in the server. Then, the three-dimensional features with a matching degree greater than a certain threshold are determined as the first three-dimensional features corresponding to the first person.

[0147] If the server stores the first 3D model corresponding to the first person's image, the server will directly send the first response to the mobile phone upon receiving the first request from the mobile phone.

[0148] Specifically, determining the first 3D model corresponding to the first person can be achieved by matching the first feature information with all 3D models stored on the server. Generally, during the matching process, the 3D feature information of the person in the 3D model is first extracted and then matched with the first feature information. Afterwards, the 3D model with a matching degree greater than a certain threshold is determined as the first 3D model corresponding to the first person.

[0149] Based on the technical solutions corresponding to S3061A and S3062A, on the one hand, the first 3D model or its related information is stored on the server, reducing the storage and computing requirements of the mobile phone for the photo optimization solution provided in this application. On the other hand, because the 3D model or its related information is stored on the server, and the server is associated with the mobile phone, when a user changes to a new mobile phone or clears historical photos, 3D features, or 3D models stored locally, the user can easily retrieve the first 3D model from the server based on the association between the mobile phone and the server. This facilitates photo optimization of the first photo using the user's current mobile phone, improving the user experience.

[0150] For example, in the technical solutions corresponding to S3061A and S3062A, the process of optimizing the photos taken by the mobile phone can be referred to... Figure 7 As shown. The difference lies in that historical photos and / or 3D features and / or 3D models are all stored on the server. Moreover, if the server does not directly store the 3D model, upon receiving the first request, the server will first generate a 3D model based on the stored historical photos or 3D features corresponding to the first person. Then, the server sends a first response carrying the first 3D model to the mobile phone.

[0151] In some embodiments, matching the first feature information with historical photos or 3D models can be cumbersome, requiring the acquisition of comparable feature information from historical photos or 3D models before matching can proceed. Therefore, to improve matching efficiency and thus photo optimization efficiency, in this application, the mobile phone can first acquire the first 3D feature matching the first feature information, and then construct the first 3D model based on that first 3D feature. Based on this, combined with... Figure 3 , refer to Figure 9 As shown, S306 may include S3061B and S3062B:

[0152] S3061B, The mobile phone acquires the first three-dimensional feature that matches the first feature information.

[0153] The first three-dimensional feature is the three-dimensional feature of the first person in the historical photograph corresponding to the first person. This three-dimensional feature may include facial three-dimensional features and / or body posture three-dimensional features.

[0154] In one feasible approach, the mobile phone can pre-determine and store the first 3D feature based on historical photos corresponding to the first person stored locally. During step S3061B, the mobile phone can retrieve this first feature from local storage. This allows for faster acquisition of the first 3D feature, improving the efficiency of photo optimization. For details on how to determine the 3D model based on historical photos, please refer to the relevant descriptions in the subsequent optimization information generation process; they will not be elaborated here.

[0155] For example, in this case, the process of optimizing the photos taken by the mobile phone can be referred to Figure 7 As shown. The difference is that the human face 3D database stores 3D features.

[0156] In another possible implementation, the first three-dimensional feature can be stored in a server corresponding to the mobile phone, and the mobile phone can obtain the first three-dimensional model from the server. Based on this, S3061B can specifically include: S1 and S2:

[0157] S1. The mobile phone sends a second request to the server.

[0158] The second request carries first feature information and is used to request a first three-dimensional feature that matches the first feature information.

[0159] S2. The mobile phone receives the second response from the server and obtains the first three-dimensional feature from the second response.

[0160] In this application, the server may actually store one or more of the following: all historical photos of the first person, and the first three-dimensional feature determined based on the historical photos. For details on how to determine the three-dimensional feature of the first person based on the historical photos, please refer to the relevant descriptions in the subsequent optimization information generation process; these will not be elaborated here. The specific content stored on the server may be factory-set or user-defined.

[0161] Under normal circumstances, the mobile phone only uploads historical photos (including historical photos corresponding to the first person's image) to the server. After receiving the historical photos, the server can generate corresponding 3D features according to factory settings or user settings for subsequent use by the mobile phone.

[0162] If the server stores all historical photos of the first person, upon receiving a second request from the mobile phone, the server can determine the first three-dimensional feature based on all the historical photos of that first person. The server can then send the second response to the mobile phone.

[0163] The specific process of determining all historical photos corresponding to the first person can involve matching the first feature information with the images of the person in all historical photos. Generally, during the matching process, the feature information of the images of the person in the historical photos is first extracted, and then matched with the first feature information. Afterwards, historical photos with a matching degree greater than a certain threshold are identified as the historical photos corresponding to the first person.

[0164] If the server stores the first three-dimensional features corresponding to the first person, the server can directly send the second response to the mobile phone.

[0165] Specifically, the process of determining the first three-dimensional feature corresponding to the first person can involve matching the first feature information with all three-dimensional features stored in the server. Then, the three-dimensional features with a matching degree greater than a certain threshold are determined as the first three-dimensional features corresponding to the first person.

[0166] For example, in the technical solutions corresponding to S1 and S2, the process of optimizing the captured photos by the mobile phone can be referred to... Figure 7 As shown. The difference lies in that both the historical photos and / or 3D features are stored on the server. Furthermore, if the server does not directly store the 3D model, upon receiving the second request, the server will generate a first 3D feature based on the stored historical photos corresponding to the first person, and then send a second response carrying the first 3D feature to the mobile phone. Alternatively, upon receiving the second request, the server will generate a second response based on the stored first 3D feature and send that second response to the mobile phone.

[0167] The technical effects of the solutions based on S1 and S2 are similar to those based on S3061A and S3062A, and will not be elaborated further here. The difference lies in the fact that, in the solutions based on S1 and S2, the mobile phone needs to construct a first three-dimensional model based on the first three-dimensional features, which consumes certain computing resources. In contrast, the solutions based on S3061A and S3062A do not require the mobile phone to consume these computing resources.

[0168] S3062B and mobile phones utilize the first three-dimensional features to construct a first three-dimensional model.

[0169] The specific method for constructing the first three-dimensional model based on the first three-dimensional features can be based on any feasible three-dimensional reconstruction technology, and this application does not impose any specific restrictions on it.

[0170] Based on the technical solutions corresponding to S3061B and S3062B mentioned above, the mobile phone can more quickly match the first three-dimensional feature according to the first feature information, and further construct the first three-dimensional model. This provides data support for subsequent photo optimization.

[0171] S307. The mobile phone merges the first 3D model with the image of the first person in the first photo to optimize the image of the first person in the first photo and obtain the second photo.

[0172] After the phone receives the second photo, it can store it in the local gallery for the user to view. Simultaneously, to clearly demonstrate the phone's optimization effects, the phone can also store the first photo for easy comparison. Of course, to save storage space, the first photo can also be omitted.

[0173] If the phone only stores the second photo, this allows users to know which photos in their gallery have been optimized. For example, refer to... Figure 10 As shown, the optimized photo (e.g., the second photo) may contain optimization marker 101.

[0174] With both the first and second photos stored on the phone, this feature aims to allow users to see which photos in their gallery have been optimized and to quickly compare the unoptimized first photo with the optimized second photo. For example, refer to... Figure 11 As shown in (a), a refresh control 112 can be displayed on the second photo 111 in the phone's gallery. When a user sees the refresh control 112 and wants to compare the first and second photos, the user can trigger an operation on the refresh control 112, such as a click. The phone responds to the user's trigger operation on the refresh control 112, referring to... Figure 11 As shown in (b), the second photo 111 can be displayed in full screen, along with a prompt message 113. For example, the prompt message could specifically be "Please swipe left to view the photo before optimization".

[0175] Afterwards, the user can perform a left swipe. The phone responds to the user's left swipe as follows: Figure 11 As shown in (c), the second photo 111 is gradually slid off the screen from the left, while the first photo 114 is gradually slid into the screen from the right to the left. After the user finishes the left swipe operation, the phone can then... Figure 11 As shown in (d), the first photo 114 is displayed in full screen.

[0176] certainly, Figure 11 The scenarios shown in (a), (b), (c), and (d) are merely examples, and in practice, any other feasible implementation method can be used. For example, the mobile phone can respond to the user's trigger operation on the optimization control 112 and simultaneously display the first and second photos on the mobile phone screen.

[0177] Based on the aforementioned technical solutions S301-S307, when a photo including a face image is captured, the mobile phone can obtain a 3D model of the person to whom the face image in the photo belongs. This 3D model is then used to optimize the image of the person in the photo, resulting in an optimized photo. This is because the 3D model represents the 3D features of the person in historical photos. The detailed information of the person in multiple historical photos will inevitably be more abundant than the detail shown in the current photo taken by the mobile phone. Therefore, the optimized photo will have richer detail and a clearer image of the person. Thus, even in unfavorable environments, users can obtain photos with good image quality, improving the user experience.

[0178] In some embodiments, photos taken with a mobile phone may contain multiple people, so image optimization often requires optimizing the images of more than one person. Based on this, combined with... Figure 3 , refer to Figure 12 As shown, in the photo optimization process provided in this application embodiment, after step S304, steps S308-S310 may also be included:

[0179] S308. When the mobile phone determines that a second face image exists in the first photo, it obtains the second feature information of the second person to whom the second face image belongs.

[0180] The second face image differs from the first face image, and the second feature information includes at least the facial feature information of the second person. Furthermore, to more accurately distinguish between images of different people and obtain a more accurate 3D model subsequently, the second feature information may also include the body posture feature information of the second person.

[0181] S309. The mobile phone acquires a second three-dimensional model that matches the second feature information.

[0182] The second 3D model is used to represent the 3D features of the second person in the historical photographs corresponding to the first person; these 3D features include at least facial 3D features; and all historical photographs corresponding to the second person include an image of the second person. To further optimize the second person's posture, these 3D features can also include posture 3D features. This way, when the second 3D model is used to fuse and optimize the images of the second person, the details of the face and body in the second person's image can be clearer, resulting in a better optimization effect.

[0183] S310 The mobile phone merges the second 3D model with the image of the second person in the first photo to optimize the image of the second person in the first photo and obtain the third photo.

[0184] In this application, S308-S310 can be executed in parallel with S305-S307, or S305-S307 can be executed first and then S308-S310, or any other feasible execution order. This application does not impose specific restrictions on this. The specific implementation of S308-S310 can refer to the relevant descriptions of S305-S307 mentioned above, and will not be repeated here.

[0185] In this application, if S304-S307 and S308-S310 are executed after S304, the final optimized photo is a combination of the second and third photos. In the optimized photo, both the first and second person images have been optimized accordingly.

[0186] Of course, if the first photo contains other facial images besides the first and second facial images, the other facial images can be optimized according to the optimization process for the first and second facial images in the foregoing embodiments, so as to obtain a photo in which all facial images are optimized. The specific optimization process can be referred to the description in the foregoing embodiments, and will not be repeated here.

[0187] Based on the technical solutions corresponding to S308-S310 mentioned above, the phone can optimize each person image in a photo taken with the phone using a corresponding 3D model. Therefore, the optimized photo will have richer details and clearer images of the people. This means that even in unfavorable environments, users can still obtain photos with good image quality, thus improving the user experience.

[0188] In some embodiments, if the current shooting environment of the mobile phone is a suitable scene for taking pictures, rather than a scene unfavorable for taking pictures (such as a low-light scene, a backlight scene, etc.), then there may be no need to optimize the image of the person in the captured photo. Based on this, in the photo optimization process provided in this application, S305 may specifically be: when the mobile phone determines that the shooting environment of the first photo is a preset environment, if it determines that the first photo includes a face image, it obtains the first feature information of the first person image to which the first face image belongs.

[0189] Similarly, S308 can specifically be: when the mobile phone determines that the shooting environment of the first photo is a preset environment, and if it determines that the first photo includes a face image, it obtains the first feature information of the second person to whom the second face image belongs.

[0190] The same principles apply to similar steps in the subsequent optimization process for stored photos, and will not be repeated hereafter.

[0191] The preset environment can be a low-light environment or a backlit environment, etc. Specifically, the mobile phone can determine whether it is in a low-light environment or a backlit environment based on the ambient brightness of the environment in which the phone is located when the user takes a picture, and the brightness difference between the image of the person and the background image (images other than the image of the person) in the camera preview interface. For example, if the ambient brightness is determined to be less than a preset brightness value, the shooting environment of the first photo can be determined to be a low-light environment; if the difference between the brightness of the background image and the brightness of the image of the person in the camera preview interface is determined to be greater than a certain threshold, the shooting environment of the first photo can be determined to be a backlit environment. Of course, in practice, the determination of the shooting environment of the first photo can be any other feasible determination method, and this application does not impose specific restrictions on this.

[0192] After S304, if the phone determines that the shooting environment of the first photo is not the preset environment, it will not execute the subsequent steps of the photo optimization process. Instead, it will directly save the first photo. This is because the phone only optimizes the image of people in photos taken under unfavorable shooting conditions, while the image quality of people in photos taken under favorable conditions is already good enough. Therefore, this significantly reduces the waste of the phone's computing and storage resources while ensuring the image quality of people in the photos taken.

[0193] In some embodiments, refer to Figure 13 As shown in (a), the first photo 131, captured or stored by the mobile phone, is directly captured, so the transition between the image of the person and their background appears very smooth. Users will not experience any unusual sensation when viewing this first photo 131. (Referring to...) Figure 13 As shown in (b), in the optimized second photo (or third photo, or a combination of the second and third photos) 132, the optimization of the first person's image 133 may result in noticeable artifacts between the edges of the first person's image 133 and the background. Consequently, while the first person's image in the second photo 132 will appear clearer and more detailed, it will create a sense of disjointedness and a poor viewing experience. To address this, [the following text is missing: "combined with..."] Figure 12 , refer to Figure 14 As shown, after steps S310 and S307, the photo optimization process also includes S311:

[0194] S311: The mobile phone merges the first and fourth photos to obtain the fifth photo.

[0195] The fourth photo is obtained by optimizing the image of the target person in the first photo using an electronic device; the target person includes the first person and / or the second person. Of course, the target person image may also include images of other people in the first photo taken by the mobile phone.

[0196] In one possible approach, the fusion of the first and fourth photos can be achieved using a pre-trained target deep learning model. After inputting the first and fourth photos into this target deep learning model, it outputs a photo—the fifth photo—that features a smooth transition between the image of the person and the background image (i.e., the image excluding the person). For example, the target deep learning model could be a deep learning model used in some short video applications for tools that alter faces or change the background of a person, or any other feasible deep learning model; this application does not impose any specific limitations on this.

[0197] In another possible implementation, the pixel value (e.g., grayscale value, RGB value, etc.) of each pixel in the merged fifth image can be the average of the corresponding pixels in the first and fourth images. Alternatively, the pixel value of each pixel in the background image of the merged fifth image can be the average of the corresponding pixels in the first and fourth images, while the pixel value of the person's image can be a preset percentage of the average of the corresponding pixels in the first and fourth images. This preset percentage can be determined based on the difference between the pixel values ​​of the edge regions of the person's image and the pixel values ​​of the edge regions of the adjacent background images.

[0198] The two implementation methods described above are merely examples; in practice, any other feasible methods may be used.

[0199] Based on the S311's technical solution, the fifth photo, ultimately processed by the phone, is a fusion of the original first photo and the optimized fourth photo. Therefore, it not only produces a clearer and more detailed image of the person, but also eliminates any sense of separation between the person and the background, resulting in a more harmonious overall picture. Consequently, the user experience is improved.

[0200] Of course, in practice, to make the final optimized photo more harmonious, other methods can be used to obtain the fifth photo without merging the first and fourth photos. For example, the pixel values ​​of the corresponding areas in the fourth photo can be adjusted based on the arrangement of pixel values ​​in the connected areas between the image of the person and the background image in the first photo to obtain the fifth photo. This application does not impose specific restrictions on which method is used to obtain the fifth photo, as long as the final fifth photo ensures a smooth transition between the image of the person and the background image, without any sense of disjointedness, and with a more harmonious overall photo.

[0201] The following uses a mobile phone as an example to describe the optimization process of stored photos in the photo optimization method provided in this application embodiment. For example... Figure 15 As shown, the optimization process for the stored photos may include S1501-S1504:

[0202] S1501: The mobile phone responds to the user's optimization operation on the first target photo in the gallery display interface and identifies the first target photo as the first photo.

[0203] The gallery display interface includes multiple photos. The gallery display interface can be displayed by the phone in response to a user's trigger action (such as clicking) on ​​the gallery app icon. Alternatively, it can be displayed by the electronic device in response to a user's trigger action on the gallery option within the camera preview interface. The gallery option can be used to trigger the opening of the gallery app and the display of the gallery interface.

[0204] For example, the gallery display interface can be as follows: Figure 16 As shown in (a), the mobile phone can receive a user's trigger operation (such as a click) on the first target photo 161 in the gallery display interface. In response to this trigger operation, the mobile phone can display as shown in (a). Figure 16 The details interface 162 of the first target photo shown in (b) includes an optimization control 163. This details interface 162 is primarily used by the user to view the first target photo, its related information (e.g., generation time and region; in the figure, it's taken as October 25, 2021, 10:00 AM, Chang'an District, Xi'an City) and to perform feasible operations on the first target photo (e.g., sharing, saving, optimizing, deleting, etc.). Subsequently, the phone can respond to the user's triggering operation on the optimization control 163 in the details interface 162, identifying the first target photo as the first photo. Then, photo optimization is performed on the first photo, i.e., the subsequent steps S1502-S1504 are executed.

[0205] Here, the user's triggering action on the first target photo in the gallery display interface and the user's triggering action on the optimization control in the details interface of the first target photo can be considered as the aforementioned optimization operation. In practice, this optimization operation can also be any other feasible implementation. For example, the optimization operation can also be a combination of the user's triggering action on the first target photo in the gallery display interface and the gesture command or voice command entered by the user in the details interface of the first target photo. This application does not impose specific limitations on this.

[0206] It should be noted that in this application, the optimization controls only exist for the details page of photos that include facial images.

[0207] S1502. When the mobile phone determines that the first photo includes a face image, it obtains the first feature information of the first person to whom the first face image belongs.

[0208] The first feature information includes at least the facial feature information of the first person. For example, the specific features in the facial feature information may include at least one of the following: facial detail information, skin color, and skin texture. The facial detail information may include information such as the relative positions and sizes of facial features.

[0209] Furthermore, to more accurately distinguish different human images and obtain a more accurate 3D model subsequently, the first feature information here can also include the body posture features of the first human figure. Specific features in the body posture features may include, but are not limited to, at least one of the following: limb shape and body proportions.

[0210] The specific implementation of S1502 can be referred to the relevant description of S305 mentioned above, and will not be repeated here.

[0211] S1503, The mobile phone acquires a first three-dimensional model that matches the first feature information.

[0212] The first 3D model is used to represent the 3D features of the first person in the historical photograph corresponding to the first person's image. These 3D features include at least facial 3D features. All historical photographs corresponding to the first person include an image of that person. To further optimize the first person's posture, the 3D features can also include posture 3D features. This way, when the first 3D model is used to fuse and optimize the first person's image, the details of the first person's face and body will be clearer, resulting in a better optimization effect.

[0213] The specific implementation of S1503 can be described in accordance with the specific description of S306 in the aforementioned embodiment, and will not be repeated here.

[0214] S1504. The mobile phone merges the first 3D model with the image of the first person in the first photo to optimize the image of the first person in the first photo and obtain the second photo.

[0215] After the phone receives the second photo, it can store it in the local gallery for the user to view. Simultaneously, to clearly demonstrate the phone's optimization effects, the phone can also store the first photo for easy comparison. Of course, to save storage space, the first photo can also be omitted.

[0216] For specific examples of situations where the phone only stores the second photo, or where the phone stores both the first and second photos, please refer to the relevant descriptions following S307 above, which will not be repeated here.

[0217] Based on the aforementioned technical solutions S1501-S1504, when the user instructs the phone to optimize the first target photo in the local gallery (i.e., to perform optimization on the first target photo), the phone can first obtain a 3D model of the person's image corresponding to the face image in the photo. Then, using this 3D model, the image of the person in the photo is optimized to obtain an optimized photo. This is because the 3D model represents the 3D features of the person in historical photos. The detail information of the person in multiple historical photos will inevitably be more abundant than the detail shown in the photo currently taken by the phone. Therefore, the optimized photo will have richer detail information and a clearer image of the person. Thus, even if the environment in which the first target photo stored on the phone was taken was unfavorable, subsequent optimization by the phone can still produce a photo with better image quality, thereby improving the user experience.

[0218] In some embodiments, there may be multiple people in the first photograph; therefore, when optimizing the image of the people in the first photograph, it is often necessary to optimize the image of more than one person. Based on this, combined with Figure 15 , refer to Figure 17 As shown, in the optimization process for stored photos provided in this application embodiment, after S1501, it may further include S1505-S1507:

[0219] S1505. When the mobile phone determines that there is a human face image in the first photo, it obtains the second feature information of the second person to whom the second human face image belongs.

[0220] The second face image differs from the first face image, and the second feature information includes at least the facial feature information of the second person. Furthermore, to more accurately distinguish between images of different people and obtain a more accurate 3D model subsequently, the second feature information may also include the body posture feature information of the second person. Specific features in the body posture feature information may include, but are not limited to, at least one of the following: limb shape and body proportions.

[0221] S1506, The mobile phone acquires a second three-dimensional model that matches the second feature information.

[0222] The second 3D model is used to represent the 3D features of the second person in the corresponding historical photograph; these 3D features include at least facial 3D features; and all corresponding historical photographs include images of the second person. To further optimize the posture of the second person's image, these 3D features can also include posture 3D features. This way, when the second 3D model is used to fuse and optimize the second person's image, the details of the face and body in the image are clearer, resulting in better optimization.

[0223] S1507 The mobile phone merges the second 3D model with the image of the second person in the first photo to optimize the image of the second person in the first photo and obtain the third photo.

[0224] In this application, S1505-S1507 can be executed in parallel with S1502-S1504, or S1502-S1504 can be executed first and then S1505-S1507, or any other feasible execution order. This application does not impose specific restrictions on this. The specific implementation of S1505-S1507 can be referred to the relevant descriptions of S1502-S1504 mentioned above, and will not be repeated here.

[0225] In this application, if S1502-S1504 and S1505-S1507 are executed after S1501, the final optimized photo is a combination of the second and third photos. In the optimized photo, the images of the first person and the second person have been optimized accordingly.

[0226] Of course, if the first photo contains other facial images besides the first and second facial images, the other facial images can be optimized according to the optimization process for the first and second facial images in the foregoing embodiments, so as to obtain a photo in which all facial images are optimized. The specific optimization process can be referred to the description in the foregoing embodiments, and will not be repeated here.

[0227] Based on the technical solutions corresponding to S1505-S1507 above, the mobile phone can optimize each person image in the first photo using the corresponding 3D model. Therefore, in the final optimized photo, the details of the person image will be richer, and the person image will be clearer. In this way, even if the environment in which the first target photo stored on the phone was not conducive to taking pictures, a photo with good image quality of the person can still be obtained through subsequent optimization by the phone, thus improving the user experience.

[0228] In some embodiments, the unoptimized first target photo stored on the phone, being directly captured, exhibits a smooth transition between the person and their background. However, the optimized second photo (or third photo, or a combination of the second and third photos), because its person image is optimized, may result in a more noticeable difference between the person and the background image (i.e., the image excluding the person), creating a sense of disjointedness. This leads to a poor user experience. To address this, a combination of... Figure 17 , refer to Figure 18 As shown, after steps S1504 and S1507, the optimization process for the stored photos also includes S1508:

[0229] S1508: The mobile phone merges the first and fourth photos to obtain the fifth photo.

[0230] The fourth photo is obtained by optimizing the image of the target person in the first photo using an electronic device; the target person includes the first person and / or the second person. Of course, the image of the target person may also include images of other people in the first photo taken by the mobile phone.

[0231] The specific implementation of S1508 can be referred to the relevant description of step S11 in the aforementioned embodiments, and will not be repeated here.

[0232] Based on the technical solution corresponding to S1508, because the fifth photo obtained by the phone is a fusion of the original first photo and the optimized fourth photo, it not only has a clear and detailed image of the person, but also eliminates any sense of separation between the person and the background, resulting in a more harmonious overall picture. Consequently, the user experience is improved.

[0233] Of course, in practice, to make the final optimized photo more harmonious, the fifth photo can be obtained in other ways that do not require merging the first and fourth photos. For example, the pixel values ​​of the corresponding areas in the fourth photo can be adjusted based on the arrangement of pixel values ​​in the connected areas between the image of the person and the background image in the first photo to obtain the fifth photo. This application does not impose specific restrictions on the method used to obtain the fifth photo, as long as the final fifth photo ensures a smooth transition between the image of the person and the background image, without any sense of disjointedness, and with a more harmonious overall photo. The following uses a mobile phone as an example to describe the optimization information generation process in the photo optimization method provided in this application. Figure 19 As shown, the optimization information generation process may include S1901-S1904:

[0234] S1901, The mobile phone obtains at least one set of historical photos prior to the current moment.

[0235] Each set of historical photos is associated with a person, and different sets of historical photos are associated with different people; each historical photo in a set of historical photos contains an image of the person associated with the set of historical photos; at least one set of historical photos includes a first set of historical photos associated with the first person.

[0236] In one possible approach, at least one set of historical photos can be stored in the phone's local gallery, and the phone can then retrieve at least one set of historical photos from the gallery.

[0237] For example, a mobile phone's gallery app can have the function of performing facial recognition on historical photos to categorize them by people. Based on this, the local gallery on the phone can be categorized as follows: Figure 20 The classification shown categorizes historical photos, resulting in at least one set of historical photos 201, which is then displayed. Figure 20 The example uses at least one historical photo set 201, including historical photo set 201-1, historical photo set 201-2, historical photo set 201-3, and historical photo set 201-4. In practice, there may be more or fewer. Each historical photo set 201 is associated with a person; that is, each photo in each historical photo set 201 includes an image of the person associated with that historical photo set 201. (Refer to...) Figure 20 As shown, in practice, to help users know who is associated with each of the multiple historical photo sets stored on their phone, the associated area (e.g., the lower right corner) of each historical photo set 201 can be labeled with the person's identifier, such as their name or nickname. This identifier can be freely set by the user according to their own needs.

[0238] In another feasible approach, at least one set of historical photos can be stored on a server associated with the mobile phone. In this case, the mobile phone can request at least one set of historical photos from the server, which then sends the combined set of historical photos to the mobile phone. Specifically, the server typically receives the historical photos reported by the mobile phone directly, rather than combining them. Therefore, after receiving historical photos reported by a mobile phone, the server can perform facial recognition on the photos to categorize them by person, obtaining at least one set of historical photos. Then, upon receiving a request from a mobile phone to combine historical photos, the server can send all the historical photos corresponding to that mobile phone, along with the classification result for each historical photo.

[0239] S1902. The mobile phone determines the three-dimensional features of the people associated with the historical photo set based on the historical photo set.

[0240] Among these, the three-dimensional features include at least facial three-dimensional features. Furthermore, in order for the mobile phone to obtain a more complete three-dimensional model using the three-dimensional features, the three-dimensional features may also include body posture three-dimensional features.

[0241] In one feasible approach, the mobile phone can extract the 3D features of the person associated with each historical photo from a set of historical photos using any feasible feature extraction method. Finally, after statistically deduplicating all extracted 3D features, the 3D features of the person are obtained.

[0242] In another feasible approach, since the image quality of people in historical photographs with poor image quality is inevitably also poor, feature extraction for each historical photograph is quite wasteful of resources. Therefore, to reduce computational resource consumption, a method is combined with... Figure 19 , refer to Figure 21 As shown, S1902 may specifically include S19021 and S19022:

[0243] S19021. The mobile phone determines at least one second target photo from the historical photo set based on the imaging quality parameters of each historical photo in the historical photo set.

[0244] The imaging quality parameters include: resolution and the completeness of the images of the people associated with the historical photo set. At least one second target photo has higher imaging quality parameters than other photos in the historical photo set.

[0245] For example, a mobile phone can identify a historical photo from a collection of photos with a resolution greater than a preset threshold, and whose associated image of a person includes at least a complete upper body, as the second target photo. In practice, more or fewer imaging quality parameters can be used to determine the second target photo, and this application does not impose specific limitations on this.

[0246] S19022. The mobile phone determines the three-dimensional features of the person associated with the historical photo set based on the second target photo.

[0247] Specifically, the mobile phone can use any feasible feature extraction method to extract the three-dimensional features of the person associated with the historical photo set to which each second target photo belongs, and then perform statistical deduplication to obtain the three-dimensional features of the person.

[0248] Based on the technical solutions corresponding to S19021 and S19022 above, the mobile phone can select a second target photo with higher imaging quality parameters from the historical photo set as the basis for extracting 3D features. In this way, the mobile phone can extract the 3D features of people associated with the historical photo set with as few computing resources as possible, saving computing resources of electronic devices.

[0249] S1903: The mobile phone constructs a 3D model of the person associated with the historical photo set based on the 3D features of the person associated with the historical photo set.

[0250] Specifically, mobile phones can use any feasible 3D reconstruction technology to construct corresponding 3D models from 3D features.

[0251] S1904, a 3D model of a person associated with a collection of historical photos stored on a mobile phone.

[0252] It should be noted that in the photo-taking optimization process, steps S1901-S1904 are executed before step S306; in the optimized process for stored photos, steps S1901-S1904 are executed before step S1503. This ensures the phone can successfully acquire the first 3D model. The specific timing of the execution of steps S1901-S1904 before or after step S306 or S1503 depends on the actual needs.

[0253] Based on the technical solutions corresponding to S1902-S1904 above, the mobile phone can pre-build 3D models of each person in historical photos, and even store these models locally for later use when optimizing the photos. This allows the mobile phone to quickly and easily obtain the necessary 3D models when optimizing photos, improving the efficiency of photo optimization.

[0254] In the photo optimization process of the photo optimization method provided in this application (including the optimization process for captured photos and the optimization process for stored photos), if the mobile phone obtains the first 3D model by first obtaining the first 3D features and then constructing the first historical 3D model, then combined with... Figure 19 , refer to Figure 22 As shown, step S1905 is executed after step S1902:

[0255] S1905, Three-dimensional features of people associated with a collection of historical photos stored on a mobile phone.

[0256] Based on this solution, the phone can pre-build 3D models of each person in historical photos, and even store the 3D features of the associated individuals in historical photos locally. This allows the phone to retrieve the initial 3D features from the local storage when optimizing photos, and then construct the initial 3D model. This enables the phone to quickly and easily obtain the necessary 3D model when optimizing photos, thus improving the efficiency of photo optimization.

[0257] Furthermore, each step in S1901-S1905 can be executed by the server, meaning the execution entity changes from the mobile phone to the server. In this case, the more steps in S1901-S1905 are executed by the server, the lower the computational and / or storage requirements of the solution on the mobile phone. However, if the mobile phone needs to retrieve 3D models or features from the server during final photo optimization, the need for back-and-forth signaling and data interaction will increase the latency of photo optimization to some extent. The specific choice between the server and the mobile phone as the execution entity depends on actual needs, and this application does not impose specific restrictions on this.

[0258] In some embodiments, to ensure that the mobile phone can obtain optimization information (3D model or 3D features) that combines all historical photos when it needs to optimize an image of a person, the mobile phone needs to update the corresponding optimization information by including the current unoptimized photo as a historical photo for the next optimization after each optimization of the image of the person in the photo.

[0259] Based on this, when the optimized information is a three-dimensional model, and based on the technical solutions corresponding to S1901-S1904, combined with... Figure 3 , refer to Figure 23 As shown, in the optimization process for this captured image, steps S304 can also include S312-S315:

[0260] S312. The mobile phone adds the first photo to the first historical photo set associated with the first person to update the first historical photo set and obtain the second historical photo set.

[0261] Of course, considering the technical solutions corresponding to S1921 and S1922 in the aforementioned embodiments, if the mobile phone, when determining the three-dimensional features, only extracts the three-dimensional features of the second target photo from the historical photo set, then in this application, before executing S312, the mobile phone can first determine whether the first photo is the second target photo. If not, then S312-S315 will not be executed; if so, then S312-S315 will be executed. In this way, since the first photo, which is not the second target photo, does not need to undergo the additional S312-S315 steps, the computational and storage resources consumed by this part of the process are saved. Subsequent similar steps are similar and will not be described in detail hereafter.

[0262] S313: The mobile phone re-determines the three-dimensional features of the first person based on the second set of historical photos.

[0263] The three-dimensional features include at least facial three-dimensional features. Furthermore, to enable the mobile phone to obtain a more complete three-dimensional model using these features, the three-dimensional features may also include body posture three-dimensional features. The three-dimensional features of the first person in this application are the first three-dimensional features mentioned in the foregoing embodiments.

[0264] The specific implementation of S313 can be referred to the relevant description of S192 mentioned above, and will not be repeated here.

[0265] S314. The mobile phone reconstructs the three-dimensional model of the first person based on the redefined three-dimensional features of the first person.

[0266] The specific implementation of S314 can be referred to the relevant description of S193 above, and will not be repeated here. The three-dimensional model of the first person in this application is the first three-dimensional model mentioned in the foregoing embodiments.

[0267] S315, the mobile phone storage reconstructs the first three-dimensional model of the character.

[0268] For example, for the purpose of saving storage resources, S315 may specifically replace the previously stored 3D model of the first character with a reconstructed 3D model of the first character.

[0269] It should be noted that the timing of S312-S315 after S304 can be determined according to actual needs, and this application does not impose specific restrictions on this.

[0270] Of course, any of the steps in S312-S315 above can be executed by the server corresponding to the mobile phone, so as to reduce the demand of the technical solution provided in this application on the computing resources and / or storage resources of the mobile phone. Whether it is executed by the mobile phone or the server depends on the actual needs, and this application does not impose specific restrictions.

[0271] Based on the technical solutions corresponding to S312-S315, after optimizing a person image (e.g., the first person image) in a captured photo, the mobile phone can add the captured photo as a historical photo to the historical photo set associated with that person. This results in an updated 3D model of the person to whom the image belongs, leading to better optimization results when subsequent images of that person need to be optimized.

[0272] Similarly, when the optimized information is a three-dimensional model, based on the technical solutions corresponding to S1901-S1904, combined with... Figure 15 , refer to Figure 24 As shown, in the optimization process for the stored photos, step S1501 can be followed by steps S1509-S1512:

[0273] S1509. The mobile phone adds the first photo to the first historical photo set associated with the first person to update the first historical photo set and obtain the second historical photo set.

[0274] S1510: The mobile phone re-determines the three-dimensional features of the first person based on the second set of historical photos.

[0275] The three-dimensional features include at least facial three-dimensional features. Furthermore, to enable the mobile phone to obtain a more complete three-dimensional model using these features, the three-dimensional features may also include body posture three-dimensional features. The three-dimensional features of the first person mentioned here are the first three-dimensional features mentioned in the aforementioned embodiment.

[0276] The specific implementation of S1510 can be referred to the relevant description of S192 mentioned above, and will not be repeated here.

[0277] S1511. The mobile phone reconstructs the three-dimensional model of the first person based on the redefined three-dimensional features of the first person.

[0278] The specific implementation of S1511 can be referred to the relevant description of S193 above, and will not be repeated here. The three-dimensional model of the first character here is the first three-dimensional model mentioned in the previous embodiment.

[0279] S1512, the first three-dimensional model of the character reconstructed from the phone's storage.

[0280] For example, for the purpose of saving storage resources, S1512 may specifically replace the previously stored 3D model of the first character with a reconstructed 3D model of the first character.

[0281] It should be noted that the timing of S1509-S1512 after S1501 can be determined according to actual needs, and this application does not impose specific restrictions on this.

[0282] Of course, any of the steps in S1509-S1512 above can be executed by the server corresponding to the mobile phone, so as to reduce the demand of the technical solution provided in this application on the computing resources and / or storage resources of the mobile phone. Whether it is executed by the mobile phone or the server depends on the actual needs, and this application does not impose specific restrictions.

[0283] Based on the technical solutions corresponding to S1509-S1512, after optimizing a person image (e.g., the first person image) in a stored photo, the mobile phone can add that photo as a historical photo to the historical photo set corresponding to that person image. This results in an updated 3D model of the person to whom the image belongs, leading to better optimization results when subsequent optimizations of that person's image are needed.

[0284] Similarly, when the optimized information is in three-dimensional features, based on the technical solutions corresponding to S1901, S1902, and S1905, combined with... Figure 9 , refer to Figure 25 As shown, after S304, the optimization methods for taking photos can also include S316-S318:

[0285] S316. The mobile phone adds the first photo to the first historical photo set associated with the first person to update the first historical photo set and obtain the second historical photo set.

[0286] S317: The mobile phone re-determines the three-dimensional features of the first person based on the second set of historical photos.

[0287] Among these, the three-dimensional features include at least facial three-dimensional features. Furthermore, in order for the mobile phone to obtain a more complete three-dimensional model using the three-dimensional features, the three-dimensional features may also include body posture three-dimensional features.

[0288] The specific implementation of S317 can be referred to the relevant description of S192 mentioned above, and will not be repeated here.

[0289] S318, the mobile phone storage re-determined the three-dimensional features of the first person.

[0290] For example, for the purpose of saving storage resources, S318 may specifically replace the previously stored three-dimensional features of the first person with the reconstructed three-dimensional features of the first person.

[0291] It should be noted that the timing of S316-S318 after S304 can be determined according to actual needs, and this application does not impose specific restrictions on this.

[0292] Of course, any of the steps in S316-S318 above can be executed by the server corresponding to the mobile phone, so as to reduce the demand of the technical solution provided in this application on the computing resources and / or storage resources of the mobile phone. Whether it is executed by the mobile phone or the server depends on the actual needs, and this application does not impose specific restrictions.

[0293] Based on the technical solutions corresponding to S316-S318, after optimizing a person image (e.g., the first person image) in a captured photo, the mobile phone can add the captured photo as a historical photo to the historical photo set corresponding to that person image. This allows for the acquisition of updated 3D features of the person to whom the image belongs, resulting in better optimization effects when subsequent images of that person require further optimization.

[0294] Similarly, when the optimized information is in three-dimensional features, based on the technical solutions corresponding to S1901, S1902, and S1905, combined with... Figure 15If S1506 specifically involves first acquiring the first three-dimensional feature that matches the first feature information, and then constructing the first historical three-dimensional model, then refer to... Figure 26 As shown, after S1501, the optimization method for the stored photos can also include S1513-S1515:

[0295] S1513. The mobile phone adds the first photo to the first historical photo set associated with the first person to update the first historical photo set and obtain the second historical photo set.

[0296] S1514. The mobile phone redetermines the three-dimensional features of the first person based on the second set of historical photos.

[0297] Among these, the three-dimensional features include at least facial three-dimensional features. Furthermore, in order for the mobile phone to obtain a more complete three-dimensional model using the three-dimensional features, the three-dimensional features may also include body posture three-dimensional features.

[0298] The specific implementation of S1514 can be referred to the relevant description of S192 mentioned above, and will not be repeated here.

[0299] S1515, The mobile phone storage re-determined the three-dimensional features of the first person.

[0300] For example, for the purpose of saving storage resources, S1515 may specifically replace the previously stored three-dimensional features of the first person with the reconstructed three-dimensional features of the first person.

[0301] It should be noted that the timing of S1513-S1515 after S1501 can be determined according to actual needs, and this application does not impose specific restrictions on this.

[0302] Of course, any of the steps in S1513-S1515 above can be executed by the server corresponding to the mobile phone, so as to reduce the demand of the technical solution provided in this application on the computing resources and / or storage resources of the mobile phone. Whether it is executed by the mobile phone or the server depends on the actual needs, and this application does not impose specific restrictions.

[0303] Based on the technical solutions corresponding to S1513-S1515, after optimizing a person image (e.g., the first person image) in a stored photo, the mobile phone can add the stored photo as a historical photo to the historical photo set corresponding to that person image. This results in updated 3D features of the person to whom the image belongs, leading to better optimization results when subsequent optimizations are needed for that person's image.

[0304] In some embodiments, the phone may not have previously photographed the first person, so the phone may not have stored optimization information (a first 3D model or first 3D features) corresponding to that first person; or, although the phone may have previously photographed the first person, it may not have stored optimization information corresponding to that first person for other possible reasons. In this case, it is currently impossible to optimize the image of the first person in the photograph. However, in order to optimize the image of the first person in the photograph next time, the 3D features of the first person in the current first photograph can be extracted as the first 3D features of the first person for subsequent use, and even a 3D model can be constructed using these 3D features as the first 3D model of the first person for subsequent use.

[0305] Based on this, with the optimized information being the first three-dimensional model, and based on the technical solutions corresponding to S1901-S1904, combined with... Figure 3 , refer to Figure 27 As shown, in the optimization process for this captured image, step S305 can also include steps S319-S322:

[0306] S319. Without obtaining the first three-dimensional model, the mobile phone uses the first photo to construct a first set of historical photos associated with the first person.

[0307] Of course, considering the technical solutions corresponding to S1921 and S1922 in the aforementioned embodiments, if the mobile phone, when determining the three-dimensional features, only extracts the three-dimensional features of the second target photo from the historical photo set, then in this application, before executing S319, the mobile phone can first determine whether the first photo is the second target photo. If not, then S319-S322 will not be executed; if so, then S319-S322 will be executed. In this way, since the first photo, which is not the second target photo, does not need to undergo the additional S319-S322 steps, the computational and storage resources consumed by this part of the process are saved. Subsequent similar steps are similar and will not be described in detail hereafter.

[0308] The S320 mobile phone determines the three-dimensional features of the first person based on the first set of historical photos.

[0309] Among these, the three-dimensional features include at least facial three-dimensional features. Furthermore, in order for the mobile phone to obtain a more complete three-dimensional model using the three-dimensional features, the three-dimensional features may also include body posture three-dimensional features.

[0310] The specific implementation of S320 can be referred to the relevant description of S192 mentioned above, and will not be repeated here.

[0311] S321. The mobile phone constructs a three-dimensional model of the first person based on the three-dimensional features of the first person.

[0312] Of course, in practice, if the first historical photo set contains only the first photo, it may be impossible to obtain a suitable 3D model of the person due to limitations of the phone itself (e.g., lack of a depth camera) and the limitations of the first photo (e.g., unique angle). In this case, S321 and S322 can be omitted, and only the 3D features of the person can be stored after S320. S321 and S322 can then be executed after acquiring multiple photos containing images of the person and obtaining more 3D features of the person. In this case, before collecting enough 3D features to construct a historical 3D model of the person, step S319 can specifically involve the phone adding the captured photo as a historical photo to the first historical photo set associated with the person.

[0313] The specific implementation of S321 can be referred to the relevant description of S192 mentioned above, and will not be repeated here.

[0314] S322, the mobile phone stores a 3D model of the first person.

[0315] Based on the technical solutions corresponding to S319-S322, when a mobile phone cannot obtain a 3D model of a person in a captured photo, it can use the photo to create a corresponding set of historical photos, thereby obtaining a 3D model of that person. This allows the phone to easily obtain the corresponding 3D model when it needs to optimize the image of that person later, completing the optimization process and improving the user experience.

[0316] Similarly, when the optimized information is the first three-dimensional feature, based on the technical solutions corresponding to S1901, S1902, and S1905, combined with... Figure 9 , refer to Figure 28 As shown, in the optimization process for this captured image, step S305 can also include steps S323-S325:

[0317] S323. Without obtaining the first 3D model, the mobile phone uses the first photo to construct a first set of historical photos associated with the first person.

[0318] S324. The mobile phone determines the three-dimensional features of the first person based on the first set of historical photos.

[0319] Among these, the three-dimensional features include at least facial three-dimensional features. Furthermore, in order for the mobile phone to obtain a more complete three-dimensional model using the three-dimensional features, the three-dimensional features may also include body posture three-dimensional features.

[0320] The specific implementation of S324 can be referred to the relevant description of S192 mentioned above, and will not be repeated here.

[0321] S325, the three-dimensional features of the first person stored on the mobile phone.

[0322] Based on the technical solutions corresponding to S323-S325, when a mobile phone cannot construct a corresponding 3D model because it cannot obtain the 3D features of a person in a captured photo, it can use the photo to create a corresponding set of historical photos, thereby obtaining the corresponding 3D features. This allows the phone to successfully obtain the person's 3D features and construct a corresponding 3D model when subsequent image optimization is needed, thus completing the optimization process and improving the user experience.

[0323] In some embodiments, for the first person, the phone may be storing a photo containing the first person's image for the first time, so optimization information (first 3D model or first 3D features) for that first person may not be stored. In this case, it is currently impossible to optimize the image of the first person in the stored photo. However, in order to optimize the image of the first person in the captured photo or the subsequently stored unoptimized photo, the 3D features of the first person in the current first photo can be extracted as the first 3D features of the first person for subsequent use, and even a 3D model can be constructed using these 3D features as the first 3D model of the first person for subsequent use.

[0324] Based on this, with the optimized information being the first three-dimensional model, and based on the technical solutions corresponding to S1901-S1904, combined with... Figure 15 , refer to Figure 29 As shown, in the optimization process for the stored photos, step S1502 may also include steps S1516-S1519:

[0325] S1516. Without obtaining the first three-dimensional model, the mobile phone uses the first photo to construct a first set of historical photos associated with the first person.

[0326] S1517. The mobile phone determines the three-dimensional features of the first person based on the first set of historical photos.

[0327] Among these, the three-dimensional features include at least facial three-dimensional features. Furthermore, in order for the mobile phone to obtain a more complete three-dimensional model using the three-dimensional features, the three-dimensional features may also include body posture three-dimensional features.

[0328] The specific implementation of S1517 can be referred to the relevant description of S192 mentioned above, and will not be repeated here.

[0329] S1518. The mobile phone constructs a three-dimensional model of the first person based on the three-dimensional features of the first person.

[0330] Of course, in practice, if the first historical photo set contains only the first photo, it may be impossible to obtain a suitable 3D model of the person due to limitations of the phone itself (e.g., lack of a depth camera) and limitations of the first photo (e.g., unique angle). In this case, S1518 and S1519 can be omitted, and only the 3D features of the person can be stored after S1517. S1518 and S1519 can then be executed after acquiring multiple photos containing images of the person and obtaining more 3D features of the person. In this case, before collecting enough 3D features to construct a historical 3D model of the person, step S1516 can specifically involve the phone adding the already stored photo as a historical photo to the first historical photo set associated with the person.

[0331] The specific implementation of S1518 can be referred to the relevant description of S192 mentioned above, and will not be repeated here.

[0332] S1519, a 3D model of the first person stored on a mobile phone.

[0333] Based on the technical solutions corresponding to S1516-S1519, when a mobile phone cannot obtain a 3D model of a person in a stored photo, it can use that photo to create a corresponding set of historical photos, thereby obtaining a 3D model. This allows the phone to successfully obtain the 3D model when it needs to optimize the image of that person later, completing the corresponding optimization process and improving the user experience.

[0334] Similarly, when the optimized information is the first three-dimensional feature, based on the technical solutions corresponding to S1901, S1902, and S1905, combined with... Figure 15 If S1506 specifically involves first acquiring the first three-dimensional feature that matches the first feature information, and then constructing the first historical three-dimensional model, refer to... Figure 30 As shown, in the optimization process for the stored photos, step S1502 can also include steps S1520-S1522:

[0335] S1520: Without obtaining the first 3D model, the mobile phone uses the first photo to construct a first set of historical photos associated with the first person.

[0336] S1521. The mobile phone determines the three-dimensional features of the first person based on the first set of historical photos.

[0337] Among these, the three-dimensional features include at least facial three-dimensional features. Furthermore, in order for the mobile phone to obtain a more complete three-dimensional model using the three-dimensional features, the three-dimensional features may also include body posture three-dimensional features.

[0338] The specific implementation of S1521 can be referred to the relevant description of S192 mentioned above, and will not be repeated here.

[0339] S1522, The three-dimensional features of the first person stored in the mobile phone.

[0340] Based on the technical solutions corresponding to S1520-S1522, when a mobile phone cannot construct a corresponding 3D model because it cannot obtain the 3D features of a person in a stored photo, it can use the photo to create a corresponding set of historical photos, thereby obtaining the 3D features. This allows the phone to successfully obtain the 3D features and construct the corresponding 3D model when it needs to optimize the image of that person later, thus completing the optimization process and improving the user experience.

[0341] It is understood that the aforementioned devices, etc., include hardware structures and / or software modules corresponding to the execution of each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware-driven or software-driven manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0342] This application embodiment can divide the above-described electronic device into functional modules based on the method example described above. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0343] When dividing each function into modules according to its corresponding function, refer to Figure 31 As shown in the figure, this application provides an electronic device, including: an acquisition module 3101, a feature extraction module 3102, a model matching module 3103, and an optimization module 3104.

[0344] The acquisition module 3101 is configured to acquire a first photograph. The feature extraction module 3102 is configured to acquire first feature information of a first person to whom the first face image in the first photograph belongs, provided that the first photograph acquired by the acquisition module 3101 includes a face image; wherein the first feature information includes at least the face feature information of the first person. The model matching module 3103 is configured to acquire a first three-dimensional model that matches the first feature information acquired by the feature extraction module 3102; wherein the first three-dimensional model is used to represent the three-dimensional features of the first person in the historical photograph corresponding to the first person; the three-dimensional features include at least the three-dimensional features of the face. The optimization module 3104 is configured to fuse the first three-dimensional model acquired by the model matching module 3103 with the image of the first person in the first photograph acquired by the acquisition module 3101 to optimize the image of the first person in the first photograph, thereby obtaining a second photograph.

[0345] Optionally, the acquisition module 3101 is specifically configured to: receive a photo-taking operation; and in response to the photo-taking operation, acquire a first photo.

[0346] Optionally, the electronic device further includes a display module 3105; in response to the photo-taking operation, before acquiring the first photo, the display module 3105 is configured to: in response to the user's trigger operation on the camera application icon, display a camera preview interface, and determine in real time whether the camera preview interface includes a face image; wherein, the camera preview interface is used to preview the shooting screen of the electronic device's camera; wherein, whether the first photo includes a face image is the latest determination result of the display module 3105 on whether the camera preview interface includes a face image before the acquisition module 3101 receives the photo-taking operation.

[0347] Optionally, the acquisition module 3101 is further configured to: in response to the user's optimization operation on the first target photo in the gallery display interface, determine the first target photo as the first photo.

[0348] Optionally, before the model matching module 3103 acquires the first 3D model, the model matching module 3103 is further configured to: acquire at least one set of historical photos prior to the current moment; wherein each set of historical photos is associated with a person, and different sets of historical photos are associated with different people; each historical photo in the set of historical photos includes an image of the person associated with the set of historical photos; at least one set of historical photos includes a first set of historical photos associated with the first person. Based on the set of historical photos, the 3D features of the person associated with the set of historical photos are determined; wherein the 3D features include at least facial 3D features. Based on the 3D features of the person associated with the set of historical photos, a 3D model of the person associated with the set of historical photos is constructed.

[0349] Optionally, the model matching module 3103 is specifically configured to: select at least one second target photo from the historical photo set based on the imaging quality parameters of each historical photo in the historical photo set; the imaging quality parameters include: resolution, the completeness of the image of the person associated with the historical photo set; the imaging quality parameters of at least one second target photo are higher than the imaging quality parameters of other photos in the historical photo set. From the second target photo, the three-dimensional features of the person associated with the historical photo set are extracted.

[0350] Optionally, after the acquisition module 3101 acquires the first photo, the model matching module 3103 is further configured to: add the first photo acquired by the acquisition module 3101 to the first historical photo set associated with the first person, thereby updating the first historical photo set and obtaining a second historical photo set; and, based on the second historical photo set, redetermine the three-dimensional features of the first person. Based on the redetermined three-dimensional features of the first person, reconstruct the three-dimensional model of the first person.

[0351] Optionally, after the feature extraction module 3102 obtains the first feature information, the model matching module 3103 is further configured to: construct a first set of historical photos associated with the first person using the first photo when the first three-dimensional model is not obtained; determine the three-dimensional features of the first person based on the first set of historical photos; and construct a three-dimensional model of the first person based on the three-dimensional features of the first person.

[0352] Optionally, the model matching module 3103 is specifically configured to: acquire a first three-dimensional feature that matches the first feature information acquired by the feature extraction module 3102; the first three-dimensional feature is the three-dimensional feature of the first person in the historical photo corresponding to the first person. A first three-dimensional model is constructed using the first three-dimensional feature.

[0353] Optionally, before the model matching module 3103 acquires the first 3D model, the model matching module 3103 is further configured to: acquire at least one set of historical photos prior to the current moment; wherein each set of historical photos is associated with a person, and different sets of historical photos are associated with different people; each historical photo in the set of historical photos contains an image of the person associated with the set of historical photos; at least one set of historical photos includes a first set of historical photos associated with the first person. Based on the set of historical photos, the 3D features of the person associated with the set of historical photos are determined; wherein the 3D features include at least facial 3D features.

[0354] Further optionally, after the acquisition module 3101 acquires the first photo, the model matching module 3103 is further configured to: add the first photo acquired by the acquisition module 3101 into the first historical photo set associated with the first person, so as to update the first historical photo set and obtain the second historical photo set; and redetermine the three-dimensional features of the first person based on the second historical photo set.

[0355] Optionally, after the feature extraction module 3102 obtains the first feature information, the model matching module 3103 is further configured to: construct a first set of historical photos associated with the first person using the first photo, in the absence of a first 3D model; and determine the 3D features of the first person based on the first set of historical photos.

[0356] Optionally, after the acquisition module 3101 acquires the first photo, the feature extraction module 3102 is further configured to acquire second feature information of the second person to whom the second face image belongs, provided that the first photo acquired by the acquisition module 3101 includes a face image; wherein the second person is different from the first person, and the second feature information includes at least the face feature information of the second person. The model matching module 3103 is further configured to acquire a second three-dimensional model that matches the second feature information acquired by the feature extraction module 3102; wherein the second three-dimensional model is used to represent the three-dimensional features of the second person in the historical photo corresponding to the second person. The optimization module 3104 is further configured to fuse the second three-dimensional model acquired by the model matching module 3103 with the image of the second person in the first photo acquired by the acquisition module 3101 to optimize the image of the second person in the first photo, thereby obtaining a third photo.

[0357] Optionally, the optimization module 3104 is further configured to merge the first photo and the fourth photo to obtain a fifth photo. The fourth photo is obtained by optimizing the image of the target person in the first photo; the target person includes the first person and / or the second person.

[0358] Regarding the electronic devices in the above embodiments, the specific methods by which each module performs its operations have been described in detail in the embodiments of the photo optimization method described above, and will not be elaborated here. The related beneficial effects can also be referred to the related beneficial effects of the aforementioned photo optimization method, and will not be repeated here.

[0359] This application also provides an electronic device, which includes: a camera, a display screen, a memory, and one or more processors; the camera, display screen, memory, and processors are coupled; wherein, the memory stores computer program code, which includes computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the photo optimization method provided in the foregoing embodiments. The specific structure of this electronic device can be referred to... Figure 2 The structure of the electronic device shown is illustrated.

[0360] This application also provides a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the photo optimization method provided in the foregoing embodiments.

[0361] This application also provides a computer program product containing executable instructions that, when run on an electronic device, cause the electronic device to perform the photo optimization method provided in the foregoing embodiments.

[0362] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0363] In the several embodiments provided in this application, it should be understood that the disclosed apparatus / device and method can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0364] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0365] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0366] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0367] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method applied to electronic devices, characterized in that, include: Display the first image; If the first image includes a face image, the first feature information of the first person to whom the first face image in the first image belongs is obtained; wherein, the first feature information includes at least the face feature information of the first person. Obtain a first three-dimensional model that matches the first feature information; wherein, the first three-dimensional model includes facial three-dimensional features extracted from historical images corresponding to the first person; The facial 3D features in the first 3D model are fused with the first facial image in the first image to obtain a second image. The first 3D model includes facial 3D features, which are used to optimize the facial details of the first person in the first image. The first image and the second image are fused to obtain a fifth image. The pixel value of the background region of the fifth image is the average pixel value of the corresponding pixels in the first image and the second image. The pixel value of the human figure region of the fifth image is the product of the average pixel value of the corresponding pixels in the first image and the second image and a preset percentage. The preset percentage is related to the target pixel difference, which is the pixel difference between the edge pixels of the human figure region and the edge pixels of the background region.

2. The method according to claim 1, characterized in that, Prior to displaying the first image, the method further includes: In response to the user's trigger action on the camera app icon, the camera preview interface is displayed; Displaying the first image includes displaying the first image on the camera preview interface.

3. The method according to claim 2, characterized in that, After displaying the camera preview interface, the method includes: The camera preview interface is used to detect in real time whether it includes a face image. The camera preview interface is used to display image data captured by the camera of the electronic device.

4. The method according to claim 1, characterized in that, The first image is an image that has been stored in the electronic device.

5. The method according to claim 4, characterized in that, The step of obtaining the first feature information of the first person to whom the first face image in the first image belongs includes: In response to a user's operation on a first control, the first feature information is extracted from the first face image, wherein the first control is a control displayed relative to the first image and is used to trigger processing for the first image.

6. The method according to claim 2, characterized in that, During the display of the first image on the camera preview interface, the method further includes: The ambient light level was detected to be lower than the preset brightness value; Alternatively, determine that the difference between the brightness of the background image of the first image and the brightness of the image of the first person is greater than a certain threshold.

7. The method according to any one of claims 1-6, characterized in that, The fifth image displays a first mark, indicating that the fifth image is a processed image.

8. The method according to claim 7, characterized in that, When displaying the fifth image, the method further includes: In response to a user's swipe gesture on the fifth image, the first image is displayed.

9. The method according to any one of claims 1-8, characterized in that, The step of obtaining the first three-dimensional model that matches the first feature information includes: Send a first request to the server, the first request carrying first characteristic information; Receive a first response sent by the server, the first response including the first 3D model that matches the first special effects information.

10. The method according to any one of claims 1-8, characterized in that, Before obtaining the first three-dimensional model matching the first feature information, the method further includes: Obtain at least one set of historical images prior to the current moment; wherein each set of historical images is associated with a person, and different sets of historical images are associated with different people; each historical image in the set of historical images includes an image of the person associated with the set of historical images; at least one set of historical images includes a first set of historical images associated with the first person; Based on the historical image set, determine the three-dimensional features of the people associated with the historical image set; wherein, the three-dimensional features include at least three-dimensional facial features; Based on the three-dimensional features of the people associated with the historical image set, construct a three-dimensional model of the people associated with the historical image set; The step of obtaining the first three-dimensional model that matches the first feature information includes: searching for the first three-dimensional model that contains the first three-dimensional feature in the constructed three-dimensional model.

11. The method according to claim 10, characterized in that, The step of determining the three-dimensional features of the individuals associated with the historical image set based on the historical image set includes: Based on the imaging quality parameters of each historical image in the historical image set, at least one second target image is selected from the historical image set; the imaging quality parameters include: resolution, the completeness of the image of the person associated with the historical image set; the imaging quality parameters of the at least one second target image are higher than the imaging quality parameters of other images in the historical image set; From the second target image, extract the three-dimensional features of the person associated with the historical image set.

12. The method according to claim 10 or 11, characterized in that, Before obtaining the first 3D model matching the first feature information, the method further includes: The first image is added to the first historical image set associated with the first person to update the first historical image set and obtain the second historical image set. Based on the second set of historical images, the three-dimensional features of the first person are re-determined; Based on the redefined three-dimensional features of the first person, a new three-dimensional model of the first person is constructed.

13. The method according to any one of claims 1-8, characterized in that, Before obtaining the first three-dimensional model matching the first feature information, the method further includes: Obtain at least one set of historical images prior to the current moment; wherein each set of historical images is associated with a person, and different sets of historical images are associated with different people; each historical image in the set of historical images includes an image of the person associated with the set of historical images; at least one set of historical images includes a first set of historical images associated with the first person; Based on the historical image set, determine the three-dimensional features of the people associated with the historical image set; wherein, the three-dimensional features include at least three-dimensional facial features; Store the 3D features of the person associated with each of the historical image sets; Determining the first three-dimensional model using the first three-dimensional feature includes: finding the first three-dimensional feature that matches the first feature information; and constructing the first three-dimensional model based on the found first three-dimensional feature.

14. The method according to claim 13, characterized in that, Before obtaining the first 3D model matching the first feature information, the method further includes: The first image is added to the first historical image set associated with the first person to update the first historical image set and obtain the second historical image set. Based on the second set of historical images, the three-dimensional features of the first person are re-determined; Update the stored 3D features of the first person.

15. The method according to any one of claims 1-8, characterized in that, The step of obtaining the first three-dimensional model that matches the first feature information includes: When no first three-dimensional model matching the first feature information is found in the electronic device, a first historical image set associated with the first person is constructed using the first image. Based on the first set of historical images, determine the three-dimensional features of the first person; Based on the three-dimensional features of the first person, construct the first three-dimensional model of the first person.

16. The method according to claim 1, characterized in that, If the first image also includes a second face image, the method further includes: Obtain second feature information of the second person to whom the second face image in the first image belongs; wherein, the second feature information includes at least the facial feature information of the second person; Obtain a second three-dimensional model that matches the second feature information; wherein, the second three-dimensional model includes facial three-dimensional features extracted from historical images corresponding to the second person; The facial 3D features of the second 3D model are fused with the second facial image in the first image to obtain a third image; The first image and the third image are merged to obtain the sixth image.

17. The method according to claim 16, characterized in that, After obtaining the second and third images, the method further includes: The first image, the second image, and the third image are merged to obtain the seventh image.

18. The method according to claim 1, characterized in that, The step of fusing the three-dimensional facial features with the first facial image in the first image includes: The facial details in the first face image are enhanced by utilizing the aforementioned three-dimensional facial features.

19. An electronic device, characterized in that, include: The device includes a camera, a display screen, a memory, and one or more processors; the camera, the display screen, the memory, and the processors are coupled together; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the image processing method as described in any one of claims 1-18.

20. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the image processing method as described in any one of claims 1-18.

Images