An image reconstruction method and related device

Abstract

The present application discloses an image reconstruction method, which can preprocess a preview image before an electronic device receives a user's shooting operation, that is, during shooting preview, and load a high-definition texture dictionary library from a disk into the running memory. After receiving the user's shooting operation, the electronic device can perform super-resolution repair on the captured low-definition image by using the high-definition texture dictionary library already loaded in the running memory to obtain a high-definition image. In this way, the time for the electronic device to perform super-resolution repair on the captured low-definition image after taking a photo can be reduced.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular, to an image reconstruction method and related device. Background Art

[0002] Since the development of electronic devices such as smart phones to date, taking pictures has become one of their most important features. Both the basic imaging devices and imaging algorithms have undergone great development and changes, which have promoted the revolution of mobile phone photography time and time again and improved the user's photography experience. Due to the influence of factors such as the shooting environment, hardware conditions, and image compression of electronic devices, the images obtained by electronic devices often have poor visual effects, low resolution, contain noise or reflections, etc. Usually, the obtained images need to be reconstructed to obtain higher-quality images.

[0003] Currently, image reconstruction methods based on deep learning are usually adopted on electronic devices. For example, methods such as super-resolution generative adversarial networks (SRGAN), enhanced super-resolution generative adversarial networks (ESRGAN), and wide activation for efficient and accurate image super-resolution (WDSR) are used to perform super-resolution reconstruction processing on single-frame images. Another example is to use a network model based on a generative adversarial network to remove glass reflections or eye reflections in a captured image. Another example is multi-frame super-resolution and denoising processing based on convolutional neural networks (CNN). These processing models have numerous parameters, and it takes a long time for electronic devices to load these models. After shooting, the electronic device needs to wait for a long time to super-resolution reconstruct a high-definition image. Therefore, when deploying these models on electronic devices, they often need to be compressed and quantized, and then fine-tuned and further optimized after becoming small models. However, as the processing model decreases, the effect of the processing model often weakens accordingly, and it is difficult to find a processing model with fast processing speed and good effect. Summary of the Invention

[0004] This application provides an image reconstruction method and related device, which can use the preview image stream during the shooting preview process to load the high-definition texture dictionary library required for the super-resolution process into the memory, and perform image reconstruction on the captured image to quickly improve the resolution of the image.

[0005] In a first aspect, the present application provides an image reconstruction method, including: an electronic device displays a shooting preview interface, on which a shooting button and a preview image stream captured in real time by a camera are displayed; the electronic device detects that the preview image stream includes a specified target content, and the type of the specified target content is the first type; the electronic device determines a first dictionary library corresponding to the first type from a dictionary library of multiple different shooting target categories, and loads the first dictionary library from the disk into the running memory, where the high-definition texture dictionary libraries of the multiple different shooting target types are stored in the disk of the electronic device; after the first dictionary library is loaded into the running memory, the electronic device receives a first input; in response to the first input, the electronic device acquires a low-definition image captured by the camera; the electronic device processes the low-definition image through the first dictionary library loaded in the running memory to obtain a high-definition image, and the resolution of the specified target content in the high-definition image is greater than the resolution of the specified target content in the low-definition image.

[0006] Through an image reconstruction method provided by the present application, it is possible to preprocess the preview image and load the dictionary library into the running memory (RAM) before the electronic device receives the user's shooting operation, that is, during shooting preview. After receiving the user's shooting operation, the electronic device can use the dictionary library loaded in the running memory to perform super-resolution repair on the captured low-definition image to obtain a high-definition image. In this way, the time for the electronic device to perform super-resolution repair on the captured low-definition image after taking a photo can be reduced.

[0007] In a possible implementation manner, the electronic device processes the low-definition image through the first dictionary library loaded in the running memory to obtain a high-definition image, specifically including: the electronic device matches a first high-definition texture feature corresponding to the low-definition image from the first dictionary library loaded in the running memory; the first dictionary library includes high-definition texture features of the same shooting target category; the electronic device fuses the first high-definition texture feature into the low-definition image to obtain the high-definition image.

[0008] In a possible implementation manner, the electronic device matches a first high-definition texture feature corresponding to the low-definition image from the first dictionary library loaded in the running memory, specifically including: the electronic device identifies and crops a first area where the specified target content is located in the low-definition image to obtain a low-definition cropped image; the electronic device extracts a first image feature from the low-definition cropped image; the electronic device matches a first high-definition texture feature with a similarity greater than a preset value to the first image feature from the first dictionary library loaded in the running memory.

[0009] In a possible implementation, the electronic device matches the first high-definition texture feature corresponding to the low-definition image from the first dictionary library loaded from the operating memory, specifically including: the electronic device determines a first region based on the position of the specified target content in the preview image stream; the electronic device crops the first region from the low-definition image to obtain a low-definition cropped image; the electronic device extracts the first image feature from the low-definition cropped image; the electronic device matches the first high-definition texture feature with a similarity greater than a preset value to the first image feature from the first dictionary library loaded from the operating memory.

[0010] In a possible implementation, the electronic device fuses the first high-definition texture feature into the low-definition image to obtain the high-definition image, specifically including: the electronic device fuses the first high-definition texture feature into the low-definition cropped image to obtain a high-definition cropped image; the electronic device replaces the low-definition cropped image with the high-definition cropped image and pastes it back to the first region in the low-definition image to obtain the high-definition image.

[0011] In a possible implementation, a first control is also displayed on the shooting preview interface; before the electronic device receives a first input, the method further includes: the electronic device receives a second input for the first control; in response to the second input, the electronic device enables the super-resolution reconstruction mode.

[0012] In a possible implementation, before the electronic device loads the first dictionary library from the disk to the operating memory, the method further includes: the electronic device determines whether the first dictionary library has been loaded into the operating memory; the electronic device loads the first dictionary library from the disk to the operating memory, specifically including: when the first dictionary library has not been loaded into the operating memory, the electronic device records the first dictionary library from the disk to the operating memory.

[0013] In a possible implementation, after the electronic device detects that the preview image stream includes the specified target content, the method further includes: the electronic device displays scene information on the shooting preview interface, and the scene information is used to indicate that the type of the specified target content in the preview image stream is the first type.

[0014] In a possible implementation, after the electronic device processes the low-definition image through the first dictionary library loaded in the operating memory to obtain a high-definition image, the method further includes: the electronic device saves the high-definition image.

[0015] In a possible implementation, after the electronic device fuses the first high-definition texture feature into the low-definition image to obtain a high-definition image, the method further includes: the electronic device determines whether the image quality of the high-definition image meets a preset condition. If so, the electronic device saves the high-definition image; if not, the electronic device saves the low-definition image.

[0016] In a possible implementation, if the electronic device determines that the image quality of the high-definition image does not meet the preset condition, the method further includes: the electronic device obtains multiple high-definition reference images of the first type from the picture library or the network, extracts high-definition texture features from the multiple high-definition reference images, and stores them in the first dictionary library.

[0017] In a possible implementation, the preset condition includes: the clarity of the high-definition image is greater than a preset clarity.

[0018] In a possible implementation, the first type includes any one of the following: human face, building, green plant, animal cat, animal dog, animal bird, etc.

[0019] In a second aspect, the present application provides an electronic device, including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors. The one or more memories are used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the image reconstruction method in any possible implementation manner of any aspect above.

[0020] In a third aspect, an embodiment of the present application provides a computer storage medium, including computer instructions. When the computer instructions run on an electronic device, the electronic device executes the image reconstruction method in any possible implementation manner of any aspect above.

[0021] In a fourth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a computer, the computer executes the image reconstruction method in any possible implementation manner of any aspect above.

[0022] In a fifth aspect, an embodiment of the present application provides a chip system, including: one or more processors, one or more memories; wherein, the one or more memories include a disk (ROM) and a running memory (RAM). The one or more memories are used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the image reconstruction system executes the image reconstruction method in any possible implementation manner of any aspect above.

[0023] Among them, one or more processors may include one or more of an application processor, an image signal processor, a digital signal processor, a neural network processor, and a graphics processor. Description of the Drawings

[0024] Figure 1 Schematic diagram of the super-resolution process of a face image provided by an embodiment of the present application;

[0025] Figure 2A Schematic diagram of a low-resolution face image provided by an embodiment of the present application;

[0026] Figure 2B Schematic diagram of key points and a face frame in a low-resolution face image provided by an embodiment of the present application;

[0027] Figure 2C Schematic diagram of expanding the face frame in a low-resolution face image provided by an embodiment of the present application;

[0028] Figure 2D Schematic diagram of a face cropped image provided by an embodiment of the present application;

[0029] Figure 2E Schematic diagram of a low-resolution face block provided by an embodiment of the present application;

[0030] Figure 2F Schematic diagram of a high-resolution face block provided by an embodiment of the present application;

[0031] Figure 2G Schematic diagram of a high-resolution face image provided by an embodiment of the present application;

[0032] Figure 3 Schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application;

[0033] Figure 4 Schematic diagram of the software structure of an electronic device provided by an embodiment of the present application;

[0034] Figures 5A - 5D Schematic diagrams of a set of interfaces provided by an embodiment of the present application;

[0035] Figures 6A - 6B Schematic diagrams of another set of interfaces provided by an embodiment of the present application;

[0036] Figures 7A - 7B Schematic diagrams of another set of interfaces provided by an embodiment of the present application;

[0037] Figures 8A - 8D Schematic diagrams of another set of interfaces provided by an embodiment of the present application;

[0038] Figure 9Schematic flowchart of an image reconstruction method provided by an embodiment of the present application;

[0039] Figure 10 Schematic flowchart of an image reconstruction method provided by another embodiment of the present application;

[0040] Figure 11 Schematic flowchart of an image reconstruction method provided by another embodiment of the present application;

[0041] Figure 12 Schematic structural diagram of an image reconstruction system provided by an embodiment of the present application. Detailed implementation manners

[0042] Hereinafter, the technical solutions in the embodiments of the present application will be clearly and elaborately described with reference to the accompanying drawings. Among them, in the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B may mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, "a plurality of" means two or more than two.

[0043] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more than two.

[0044] The following introduces a process of face super-resolution processing provided in the embodiments of the present application.

[0045] Figure 1 Schematic flowchart of a face super-resolution processing provided in the embodiments of the present application is shown.

[0046] As Figure 1 shown, the process of the face super-resolution processing may include the following steps:

[0047] 1. The electronic device may capture a low-resolution face (LR Face) image in response to the user's input.

[0048] Exemplarily, the low-resolution face image may be as Figure 2A shown.

[0049] Data preprocessing and model loading stage:

[0050] 2. The electronic device extracts the key face (Face landmark) points in the low-resolution face image.

[0051] Exemplarily, the key face points in the low-resolution face image can be as Figure 2B shown. The electronic device can determine the coordinates of the face bounding box in the low-resolution face image based on the key face points.

[0052] Exemplarily, as Figure 2B shown, the face bounding box can include the facial features of the face image.

[0053] 3. The electronic device can expand the area of the face bounding box on the low-resolution image.

[0054] Exemplarily, as Figure 2C shown, the electronic device can expand the face bounding box so that the edge of the face bounding box maintains a certain distance from the edge contour of the face. By expanding the area of the face bounding box on the low-resolution image in this way, the edge contour of the face can be completely cropped out when cropping the face, reducing the loss of information on the edge contour of the face.

[0055] 4. The electronic device can crop the image within the face bounding box based on the expanded face bounding box to obtain a face cropped image.

[0056] Exemplarily, the face cropped image can be as Figure 2D shown.

[0057] 5. The electronic device can re-define the face area in the face cropped image and crop out the low-resolution face block in the face area.

[0058] Exemplarily, as Figure 2D shown, the electronic device can re-define the face area in the face cropped image based on the face landmark points. In this way, super-resolution processing and restoration can be accurately performed on the face area. Among them, the low-resolution face block cropped out by the electronic device from the face area can be as Figure 2E shown.

[0059] 6. The electronic device loads the face processing model into the memory.

[0060] Among them, the face processing model can be a processing model such as SRGAN, ESRGAN, WDSR, etc., which is not limited here.

[0061] Model inference stage:

[0062] 7. The electronic device can send the low-resolution face block into the face processing model for super-resolution processing to obtain a high-resolution face block.

[0063] Exemplarily, the high-resolution face block can be as Figure 2F shown.

[0064] Post - processing stage:

[0065] 8. The electronic device can use the binary mask image of the five - sense organ information to identify the face area in the low - resolution image, and replace the low - resolution face block with the high - definition face block and paste it back to the face area in the low - resolution face image to obtain a high - definition face image. Among them, the high - definition face image can be as Figure 2G shown. The resolution of the face area in the high - definition face image is greater than that of the face area in the low - resolution face image.

[0066] In the above super - resolution process, after the electronic device responds to the user's input and captures a low - resolution image, it needs to go through the above data pre - processing stage, model loading stage, model inference stage, and post - processing stage before it can obtain the super - resolution - repaired high - definition image. However, due to the relatively large computational amount of the deep - learning - based processing model and the limited processing power of the electronic device, the processing time of the entire above - mentioned super - resolution process is still relatively long. If the processing model is compressed and quantized to save the model inference time, the super - resolution effect of the processing model will be affected. Moreover, when the model is compressed and quantized, the model needs to be continuously optimized, which is time - consuming and laborious.

[0067] Therefore, in the embodiments of the present application, an image reconstruction method is provided, which can pre - process the preview image and load the high - definition texture dictionary library into the running memory (RAM) before the electronic device receives the user's shooting operation, that is, during the shooting preview. After receiving the user's shooting operation, the electronic device can perform super - resolution repair on the captured low - resolution image by using the high - definition texture dictionary library loaded in the running memory to obtain a high - definition image. In this way, the time for the electronic device to perform super - resolution repair on the captured low - resolution image after taking a photo can be reduced.

[0068] Next, the structural schematic diagram of the electronic device 100 provided in the embodiments of the present application is introduced.

[0069] Figure 3 The structural schematic diagram of the electronic device 100 is shown.

[0070] Next, the embodiments will be specifically described by taking the electronic device 100 as an example. It should be understood that Figure 3 the shown electronic device 100 is only an example, and the electronic device 100 can have more or fewer components than Figure 3 shown, can combine two or more components, or can have different component configurations. Figure 3 The various components shown can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application - specific integrated circuits.

[0071] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0072] It can be understood that the structure schematically shown in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figures, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0073] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0074] Among them, the controller may be the nerve center and command center of the electronic device 100. The controller may generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching instructions and executing instructions.

[0075] A memory can also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can hold the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the said memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0076] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0077] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193, etc. through different I2C bus interfaces. For example: The processor 110 can be coupled to the touch sensor 180K through the I2C interface, enabling the processor 110 to communicate with the touch sensor 180K through the I2C bus interface to implement the touch function of the electronic device 100.

[0078] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple groups of I2S buses. The processor 110 can be coupled to the audio module 170 through the I2S bus to implement communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface to implement the function of answering a phone call through a Bluetooth headset.

[0079] The PCM interface can also be used for audio communication to sample, quantize, and encode analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface to implement the function of answering calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0080] The UART interface is a general-purpose serial data bus for asynchronous communication. This bus can be a two-way communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0081] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate through the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the electronic device 100.

[0082] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0083] The USB interface 130 is an interface that conforms to the USB standard specification, and can specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used for data transmission between the electronic device 100 and peripheral devices. It can also be used to connect a headset to play audio through the headset. This interface can also be used to connect other electronic devices, such as AR devices, etc.

[0084] It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present invention is only for illustrative purposes and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods or a combination of multiple interface connection methods in the above embodiments.

[0085] The charging management module 140 is used to receive a charging input from a charger. The charger can be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 can receive the charging input of the wired charger through the USB interface 130. In some embodiments of wireless charging, the charging management module 140 can receive the wireless charging input through the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device through the power management module 141.

[0086] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives the inputs from the battery 142 and / or the charging management module 140 and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as the battery capacity, the number of battery cycles, and the battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be disposed in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can also be disposed in the same device.

[0087] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0088] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0089] The mobile communication module 150 may provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 may receive electromagnetic waves through the antenna 1, filter, amplify, and perform other processing on the received electromagnetic waves, and then transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 may be provided in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be provided in the same device.

[0090] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, receiver 170B, etc.), or displays an image or video through the display screen 194. In some embodiments, the modulation and demodulation processor may be an independent device. In other embodiments, the modulation and demodulation processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0091] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSSs), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive signals to be sent from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0092] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, such that electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0093] Electronic device 100 implements a display function through a GPU, display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, and is connected to display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0094] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0095] The electronic device 100 can implement the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc.

[0096] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light passes through the lens and is transmitted to the camera photosensitive element. The optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0097] The camera 193 is used to capture static images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV, etc. formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0098] The digital signal processor is used to process digital signals. Besides being able to process digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0099] The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0100] The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission pattern between human brain neurons, it can quickly process the input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0101] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to achieve the data storage function. For example, files such as music and videos are saved in the external memory card.

[0102] The internal memory 121 can be used to store computer-executable program code, and the executable program code includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, the image playback function, etc.). The data storage area can store the data created during the use of the electronic device 100 (such as audio data, phone book, etc.). In addition, the internal memory 121 can include high-speed random access memory and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0103] The electronic device 100 can implement audio functions through the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and the application processor, etc. For example, music playback, recording, etc.

[0104] The audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some functional modules of the audio module 170 can be disposed in the processor 110.

[0105] The speaker 170A, also known as the "loudspeaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or hands-free calls through the speaker 170A.

[0106] The receiver 170B, also known as the "earpiece", is used to convert an audio electrical signal into a sound signal. When the electronic device 100 answers a call or a voice message, the voice can be listened to by placing the receiver 170B close to the human ear.

[0107] The microphone 170C, also known as the "microphone" or "transmitter", is used to convert a sound signal into an electrical signal. When making a call or sending a voice message, the user can speak by bringing the mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In some other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also implement a noise reduction function. In some other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and implement functions such as directional recording.

[0108] The headphone jack 170D is used to connect a wired headphone. The headphone jack 170D can be a USB interface 130, or a 3.5 mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0109] The pressure sensor 180A is used to sense pressure signals and can convert pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor may include at least two parallel plates having conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, the instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

[0110] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake during shooting. Exemplarily, when the shutter is pressed, the gyroscope sensor 180B detects the angle of jitter of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and makes the lens offset the jitter of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenarios.

[0111] The barometric pressure sensor 180C is used to measure barometric pressure. In some embodiments, the electronic device 100 calculates the altitude according to the barometric pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0112] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the leather case or the opening and closing state of the flip, features such as automatic flip unlocking are set.

[0113] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0114] A distance sensor 180F for measuring distance. The electronic device 100 can measure distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0115] The proximity light sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The light-emitting diode may be an infrared light-emitting diode. The electronic device 100 emits infrared light outward through the light-emitting diode. The electronic device 100 uses the photodiode to detect the infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear during a call, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used for automatic unlocking and locking of the holster mode and pocket mode.

[0116] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance during photography. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket to prevent accidental touch.

[0117] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering calls, etc.

[0118] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 caused by low temperature. In some other embodiments, when the temperature is lower than yet another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0119] The touch sensor 180K, also known as the "touch panel". The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also known as the "touch control screen". The touch sensor 180K is used to detect touch operations acting thereon or nearby. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a different position from that of the display screen 194.

[0120] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals of the vibrating bone mass of the human vocal part. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure pulsation signals. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone to form a bone conduction earphone. The audio module 170 can parse out voice signals based on the vibration signals of the vibrating bone mass of the vocal part acquired by the bone conduction sensor 180M to implement the voice function. The application processor can parse out heart rate information based on the blood pressure pulsation signals acquired by the bone conduction sensor 180M to implement the heart rate detection function.

[0121] The button 190 includes a power-on button, a volume button, etc. The button 190 can be a mechanical button or a touch button. The electronic device 100 can receive button inputs to generate key signal inputs related to the user settings and function controls of the electronic device 100.

[0122] The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration prompts and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. Touch operations acting on different regions of the display screen 194 can also correspond to different vibration feedback effects for the motor 191. Different application scenarios (such as time reminder, receiving messages, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0123] The indicator 192 can be an indicator light and can be used to indicate the charging state, power change, and can also be used to indicate messages, missed calls, notifications, etc.

[0124] The SIM card interface 195 is used to connect to a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact with and separation from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communication. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0125] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present invention, the Android system with a layered architecture is taken as an example to exemplarily illustrate the software structure of the electronic device 100.

[0126] Figure 4 It is a software structure block diagram of the electronic device 100 in the embodiments of the present invention.

[0127] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0128] The application layer may include a series of application packages.

[0129] As Figure 4 shown, the application packages may include applications such as a camera, a gallery, a calendar, a call, a map, a navigation, a WLAN, a Bluetooth, music, a video, and a short message.

[0130] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0131] As Figure 4 shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0132] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0133] The content provider is used to store and obtain data, and make this data accessible to application programs. The data may include videos, images, audio, incoming and outgoing calls, browsing history and bookmarks, phone books, etc.

[0134] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build application programs. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.

[0135] The phone manager is used to provide the communication function of the electronic device 100. For example, the management of call states (including answering, hanging up, etc.).

[0136] The resource manager provides various resources for application programs, such as localized strings, icons, pictures, layout files, video files, etc.

[0137] The notification manager enables application programs to display notification information in the status bar. It can be used to convey message types of notifications, which can automatically disappear after a short stay without user interaction. For example, the notification manager is used to inform that the download is completed, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or a scroll bar text, such as a notification of a background running application program, or a notification that appears in the form of a dialogue window on the screen. For example, prompting text information in the status bar, emitting a prompt sound, vibrating the electronic device, flashing the indicator light, etc.

[0138] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for the scheduling and management of the Android system.

[0139] The core libraries contain two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core libraries of Android.

[0140] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as the management of object life cycles, stack management, thread management, security and exception management, and garbage collection.

[0141] The system library may include one or more high-definition texture dictionary libraries and multiple functional modules. For example: surface manager, Media Libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0142] Among them, one high-definition texture dictionary library corresponds to one type of shooting target. The high-definition texture dictionary library may include a large number of high-definition texture features of the shooting target type. For example, the shooting target types may include human faces, buildings, green plants, animals such as cats, dogs, birds, and so on. A human face may correspond to the high-definition texture dictionary library 1, and the high-definition texture dictionary library 1 may include a large number of high-definition human face texture features. A building may correspond to the high-definition texture dictionary library 2, and the high-definition texture dictionary library 2 may include a large number of high-definition building texture features. Green plants may correspond to the high-definition texture dictionary library 3, and the high-definition texture dictionary library 3 may include a large number of high-definition green plant texture features. Animals such as cats may correspond to the high-definition texture dictionary library 4, and the high-definition texture dictionary library 4 may include a large number of high-definition texture features of animals such as cats. Animals such as dogs may correspond to the high-definition texture dictionary library 5, and the high-definition texture dictionary library 5 may include a large number of high-definition texture features of animals such as dogs. Animals such as birds may correspond to the high-definition texture dictionary library 6, and the high-definition texture dictionary library 6 may include a large number of high-definition texture features of animals such as birds.

[0143] The above examples are only used to explain the present application and should not constitute a limitation.

[0144] The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple application programs.

[0145] The media library supports the playback and recording of a variety of common audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0146] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc.

[0147] The 2D graphics engine is a drawing engine for 2D drawing.

[0148] The kernel layer is the layer between hardware and software. The kernel layer at least includes a display driver, a camera driver, an audio driver, and a sensor driver.

[0149] Next, in combination with the capture and photo-taking scenario, the working processes of the software and hardware of the electronic device 100 are exemplarily described.

[0150] When the touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including information such as touch coordinates and the timestamp of the touch operation). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation and the control corresponding to the click operation being the control of the camera application icon as an example, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture a static image or video through the camera 193.

[0151] Next, in combination with an application scenario, a method for image reconstruction provided by an embodiment of the present application will be introduced.

[0152] In some application scenarios, when a user takes a photo using the electronic device 100, the user can open the camera application to take a photo. When the electronic device 100 displays a shooting preview interface, the electronic device 100 can preprocess the preview image stream, detect the specified target content, and load the high-definition texture dictionary library corresponding to the specified target content into the running memory. When the electronic device 100 receives a shooting operation from the user on the shooting preview interface, the electronic device 100 can obtain the low-definition image captured by the camera at this time, and crop the area where the specified target content is located from the low-definition image to obtain a low-definition cropped image. Then, the electronic device 100 can extract low-definition features from the low-definition cropped image, match the high-definition texture features corresponding to the content similar to the low-definition features from the high-definition texture dictionary library loaded into the running memory, and fuse the high-definition texture features into the low-definition cropped image to obtain a high-definition cropped image. Then, the high-definition cropped image is used to replace the low-definition cropped image and is pasted back to the area where the low-definition cropped image is located in the low-definition image to obtain a high-definition image. The electronic device 100 can save the high-definition image locally. In this way, the time for the electronic device to perform super-resolution repair on the captured low-definition image after taking a photo can be reduced.

[0153] Exemplarily, such as Figure 5AAs shown, the electronic device 100 can display an interface 510 of the main screen, and a page with application icons is displayed in the interface 510. This page includes multiple application icons (for example, weather application icon, stock application icon, calculator application icon, settings application icon, mail application icon, Alipay application icon, Facebook application icon, browser application icon, gallery application icon 512, music application icon, video application icon, WeChat application icon, etc.). Below the multiple application icons, a page indicator is also displayed to indicate the positional relationship between the currently displayed page and other pages. Below the page indicator, there are multiple tray icons (such as dial application icon, message application icon, contacts application icon, camera application icon 513), and the tray icons remain displayed during page switching. In some embodiments, the above page may also include multiple application icons and a page indicator. The page indicator may not be part of the page and exists separately. The above picture icons are also optional, and the embodiments of the present application do not limit this. In the upper part of the interface 510, a status bar 511 is displayed. The status bar 511 may include: one or more signal strength indicators of mobile communication signals (also known as cellular signals), battery status indicator, time indicator, Wi-Fi signal indicator, etc.

[0154] The electronic device 100 can receive an input operation (such as a click) by the user on the camera application icon 513. In response to this input operation, the electronic device 100 can display as Figure 5B shown in the shooting preview interface 520.

[0155] As Figure 5B shown, the shooting preview interface 520 can display including an echo control 521 of the captured image, a shooting control 522, a camera switching control 523, a preview screen 524 captured by the camera, a settings control 525, a zoom ratio control 526, one or more shooting mode controls (for example, "night mode" control 527A, "portrait mode" control 527B, "ordinary shooting mode" control 527C, "video recording mode" control 527D, "professional mode" control 527E, more mode control 527F, "large aperture mode" control 527H, etc.). Among them, the echo control 521 of the captured image can be used to display the captured image. The shooting control 522 is used to trigger saving the image captured by the camera. The camera switching control 523 can be used to switch the camera for taking pictures. The settings control 525 can be used to set the shooting function. The zoom ratio control 526 can be used to set the zoom ratio of the camera. The shooting mode control can be used to trigger starting the image processing process corresponding to the shooting mode. For example, the "night mode" control 527A can be used to trigger increasing the brightness and color richness in the captured image, etc. The "portrait mode" control 572B can be used to trigger blurring the background of the person in the captured image, etc. AsFigure 5B As shown, the shooting mode selected by the current user is the "ordinary shooting mode".

[0156] Among them, when the electronic device 100 displays the shooting preview interface 520 as Figure 5B shown, the electronic device 100 can detect whether the preview screen 524 includes specified image content (for example, a Ferris wheel building, etc.). If it does, the electronic device 100 can load the high-definition texture dictionary library corresponding to the specified image content into the running memory.

[0157] Optionally, when the electronic device 100 identifies the category of the specified target content in the preview stream, it can display scene information 529 in the above-mentioned shooting preview interface 520. The scene information 529 can be used to indicate the type of the specified target content in the preview stream. For example, the scene information 529 can be the text "Ferris wheel".

[0158] In a possible implementation manner, after the electronic device 100 loads the high-definition texture dictionary library corresponding to the specified image content into the running memory, the electronic device 100 can display the above-mentioned scene information 529.

[0159] In a possible implementation manner, when the electronic device 100 identifies that the preview screen 524 includes the specified image content, it can display an identification prompt on the above-mentioned shooting preview interface 520. The identification prompt is used to indicate the category of the specified image content. For example, if the specified image content is a "Ferris wheel", the electronic device 100 can display the text prompt "Ferris wheel" on the shooting preview interface 520 or specifically at the position where the specified image content is located.

[0160] After the electronic device 100 loads the high-definition texture dictionary library corresponding to the specified image content into the running memory, the electronic device 100 can receive an input operation (such as a click) from the user for the shooting control 522 in the shooting preview interface 520. In response to this input, the electronic device 100 can obtain a low-definition image captured by the camera and perform super-resolution restoration and reconstruction on the low-definition image through the high-definition texture dictionary library corresponding to the specified image content to obtain a high-definition image.

[0161] After the electronic device 100 performs super-resolution restoration and reconstruction on the low-definition image through the high-definition texture dictionary library to obtain a high-definition image, it can save the high-definition image locally.

[0162] As Figure 5C shown, the electronic device 100 can receive an input operation (such as a click) from the user for the captured image echo control 521. In response to this input operation, the electronic device 100 can display a photo browsing interface 550 as Figure 5D shown.

[0163] As shown Figure 5D In the photo browsing interface 550, there may be a high-definition image 551 after super-resolution reconstruction, image-related information 553, a menu 554, and a gallery control 555. Among them, the image-related information 553 may include the shooting time, shooting weather, geographical location information, etc. of the high-definition image 551. For example, the shooting time may be "8:00 AM, December 3, 2019", the shooting weather may be "cloudy", and the shooting location may be "Shanghai·Disneyland", etc. The menu 554 may include a share button, a favorite button, an edit button, a delete button, and a more button. The share button can be used to trigger the sharing of the high-definition image 551. The favorite button can be used to trigger the collection of the high-definition image 551 to the picture collection folder. The edit button can be used to trigger editing functions such as rotating, trimming, adding filters, and blurring of the high-definition image 551. The delete button can be used to trigger the deletion of the high-definition image 551. The more button can be used to trigger the opening of more functions related to the high-definition image 551. The gallery control 555 can be used to trigger the electronic device 100 to open the gallery application.

[0164] In some embodiments, when the user uses the electronic device 100 to take a photo, the camera application can be opened to take a photo. When the electronic device 100 displays the shooting preview interface, the shooting preview interface may include an image reconstruction control and a shooting control. The image reconstruction control can be used to trigger the electronic device 100 to perform super-resolution reconstruction on the captured low-definition image. After the electronic device 100 receives the input from the user for the image reconstruction control, it can preprocess the preview image stream, detect the specified target content, and load the high-definition texture dictionary library corresponding to the specified target content into the running memory. When the electronic device 100 receives the shooting operation on the shooting preview interface from the user, the electronic device 100 can obtain the low-definition image captured by the camera at this time, and crop the area where the specified target content is located from the low-definition image to obtain a low-definition cropped image. Then, the electronic device 100 can extract low-definition features from the low-definition cropped image, match the high-definition texture features corresponding to the content similar to the low-definition features from the high-definition texture dictionary library loaded into the running memory, and fuse the high-definition texture features into the low-definition cropped image to obtain a high-definition cropped image. Then, the high-definition cropped image is used to replace the low-definition cropped image and pasted back to the area where the low-definition cropped image is located in the low-definition image to obtain a high-definition image. The electronic device 100 can save the high-definition image locally. In this way, the time for the electronic device to perform super-resolution repair on the captured low-definition image after taking a photo is reduced.

[0165] Exemplarily, the electronic device 100 can receive an input operation (such as a click) on the camera application icon 513 in the interface 510 shown above by the earphone. In response to this input operation, the electronic device 100 can display as shown Figure 5A In the interface 510, the input operation (such as a click) on the camera application icon 513. In response to this input operation, the electronic device 100 can display as shown Figure 6AThe captured preview interface 620 shown.

[0166] As Figure 6A shown, the captured preview interface 620 can display an echo control 621 for the captured image, a capture control 622, a camera switching control 623, a preview screen 624 captured by the camera, an image reconstruction control 628, a setting control, a zoom ratio control, one or more shooting mode controls (for example, "night scene mode" control 627A, "portrait mode" control 627B, "ordinary photo shooting mode" control 627C, "video recording mode" control 627D, "professional mode" control 627E, more mode control 627F, "large aperture mode" control 627H, etc.). Among them, the image reconstruction control 628 can be used to trigger the electronic device 100 to perform super-resolution reconstruction on the captured low-resolution image. For the textual descriptions of the echo control 621 for the captured image, the capture control 622, the camera switching control 623, the setting control, the zoom ratio control, and one or more shooting mode controls, reference can be made to the foregoing Figure 5B shown embodiments and will not be elaborated herein.

[0167] The electronic device 100 can receive an input operation (such as a click) from the user for the image reconstruction control 628. In response to this input operation, the electronic device 100 can enable the image reconstruction function. After enabling the image reconstruction function, the electronic device 100 can detect whether the preview screen 624 includes a specified image content (such as a Ferris wheel building, etc.). If it does, the electronic device 100 can load the high-definition texture dictionary library corresponding to the specified image content into the operating memory.

[0168] In the embodiments of the present application, the above-mentioned image reconstruction control 628 can be referred to as the first control. The input for the image reconstruction control 628 can be referred to as the second input.

[0169] As Figure 6B shown, when the electronic device 100 enables the image reconstruction function, the image reconstruction control 628 can be displayed as an enabled state.

[0170] After the electronic device 100 loads the high-definition texture dictionary library corresponding to the specified image content into the operating memory, the electronic device 100 can receive an input operation (such as a click) from the user for the capture control 622 in the captured preview interface 620. In response to this input, the electronic device 100 can obtain the low-resolution image captured by the camera and perform super-resolution restoration and reconstruction on the low-resolution image through the high-definition texture dictionary library corresponding to the specified image content to obtain a high-definition image.

[0171] Optionally, after the electronic device 100 identifies the category of the specified target content in the preview stream, it may display the scene information 629 in the above-mentioned shooting preview interface 620. The scene information 629 can be used to indicate the type of the specified target content in the preview stream. For example, the scene information 629 can be the text "Ferris wheel".

[0172] In a possible implementation manner, after the electronic device 100 loads the high-definition texture dictionary library corresponding to the specified image content into the operating memory, the electronic device 100 may display the above-mentioned scene information 629.

[0173] After the electronic device 100 performs super-resolution restoration and reconstruction on the low-resolution image through the high-definition texture dictionary library to obtain a high-definition image, it may save the high-definition image locally.

[0174] In some embodiments, when the user uses the electronic device 100 to take a photo, the user can take a photo in different shooting modes of the electronic device 100. When the user feels that the resolution of the preview image displayed in the shooting preview interface is low, the user can select the "Image Super-Resolution Mode" in the camera application to take a photo. In the "Image Super-Resolution Mode", the electronic device 100 can preprocess the preview image stream, detect the specified target content, and load the high-definition texture dictionary library corresponding to the specified target content into the operating memory. When the electronic device 100 receives the shooting operation of the user on the shooting preview interface, the electronic device 100 can obtain the low-resolution image captured by the camera at this time, and crop the area where the specified target content is located from the low-resolution image to obtain a low-resolution cropped image. Then, the electronic device 100 can extract low-resolution features from the low-resolution cropped image, match the high-definition texture features corresponding to the content similar to the low-resolution features from the high-definition texture dictionary library loaded into the operating memory, and fuse the high-definition texture features into the low-resolution cropped image to obtain a high-definition cropped image. Then, the high-definition cropped image is used to replace the low-resolution cropped image and pasted back to the area where the low-resolution cropped image is located in the low-resolution image to obtain a high-definition image. The electronic device 100 can save the high-definition image locally. In this way, the time for the electronic device to perform super-resolution restoration on the captured low-resolution image after taking a photo is reduced.

[0175] Exemplarily, the electronic device 100 can receive an input operation (such as a click) of the user on the camera application icon 513 in the interface 510 shown above Figure 5A In response to this input operation, the electronic device 100 can display the shooting preview interface 720 as shown Figure 7A In the figure.

[0176] As shown Figure 7AAs shown, the shooting preview interface 720 may display an echo control 721 for the captured image, a shooting control 722, a camera switching control 723, a preview screen 724 captured by the camera, a setting control, a zoom ratio control, one or more shooting mode controls (e.g., "night scene mode" control 727A, "portrait mode" control 727B, "ordinary photo shooting mode" control 727C, "image super-resolution mode" control 727G, "video recording mode" control 727D, "professional mode" control 727E, more mode control 727F, etc.). Among them, the "image super-resolution mode" control 727G can be used to trigger the electronic device 100 to perform super-resolution reconstruction on the captured low-resolution image. For the text descriptions of the echo control 721 for the captured image, the shooting control 722, the camera switching control 723, the setting control, the zoom ratio control, and one or more shooting mode controls, reference can be made to the foregoing Figure 5B illustrated embodiments and will not be elaborated herein. As Figure 7A shown, the currently selected shooting mode by the current user is the "ordinary photo shooting mode".

[0177] The electronic device 100 may receive an input operation (such as a click) by the user on the "image super-resolution mode" control 727G. In response to this input operation, as Figure 7B shown, the electronic device 100 may adjust the currently selected shooting mode to the "image super-resolution mode".

[0178] In the embodiments of the present application, the above "image super-resolution mode" control 727G may be referred to as the first control. The input of the "image super-resolution mode" control 727G may be referred to as the second input.

[0179] After the shooting mode of the electronic device 100 is adjusted to the "image super-resolution mode", the electronic device 100 may detect whether the preview screen 624 includes a specified image content (such as a Ferris wheel building, etc.). If it does, the electronic device 100 may load the high-definition texture dictionary library corresponding to the specified image content into the running memory.

[0180] After the electronic device 100 loads the high-definition texture dictionary library corresponding to the specified image content into the running memory, the electronic device 100 may receive an input operation (such as a click) by the user on the shooting control 622 in the shooting preview interface 620. In response to this input, the electronic device 100 may obtain the low-resolution image captured by the camera and perform super-resolution restoration and reconstruction on the low-resolution image through the high-definition texture dictionary library corresponding to the specified image content to obtain a high-definition image.

[0181] Optionally, after the electronic device 100 identifies the category of the specified target content in the preview stream, it may display scene information 729 in the above-mentioned shooting preview interface 720. The scene information 729 can be used to indicate the type of the specified target content in the preview stream. For example, the scene information 729 may be the text "Ferris wheel".

[0182] In a possible implementation manner, after the electronic device 100 loads the high-definition texture dictionary library corresponding to the specified image content into the operating memory, the electronic device 100 may display the above-mentioned scene information 729.

[0183] After the electronic device 100 performs super-resolution repair and reconstruction on the low-resolution image through the high-definition texture dictionary library to obtain a high-definition image, it may save the high-definition image locally.

[0184] In some application scenarios, the user may open the camera application of the electronic device 100 to shoot a video. When the electronic device 100 displays the video recording preview interface, the electronic device 100 may preprocess the preview image stream, detect the specified target content, and load the high-definition texture dictionary library corresponding to the specified target content into the operating memory. After the electronic device 100 receives the video recording start operation of the user on the video recording preview interface, the electronic device 100 may perform super-resolution reconstruction on each frame of the video stream obtained by the camera through the high-definition texture dictionary library loaded into the operating memory. The electronic device 100 may display the high-definition images reconstructed from each frame of the video stream in sequence according to the frame order on the video recording interface. Among them, the process of performing super-resolution reconstruction on each frame of the video stream by using the high-definition texture dictionary library may refer to the foregoing embodiments and will not be elaborated herein. After the electronic device 100 receives the video recording end operation, the electronic device 100 may save the image stream displayed on the video recording interface during the period from the start to the end of the video recording as a video file. In this way, the time for the electronic device to perform super-resolution repair on the low-definition video recording screen during video recording can be reduced.

[0185] Exemplarily, the electronic device 100 may receive an input operation (such as a click) of the user on the camera application icon 513 in the interface 510 shown above Figure 5A In response to the input operation, the electronic device 100 may display a shooting preview interface 820 as Figure 8A shown.

[0186] As Figure 8AAs shown, the shooting preview interface 820 may display an echo control 821 for the captured image, a shooting control 822, a camera switching control 823, a preview screen 824 captured by the camera, a setting control, a zoom ratio control, one or more shooting mode controls (e.g., "night scene mode" control 827A, "portrait mode" control 827B, "ordinary shooting mode" control 827C, "video recording mode" control 827D, "professional mode" control 827E, more mode control 827F, "large aperture mode" control 827E, etc.). For the text descriptions of the echo control 821 for the captured image, the shooting control 822, the camera switching control 823, the setting control, the zoom ratio control, and one or more shooting mode controls, reference may be made to the foregoing Figure 5B illustrated embodiment, which will not be elaborated herein. As Figure 8A shown, the shooting mode currently selected by the user is the "ordinary shooting mode".

[0187] The electronic device 100 may receive an input operation (e.g., a click) by the user on the "video recording mode" control 827G. In response to this input operation, as Figure 8B shown, the electronic device 100 may adjust the currently selected shooting mode to the "video recording mode", and the electronic device 100 may display a video recording start control 825 on the shooting preview interface 820.

[0188] After the shooting mode of the electronic device 100 is adjusted to the "video recording mode", the electronic device 100 may detect whether the preview screen 824 includes specified image content (e.g., a Ferris wheel building, etc.). If so, the electronic device 100 may load the high-definition texture dictionary library corresponding to the specified image content into the operating memory.

[0189] After the electronic device 100 loads the high-definition texture dictionary library corresponding to the specified image content into the operating memory, the electronic device 100 may receive an input operation (e.g., a click) by the user on the video recording start control 825 in the shooting preview interface 820. In response to this input operation, the electronic device 100 may display a Figure 8C video recording interface 830 as shown, and obtain the image stream captured by the camera. The electronic device 100 may perform super-resolution reconstruction on each frame of the image in the image stream obtained by the camera through the high-definition texture dictionary library loaded into the operating memory. The electronic device 100 may sequentially display the high-definition images reconstructed from each frame of the image in the image stream in the video recording interface according to the frame order.

[0190] As Figure 8C shown, the video recording interface 830 may display a recording time box 831, a video recording end control 832, and a high-definition image 834. The recording time box 831 is used to display the recorded time.

[0191] For example, when the recording time is the 1st second, the electronic device 100 can super-resolution reconstruct a high-definition image 834 from the image stream obtained by the camera based on the high-definition texture dictionary library, and display the high-definition image 834 on the recording interface 830.

[0192] As Figure 8D shown, when the recording time is the 3rd second, the electronic device 100 can super-resolution reconstruct a high-definition image 844 from the image stream obtained by the camera based on the high-definition texture dictionary library, and display the high-definition image 844 on the recording interface 830.

[0193] After the electronic device 100 receives the video recording end operation, the electronic device 100 can save the image stream displayed on the recording interface during the period from the start to the end of the video recording as a video file.

[0194] Next, a method for image reconstruction provided in an embodiment of the present application will be introduced.

[0195] Figure 9 Exemplarily, a flowchart of a method for image reconstruction provided in an embodiment of the present application is shown.

[0196] As Figure 9 shown, the method may include the following steps:

[0197] S901. The electronic device 100 displays a shooting preview interface, and a shooting button and a preview image stream are displayed on the shooting preview interface.

[0198] Among them, the shooting preview interface may be the shooting preview interface 520 in the above Figure 5B shown embodiment, or may be the shooting preview interface 620 in the above Figure 6B shown embodiment, or may be the shooting preview interface 720 in the above Figure 7B shown embodiment, or may be the shooting preview interface 820 in the above Figure 8B shown embodiment. For specific content, reference may be made to the foregoing Figure 5B , Figure 6B , Figure 7B , Figure 8B shown embodiments, which will not be elaborated herein.

[0199] S902. The electronic device 100 detects a first shooting scene of the preview image stream.

[0200] Specifically, recognition algorithm models corresponding to different target types are pre - installed on the electronic device 100. The electronic device 100 can respectively use these recognition algorithm models corresponding to different target types to detect whether the preview image stream includes specified target content. If it includes the specified target content, the electronic device 100 can determine the first shooting scene based on the detected specified target content. Among them, the type of the specified target content can include any one of image contents such as human faces, buildings, green plants, cats, dogs, birds, etc.

[0201] Among them, multiple feature templates of specified target content can be stored on the electronic device 100. The electronic device 100 can select one or more preview images from the preview image stream and detect whether there are similar regions in these one or more preview images whose feature similarity to the feature template of the first specified target content is greater than a preset value. If so, the electronic device 100 can determine that the type of the first specified target content is the first shooting scene.

[0202] For example, when the electronic device 100 detects that the preview image stream includes an image content of a human face, the electronic device 100 can determine that the first shooting scene is a face - shooting scene. When the electronic device 100 detects that the preview image stream includes an image content of a building, the electronic device 100 can determine that the first shooting scene is a building - shooting scene. When the electronic device 100 detects that the preview image stream includes an image content of green plants, the electronic device 100 can determine that the first shooting scene is a green - plant - shooting scene. When the electronic device 100 detects that the preview image stream includes an image content of a cat, the electronic device 100 can determine that the first shooting scene is a cat - shooting scene. When the electronic device 100 detects that the preview image stream includes an image content of a dog, the electronic device 100 can determine that the first shooting scene is a dog - shooting scene. When the electronic device 100 detects that the preview image stream includes an image content of a bird, the electronic device 100 can determine that the first shooting scene is a bird - shooting scene. The above examples are only used to explain the present application and should not constitute a limitation.

[0203] Optionally, when the electronic device 100 recognizes the category of the specified target content in the preview stream, it can display scene information in the above - mentioned shooting preview interface. The scene information can be used to indicate the category of the specified target content in the preview stream. For example, the scene information can be the text "Ferris wheel".

[0204] In a possible implementation, after the electronic device 100 loads the high - definition texture dictionary library corresponding to the specified image content into the operating memory, the electronic device 100 can display the above - mentioned scene information.

[0205] In a possible implementation, when the electronic device 100 can detect multiple specified target contents from the preview image stream, the electronic device 100 can mark the types of these multiple specified target contents on the shooting preview interface. The electronic device 100 can receive a selection input from the user for the first specified target content among the multiple specified target contents (for example, click on the area where the first specified target is located on the shooting preview interface). In response to this selection input, the electronic device 100 can determine the type of the first specified target content as the first shooting scene.

[0206] For example, the electronic device 100 can detect the target "human face" and the target "animal bird" in the preview image stream. The electronic device 100 can mark the positions of the target "human face" and the target "animal bird" on the shooting preview interface. The electronic device 100 can receive a click operation from the user on the position of the target "human face" on the shooting preview interface. In response to this click operation, the electronic device 100 can determine that the first shooting scene is a human face shooting scene.

[0207] S903. The electronic device 100 determines the first high-definition texture dictionary library corresponding to the first shooting scene and loads the first high-definition texture dictionary library into the running memory.

[0208] Among them, since the texture features of different target contents are different, when performing super-resolution reconstruction on a low-resolution image, it is necessary to repair the texture of the low-resolution image based on the high-definition texture features of the target content. Therefore, a high-definition texture dictionary library corresponding to each of multiple shooting scenes can be stored in the disk (i.e., ROM) of the electronic device 100. Each high-definition texture dictionary library stores the high-definition texture features of a specified target content.

[0209] Exemplarily, the multiple shooting scenes stored on the electronic device 100 can be as shown in Table 1 below:

[0210] Table 1

[0211] Shooting Scenario High - Definition Texture Dictionary Library Face Shooting Scenario High - Definition Texture Dictionary Library 1 Building Shooting Scenario High - Definition Texture Dictionary Library 2 Green Plant Shooting Scenario High - Definition Texture Dictionary Library 3 Animal Cat Shooting Scenario High - Definition Texture Dictionary Library 4 Animal Dog Shooting Scenario High - Definition Texture Dictionary Library 5 Animal Bird Shooting Scenario High - Definition Texture Dictionary Library 6 … …

[0212] As can be seen from Table 1 above, the disk of the electronic device 100 can store High-Definition Texture Dictionary Library 1, High-Definition Texture Dictionary Library 2, High-Definition Texture Dictionary Library 3, High-Definition Texture Dictionary Library 4, High-Definition Texture Dictionary Library 5, High-Definition Texture Dictionary Library 6, and so on. Among them, High-Definition Texture Dictionary Library 1 corresponds to the face shooting scene, and this High-Definition Texture Dictionary Library 1 includes the high-definition texture features of the face. High-Definition Texture Dictionary Library 2 corresponds to the building shooting scene, and this High-Definition Texture Dictionary Library 2 includes the high-definition texture features of the building. High-Definition Texture Dictionary Library 3 corresponds to the green plant shooting scene, and this High-Definition Texture Dictionary Library 3 includes the high-definition texture features of the green plants. High-Definition Texture Dictionary Library 4 corresponds to the animal cat shooting scene, and this High-Definition Texture Dictionary Library 4 includes the high-definition texture features of the animal cat. High-Definition Texture Dictionary Library 5 corresponds to the animal dog shooting scene, and this High-Definition Texture Dictionary Library 4 includes the high-definition texture features of the animal dog. High-Definition Texture Dictionary Library 6 corresponds to the animal bird shooting scene, and this High-Definition Texture Dictionary Library 6 includes the high-definition texture features of the animal bird. Table 1 above is only used for exemplary explanation of the present application and should not constitute a limitation.

[0213] In specific implementation, the ROM on the electronic device 100 can store more or fewer high-definition texture dictionary libraries corresponding to shooting scenes, which is not limited in the present application.

[0214] After the electronic device 100 determines the first high-definition texture dictionary library corresponding to the first shooting scene, it can load the first high-definition texture dictionary library from the disk (ROM) into the running memory (RAM). In this way, it can save the time for super-resolution reconstruction of the captured photo after the user takes a photo.

[0215] Among them, the disk (i.e., readonlymemory, ROM) can still retain data after power-off and can be used to save data such as documents, pictures, music, videos, processing algorithm models, dictionary libraries, etc. The running memory (i.e., randomaccessmemory, RAM) can be used to store temporary data generated during the operation of the operating system and applications. For example, when the electronic device 100 needs to process the captured picture based on the dictionary library, it needs to load the dictionary library in the disk into the running memory and also load the picture into the running memory. Then, the electronic device 100 calls one or more processors to process the picture based on the dictionary library in the running memory.

[0216] In the embodiments of the present application, the disk on the electronic device 100 may store other types of dictionary libraries in addition to the high-definition texture dictionary library, such as filter template algorithms for different shooting scenes, etc. In the embodiments of the present application, the first dictionary library may refer to the high-definition texture dictionary library corresponding to the first shooting scene, or may also refer to other types of dictionary libraries corresponding to the first shooting scene (for example, the filter template algorithm corresponding to the first shooting scene).

[0217] S904. The electronic device 100 obtains a low - definition image captured by the camera in response to the user's input for the shooting key in the shooting preview interface.

[0218] Among them, the shooting key can be the shooting control 522 in the shooting preview interface 520 shown above, Figure 5B or the shooting control 622 in the shooting preview interface 620 shown above, Figure 6B or the shooting control 722 in the shooting preview interface 720 shown above. Figure 7B The shooting key can be the shooting control 722 in the shooting preview interface 720 shown above.

[0219] In a possible implementation, the shooting key can also be the video recording start control 825 in the shooting preview interface 820 shown above. The low - definition image can refer to each frame of the image captured by the camera after the electronic device 100 starts video recording, or it can refer to each frame of the image obtained by sampling at a certain frame rate interval from the image stream captured by the camera after the electronic device 100 starts video recording. Figure 8B

[0220] S905. The electronic device 100 identifies and crops the first region where the specified target content is located in the low - definition image to obtain a low - definition cropped image.

[0221] S906. The electronic device 100 can match the high - definition texture features similar to the content in the low - definition cropped image from the first high - definition dictionary library loaded in the running memory, and fuse the matched high - definition texture features into the low - definition cropped image to obtain a high - definition cropped image.

[0222] S907. The electronic device 100 can replace the low - definition cropped image with the high - definition cropped image and paste it back to the first region in the low - definition image to obtain a high - definition image.

[0223] The electronic device 100 can crop the image in the face frame based on the enlarged face frame to obtain a low - definition cropped image. The electronic device 100 can extract the feature information of each facial feature position from the low - definition cropped image based on the facial feature information. Then, the electronic device 100 can match the high - definition texture features corresponding to each facial feature from the first high - definition dictionary library recorded in the running memory. After the electronic device 100 obtains the high - definition texture features corresponding to each facial feature, it can fuse the high - definition texture features corresponding to each facial feature into the low - definition cropped image to obtain a high - definition cropped image. Then, the electronic device 100 can paste the high - definition cropped image back to the face area in the low - definition image based on the binary mask image of the facial feature information to obtain a high - definition image.

[0224] The above examples are only used to explain the present application and should not constitute a limitation.

[0225] In a possible implementation, when the electronic device 100 detects a first shooting scene in the preview image stream, it can determine a first area based on the first shooting scene. Specifically, the electronic device 100 can determine the first area in the low-resolution image based on the position of the specified target content in the preview image stream. For example, when the specified target content included in the preview image stream is a human face, the electronic device 100 can extract the face key points from the preview image stream, and the electronic device 100 can obtain the facial feature information and hair information in the preview image stream based on the face key points. The electronic device can generate a binary mask image of the preview image stream based on the facial feature information. After the electronic device 100 captures a low-resolution image, the electronic device 100 can determine the first area in the low-resolution image based on the position of the specified target content marked in the binary mask image of the preview image stream, and crop the first area in the low-resolution image to obtain a low-resolution cropped image. In this way, regarding the position of the specified target content in the preview image stream as the position of the specified target content in the actually captured low-resolution image can reduce the super-resolution processing time after actually capturing the low-resolution image.

[0226] S908. The electronic device 100 saves the high-resolution image.

[0227] Among them, the resolution of the specified target content in the high-resolution image is greater than the resolution of the specified target content in the low-resolution image.

[0228] Among them, the electronic device 100 can save the high-resolution image to the gallery. When the above low-resolution image can refer to each frame of the image captured by the camera after the electronic device 100 starts recording, or each frame of the image obtained by sampling at a certain frame rate interval in the image stream captured by the camera, the electronic device 100 can display the high-resolution image on the recording interface. For example, the recording interface can be the above Figure 8C or Figure 8D shown recording interface 830. The electronic device 100 can save the multiple high-resolution images displayed on the recording interface during recording as the pictures in the video file when ending the recording.

[0229] Through the image reconstruction method provided in the embodiments of the present application, it is possible to preprocess the preview image and load the high-definition texture dictionary library into the running memory (RAM) before the electronic device receives the user's shooting operation, that is, during the shooting preview. After receiving the user's shooting operation, the electronic device can perform super-resolution repair on the captured low-resolution image by using the high-definition texture dictionary library loaded in the running memory to obtain a high-resolution image. In this way, the time for the electronic device to perform super-resolution repair on the captured low-resolution image after taking a photo can be reduced.

[0230] Next, an image reconstruction method provided in another embodiment of the present application will be introduced.

[0231] Figure 10 The flowchart of an image reconstruction method provided by an embodiment of the present application is exemplarily shown.

[0232] As Figure 10 shown, the method may include the following steps:

[0233] S1001. The electronic device 100 displays a shooting preview interface, on which a shooting button and a preview image stream are displayed.

[0234] For specific content, reference may be made to step S901 in the foregoing Figure 9 shown embodiment, which will not be elaborated here.

[0235] S1002. The electronic device 100 detects a first shooting scene of the preview image stream.

[0236] For specific content, reference may be made to step S902 in the foregoing Figure 9 shown embodiment, which will not be elaborated here.

[0237] S1003. The electronic device 100 determines a first high-definition dictionary library corresponding to the first shooting scene.

[0238] For specific content, reference may be made to step S903 in the foregoing Figure 9 shown embodiment, which will not be elaborated here.

[0239] S1004. The electronic device 100 determines whether the first high-definition dictionary library has been loaded into the operating memory. If not, the electronic device 100 executes S1005. The electronic device 100 loads the first high-definition dictionary library into the operating memory. If so, the electronic device 100 directly executes S1006.

[0240] Among them, since the target content included in the preview video stream changes during the shooting preview process of the electronic device 100, the electronic device 100 may recognize that the target content 1 is included in the preview video stream at the first moment and determine that the preview image stream is the shooting scene 1. Therefore, the electronic device 100 can load the high-definition texture dictionary library 1 corresponding to the shooting scene 1 from the disk into the running memory. Then, when the electronic device 100 recognizes that the target content 2 is included in the preview video stream at the second moment, the electronic device 100 can determine that the preview image stream is the shooting scene 2. At this time, the electronic device 100 can determine whether the high-definition texture dictionary library 2 corresponding to the shooting scene 2 has been loaded into the running memory. If not, the electronic device 100 can load the high-definition texture dictionary library 2 corresponding to the shooting scene 2 into the running memory of the electronic device 100. If the high-definition texture dictionary library 2 corresponding to the shooting scene 2 has been loaded into the running memory, the electronic device 100 does not need to reload the high-definition texture dictionary library 2 into the running memory. In this way, the loading time of the high-definition texture dictionary library can be saved.

[0241] S1006. When the first high-definition dictionary library has been loaded into the running memory, the electronic device 100 can obtain a low-definition image captured by the camera in response to the user's input on the shooting button in the shooting preview interface.

[0242] For specific content, reference can be made to step S904 in the foregoing Figure 9 illustrated embodiment, which will not be elaborated here.

[0243] S1007. The electronic device 100 recognizes and crops the first area where the specified target content is located in the low-definition image to obtain a low-definition cropped image.

[0244] For specific content, reference can be made to step S905 in the foregoing Figure 9 illustrated embodiment, which will not be elaborated here.

[0245] S1008. The electronic device 100 matches the high-definition texture features similar to the content of the low-definition cropped image from the first high-definition dictionary library loaded into the running memory, and pastes the matched texture features back to the low-definition cropped image to obtain a high-definition cropped image.

[0246] For specific content, reference can be made to step S906 in the foregoing Figure 9 illustrated embodiment, which will not be elaborated here.

[0247] S1009. The electronic device 100 can replace the low-definition cropped image with the high-definition cropped image and paste it back to the first area in the low-definition image to obtain a high-definition image.

[0248] For specific content, reference can be made to step S907 in the foregoing Figure 9 illustrated embodiment, which will not be elaborated here.

[0249] S1010. The electronic device 100 stores the high-definition image.

[0250] For specific content, reference can be made to step S908 in the foregoing Figure 9 illustrated embodiment, which will not be elaborated herein.

[0251] Through the image reconstruction method provided in the embodiments of the present application, it is possible to preprocess the preview image, identify the shooting scene, and determine whether the running memory (RAM) includes a high-definition texture dictionary library corresponding to the shooting scene before the electronic device 100 receives the user's shooting operation, that is, during shooting preview. If not, the electronic device 100 loads the high-definition texture dictionary library corresponding to the shooting scene into the running memory. If the high-definition texture dictionary library corresponding to the shooting scene has been loaded into the running memory, the electronic device 100 does not need to reload the high-definition texture dictionary library corresponding to the shooting scene into the running memory. After receiving the user's shooting operation, the electronic device 100 can perform super-resolution repair on the low-definition image captured by using the high-definition texture dictionary library loaded in the running memory to obtain a high-definition image. In this way, the time for the electronic device 100 to perform super-resolution repair on the captured low-definition image after taking a photo can be reduced.

[0252] Next, an image reconstruction method provided in another embodiment of the present application will be introduced.

[0253] Figure 11 A schematic flowchart of an image reconstruction method provided in the embodiments of the present application is exemplarily shown.

[0254] As Figure 11 shown, the method may include the following steps:

[0255] S1101. The electronic device 100 displays a shooting preview interface, and a shooting button and a preview image stream are displayed on the shooting preview interface.

[0256] For specific content, reference can be made to step S901 in the foregoing Figure 10 illustrated embodiment, which will not be elaborated herein.

[0257] S1102. The electronic device 100 detects a first shooting scene of the preview image stream.

[0258] For specific content, reference can be made to step S902 in the foregoing Figure 10 illustrated embodiment, which will not be elaborated herein.

[0259] S1103. The electronic device 100 determines a first high-definition texture dictionary library corresponding to the first shooting scene and loads the first high-definition texture dictionary library into the running memory.

[0260] For specific content, reference can be made to step S903 in the foregoing Figure 10 illustrated embodiment, which will not be elaborated herein.

[0261] S1104. In response to the user's input for the shooting button in the shooting preview interface, the electronic device 100 obtains a low-definition image captured by the camera.

[0262] For specific content, reference can be made to step S904 in the foregoing Figure 10 illustrated embodiment, which will not be elaborated herein.

[0263] S1105. The electronic device 100 identifies and crops the first region where the specified target content is located in the low-definition image to obtain a low-definition cropped image.

[0264] For specific content, reference can be made to step S905 in the foregoing Figure 10 illustrated embodiment, which will not be elaborated herein.

[0265] S1106. The electronic device 100 can match the high-definition texture features similar to the content in the low-definition cropped image from the first high-definition dictionary library loaded in the running memory, and fuse the matched high-definition texture features into the low-definition cropped image to obtain a high-definition cropped image.

[0266] For specific content, reference can be made to step S906 in the foregoing Figure 10 illustrated embodiment, which will not be elaborated herein.

[0267] S1107. The electronic device 100 can replace the low-definition cropped image with the high-definition cropped image and paste it back to the first region in the low-definition image to obtain a high-definition image.

[0268] For specific content, reference can be made to step S907 in the foregoing Figure 10 illustrated embodiment, which will not be elaborated herein.

[0269] S1108. The electronic device 100 can determine whether the repair effect of the high-definition image meets a preset condition.

[0270] Among them, the electronic device 100 can determine whether the clarity of the high-definition image is greater than a preset clarity. If the clarity of the high-definition image is greater than the preset clarity, the electronic device 100 can determine that the repair effect of the high-definition image meets the preset condition. If the clarity of the high-definition image is less than the preset clarity, the electronic device 100 can determine that the repair effect of the high-definition image meets the preset condition.

[0271] Among them, the clarity algorithm used by the electronic device 100 to calculate the clarity of the high-definition image may include any one of the following: Brenner gradient function, Tenengrad gradient function, Laplacian gradient function, gray variance (SMD) function, gray variance product (SMD2) function, variance function, energy gradient function, Vollath function, entropy function, EAV point sharpness algorithm function, Reblur secondary blurring algorithm, NRSS gradient structural similarity algorithm, and so on.

[0272] If the repair effect of the high-definition image meets the preset conditions, the following can be executed:

[0273] S1109. The electronic device 100 can save the high-definition image.

[0274] For specific content, reference can be made to step S908 in the foregoing Figure 10 illustrated embodiment, which will not be elaborated here.

[0275] If the repair effect of the high-definition image does not meet the preset conditions, the following can be executed:

[0276] S1110. The electronic device 100 can save the low-definition image.

[0277] In a possible implementation manner, while saving the low-definition image, the electronic device 100 can also save the above-mentioned high-definition image.

[0278] S1111. After saving the low-definition image, the electronic device 100 updates the first high-definition texture dictionary library.

[0279] Specifically, after saving the low-definition image, the electronic device 100 can obtain multiple high-definition reference images under the first shooting scene from the picture library or the network, extract high-definition texture features from these multiple high-definition reference images, and store them in the first high-definition texture dictionary library to complete the update of the first high-definition texture dictionary library.

[0280] Exemplarily, for example, the first shooting scene is a face shooting scene. The electronic device 100 can obtain multiple high-definition face reference images from the picture library or the network. The electronic device 100 can perform the following operations for each high-definition face reference image: The electronic device 100 can extract the features of the high-definition face reference image at different scale sizes. Then, the electronic device 100 can detect the landmark points in the high-definition face reference image, and crop and resample the five sense organs such as the left eye, right eye, nose, and mouth at each scale size, so that the features of these five sense organs reach a fixed size. Then, the electronic device 100 can generate K clusters for each five sense organs through the K-means algorithm to obtain the high-definition texture dictionary features of each five sense organs. The electronic device 100 can store the high-definition texture dictionary features of each five sense organs into the first high-definition dictionary library to complete the update of the first high-definition texture dictionary library.

[0281] In a possible implementation manner, when the electronic device 100 detects that the user deletes the high-definition images super-resolution reconstructed in the first shooting scene more than a specified number of times (for example, 3 times) within a period of time (for example, 1 hour), the electronic device 100 can update the first high-definition texture dictionary library. The process of updating the first high-definition texture dictionary library can refer to the foregoing embodiments and will not be elaborated here.

[0282] Through the image reconstruction method provided by the embodiments of the present application, it is possible to preprocess the preview image, identify the shooting scene, and load the high-definition texture dictionary library corresponding to the shooting scene into the running memory before the electronic device 100 receives the user's shooting operation, that is, during shooting preview. After receiving the user's shooting operation, the electronic device 100 can perform super-resolution repair on the low-definition image captured by using the high-definition texture dictionary library loaded in the running memory to obtain a high-definition image. If the repair effect of the high-definition image is not good, the electronic device 100 can self-learn and update the high-definition texture dictionary library. In this way, the time for the electronic device 100 to perform super-resolution repair on the captured low-definition image after shooting can be reduced, and the effect of super-resolution reconstruction using the high-definition texture dictionary library can be improved.

[0283] Next, an image reconstruction system 1200 involved in the embodiments of the present application will be introduced.

[0284] Figure 12 The architecture diagram of an image reconstruction system 1200 provided in the embodiments of the present application is shown. Among them, the image reconstruction system 1200 can be applied to the above-mentioned electronic device 100.

[0285] As Figure 12As shown, the image reconstruction system 1200 may include a camera 1211, an Image Signal Processor (ISP) 1212, a Digital Signal Processor (DSP) 1213, an image processing module 1214, an Application Processor (AP) 1215, a disk (ROM) 1216, and a Random Access Memory (RAM) 1217. Among them, in some embodiments, the image processing module 1214 may be a Neural Network Processing Unit (NPU) or a Graphics Processing Unit (GPU). In other embodiments, the image processing module 1214 may also be a software processing module in the application processor 1215.

[0286] Among them, the camera 1211 can be used to capture an optical signal when starting a camera application or function, convert the optical signal into an electrical signal, and send it to the image signal processor 1212.

[0287] The image signal processor 912 can be used to convert the electrical signal sent by the camera 1211 into a digital image signal and send the digital image signal to the digital signal processor 1213.

[0288] The digital signal processor 1213 can be used to process the digital image signal into a preview image stream in a specified image format, and the specified image format can be a Raw format, a YUV format, an RGB format, etc.

[0289] The image processing module 1214 can be used to identify a first shooting scene from the preview image stream and send indication information of the first shooting scene to the application processor 1215.

[0290] The application processor 1215 can be used to match a first high-definition texture dictionary library corresponding to the first shooting scene from multiple high-definition texture dictionary libraries stored in the disk 1216, and load the first high-definition texture dictionary library from the disk 1216 into the random access memory 1217.

[0291] When the application processor 1215 detects a photographing event input by the user, the application processor 1215 can send a super-resolution start instruction to the image processing module 1214. The image processing module 1214 can be used to obtain a low-definition image from the image stream output by the digital signal processor 1213 in response to the super-resolution start instruction. After obtaining the low-definition image, the image processor module 1214 can perform super-resolution reconstruction on the low-definition image by using the first high-definition texture dictionary library loaded in the random access memory and output a high-definition image. Among them, for the process of performing super-resolution reconstruction on the low-definition image by using the first high-definition texture dictionary library loaded in the random access memory, reference can be made to the foregoing embodiments, and details are not described herein again.

[0292] An image reconstruction system 1200 is provided in an embodiment of the present application, which can preprocess a preview image and load a high-definition texture dictionary library into the running memory (RAM) before detecting a user's shooting operation, that is, during shooting preview. After receiving the user's shooting operation, the high-definition texture dictionary library already loaded in the running memory is used to perform super-resolution repair on the captured low-definition image to obtain a high-definition image. In this way, the time for performing super-resolution repair on the captured low-definition image after taking a photo can be reduced.

[0293] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An image reconstruction method, characterized in that, Including: The electronic device displays a shooting preview interface, and a shooting button and a preview image stream captured in real time by the camera are displayed on the shooting preview interface; The electronic device detects that the preview image stream includes a specified target content, and the type of the specified target content is the first type; The electronic device determines a first dictionary library corresponding to the first type from a high-definition texture dictionary library of multiple different shooting target categories, and loads the first dictionary library from the disk into the running memory, wherein the high-definition texture dictionary libraries of the multiple different shooting target types are stored in the disk of the electronic device; After the first dictionary library is loaded into the running memory, the electronic device receives a first input; In response to the first input, the electronic device acquires a low-definition image captured by the camera; The electronic device processes the low-definition image through the first dictionary library loaded in the running memory to obtain a high-definition image, and the resolution of the specified target content in the high-definition image is greater than the resolution of the specified target content in the low-definition image.

2. The method according to claim 1, wherein The electronic device processes the low-definition image through the first dictionary library loaded in the running memory to obtain a high-definition image, specifically including: The electronic device matches a first high-definition texture feature corresponding to the low-definition image from the first dictionary library loaded in the running memory; the first dictionary library includes high-definition texture features of the same shooting target category; The electronic device fuses the first high-definition texture feature into the low-definition image to obtain the high-definition image.

3. The method according to claim 2, wherein The electronic device matches a first high-definition texture feature corresponding to the low-definition image from the first dictionary library loaded in the running memory, specifically including: The electronic device identifies and crops a first area where the specified target content is located in the low-definition image to obtain a low-definition cropped image; The electronic device extracts a first image feature from the low-definition cropped image; The electronic device matches a first high-definition texture feature in the first dictionary library loaded in the running memory, the similarity of which to the first image feature is greater than a preset value.

4. The method according to claim 2, wherein The electronic device matches a first high-definition texture feature corresponding to the low-definition image from the first dictionary library loaded in the running memory, specifically including: The electronic device determines a first area based on the position of the specified target content in the preview image stream; The electronic device crops the first area in the low-definition image to obtain a low-definition cropped image; The electronic device extracts a first image feature from the low-definition cropped image; The electronic device matches a first high-definition texture feature in the first dictionary library loaded in the running memory, the similarity of which to the first image feature is greater than a preset value.

5. The method according to any one of claims 2-4, characterized in that, The electronic device fuses the first high-definition texture feature into the low-definition image to obtain the high-definition image, specifically including: The electronic device fuses the first high-definition texture feature into the low-definition cropped image to obtain a high-definition cropped image; The electronic device replaces the low-definition cropped image with the high-definition cropped image and pastes it back to the first area in the low-definition image to obtain the high-definition image.

6. The method according to any one of claims 1-4, characterized in that, A first control is further displayed on the shooting preview interface; before the electronic device receives a first input, the method further includes: The electronic device receives a second input for the first control; In response to the second input, the electronic device enables the super-resolution reconstruction mode.

7. The method according to claim 1, characterized in that, Before the electronic device loads the first dictionary library from the disk into the running memory, the method further includes: The electronic device determines whether the first dictionary library has been loaded into the running memory; The electronic device loading the first dictionary library from the disk into the running memory specifically includes: When the first dictionary library is not loaded in the running memory, the electronic device records the first dictionary library from the disk into the running memory.

8. The method according to claim 1, wherein After the electronic device detects that the preview image stream includes a specified target content, the method further includes: The electronic device displays scene information on the shooting preview interface, and the scene information is used to indicate that the type of the specified target content in the preview image stream is the first type.

9. The method according to any one of claims 1 to 4, characterized in that After the electronic device processes the low-definition image through the first dictionary library loaded in the running memory to obtain a high-definition image, the method further includes: The electronic device saves the high-definition image.

10. The method according to any one of claims 1-4, characterized in that After the electronic device fuses the first high-definition texture feature into the low-definition image to obtain a high-definition image, the method further includes: The electronic device determines whether the image quality of the high-definition image meets a preset condition. If so, the electronic device saves the high-definition image; If not, the electronic device saves the low-definition image.

11. The method according to claim 10, wherein If the electronic device determines that the image quality of the high-definition image does not meet the preset condition, the method further includes: The electronic device obtains multiple high-definition reference images of the first type from the gallery or the network, extracts high-definition texture features from the multiple high-definition reference images, and stores them in the first dictionary library.

12. The method according to claim 10, wherein The preset condition includes: the clarity of the high-definition image is greater than a preset clarity.

13. The method according to claim 1 or 8, characterized in that, The first type includes any one of the following: face, building, green plant, animal cat, animal dog, animal bird.

14. An electronic device, characterized in that, Including: One or more processors, one or more memories, and a display screen; the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code. The computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the image reconstruction method according to any one of claims 1-13 above.

15. A readable storage medium for storing computer instructions, which, when running on an electronic device, cause the electronic device to execute the image reconstruction method according to any one of claims 1-13 above.

Images