Image processing method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to an image processing method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the steps that in response to photographing operation, a first image sequence is obtained, and the first image sequence comprises one or more frames of images collected before the photographing operation and one or more frames of images collected after the photographing operation; the brightness of the images in the first image sequence is within a brightness threshold range; acquiring multiple frames of images from the first image sequence for fusion to obtain a representative image; acquiring continuous multi-frame images from the first image sequence based on a photographing moment of the photographing operation to obtain a second image sequence; and generating a dynamic image according to the representative image and the second image sequence. By adopting the method, the condition of brightness change can be reduced, and picture jump of the dynamic image can be avoided.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, electronic device, computer-readable storage medium and computer program product. Background Art

[0002] As the photographic functions of electronic devices are increasingly optimized, shooting needs to be performed in specific shooting environments to meet different shooting needs.

[0003] However, when the camera device of the electronic device generates dynamic images, the images may suddenly become brighter or darker, or even jump. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, device, electronic device, and computer-readable storage medium, which can both reduce brightness changes and avoid screen jumps in dynamic images.

[0005] In a first aspect, the present application provides an image processing method, comprising:

[0006] In response to a photographing operation, a first image sequence is acquired, wherein the first image sequence includes one or more frames of images acquired before the photographing operation and one or more frames of images acquired after the photographing operation; the brightness of the images in the first image sequence is within a brightness threshold range;

[0007] Acquire multiple frames of images from the first image sequence and fuse them to obtain a representative image;

[0008] Acquire a plurality of continuous frames of images from the first image sequence based on the photographing moment of the photographing operation to obtain a second image sequence;

[0009] A dynamic image is generated based on the representative image and the second image sequence.

[0010] In a second aspect, the present application further provides an image processing device, comprising:

[0011] an acquisition module, configured to acquire a first image sequence in response to a photographing operation, wherein the first image sequence includes one or more frames of images acquired before the photographing operation and one or more frames of images acquired after the photographing operation; the brightness of the images in the first image sequence is within a brightness threshold range;

[0012] A fusion module, used for acquiring multiple frames of images from the first image sequence and fusing them to obtain a representative image;

[0013] A selection module, configured to obtain a plurality of continuous frames of images from the first image sequence based on the photographing moment of the photographing operation to obtain a second image sequence;

[0014] A generating module is used to generate a dynamic image according to the representative image and the second image sequence.

[0015] In a third aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the image processing method of the first aspect is implemented.

[0016] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the image processing method of the first aspect.

[0017] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements the image processing method of the first aspect when executed by a processor.

[0018] The above-mentioned image processing method, device, electronic device, computer-readable storage medium and computer program product obtain a first image sequence in response to a photographing operation. Since the first image sequence contains at least two frames of images before and after the photographing operation, the first image sequence can represent information before and after the photographing moment, and the brightness of the images in the first image sequence are all within the brightness threshold range. On the one hand, more images can be screened, so as to ensure that the brightness and other attributes of the representative image are in a better state after obtaining multiple frames of images from the first image sequence and fusing them; on the other hand, because the brightness of the images in the first image sequence are all within the brightness threshold range, continuous multiple frames of images are selected to obtain the second image sequence. After generating a dynamic image based on the representative image and the second image sequence, the dynamic image will not have problems of flickering or picture jumps. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A diagram showing an application environment of an image processing method in an embodiment;

[0021] Figure 2 A schematic diagram of a process of image preview in one embodiment;

[0022] Figure 3 is a schematic diagram of a process of storing a first image in one embodiment;

[0023] Figure 4 is a schematic diagram of a plurality of continuous frames of images before a shooting moment in one embodiment;

[0024] Figure 5 is a schematic diagram of a plurality of consecutive frames of images after a shooting moment in one embodiment;

[0025] Figure 6 is a schematic diagram of a plurality of continuous frames of images before and after a shooting moment in one embodiment;

[0026] Figure 7 A schematic diagram of a specific process of an image processing method in one embodiment;

[0027] Figure 8 A schematic diagram of a preview process in an embodiment;

[0028] Fig. 9 is a schematic diagram of an image shooting process in one embodiment;

[0029] Fig.10 is a structural block diagram of an image processing device in one embodiment;

[0030] Fig.11 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] In one embodiment, Figure 1 As shown, an image processing method is provided. This embodiment uses the method applied to an electronic device as an example. It can be understood that the method can also be applied to a system including an electronic device and a server, and is implemented through the interaction between the electronic device and the server. Among them, the electronic device can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.

[0033] In this embodiment, the image processing method includes the following steps 102 to 108.

[0034] Step 102, in response to a photographing operation, a first image sequence is acquired, the first image sequence including one or more frames of images captured before the photographing operation and one or more frames of images captured after the photographing operation; the brightness of the images in the first image sequence is within a brightness threshold range.

[0035] The photo-taking operation is an operation instruction detected by the electronic device, which is used to instruct the electronic device to capture dynamic images. For example, the photo-taking operation may be a shutter button pressing operation, or a gesture operation detected to indicate taking a photo.

[0036] The first image sequence is at least two frames of images captured by the electronic device during the process of capturing dynamic images. The first image sequence includes images captured before the photo operation and images captured after the photo operation, and the brightness of the images captured before the photo operation and the brightness of the images captured after the photo operation are both within the brightness threshold range.

[0037] The brightness threshold range is a range used to determine whether the brightness of an image is abnormal. An image whose brightness is within the brightness threshold range does not have the problem of overexposure or underexposure; an image whose brightness is outside the brightness threshold range has the problem of overexposure or underexposure. Optionally, in the process of acquiring the first image sequence, if the image captured by the image sensor installed in the electronic device is outside the brightness threshold range, the brightness of the image can be adjusted to be within the brightness threshold range.

[0038] In some embodiments, obtaining a first image sequence includes: before a photographing operation, adjusting the brightness of an image captured by a camera sensor to obtain one or more frames of images captured before the photographing operation; after the photographing operation, adjusting the brightness of an image captured by a camera sensor to obtain one or more frames of images captured before and after the photographing operation.

[0039] In some embodiments, acquiring a first image sequence includes: in response to a shutter button pressing operation, the electronic device groups together images captured before the shutter button pressing operation and an image sequence captured after the shutter button pressing operation, and adjusts the brightness of the grouped image sequence to obtain a first image sequence; wherein the pressing of the shutter button captures a moment near a certain moment in time, rather than just that moment.

[0040] In some embodiments, in response to a photographing operation, a first image sequence is acquired, including: before responding to the photographing operation, updating in real time one or more frames of images captured before the photographing operation based on image data captured by the image sensor, and updating in real time one or more frames of images captured before the photographing operation; in response to the photographing operation, adjusting the brightness of one or more frames of images captured after the photographing operation to obtain one or more frames of images captured after the photographing operation; and combining one or more frames of images captured before the photographing operation and one or more frames of images captured after the photographing operation to obtain a first image sequence.

[0041] Step 104 , acquiring multiple frames of images from the first image sequence and fusing them to obtain a representative image.

[0042] The representative image is an image captured and fused in a still image mode. Since the representative image is obtained by fusion of multiple frames in the first image sequence, the brightness, color and other attributes of the representative image representation content can be adaptively adjusted to adaptively adjust the representative image according to the corresponding still image requirements.

[0043] The multi-frame image is an image to be fused obtained from the first image sequence. The multi-frame image may be all images of the first image sequence, or a preset number of images, or images whose brightness meets corresponding conditions. Optionally, the multi-frame image includes a first image to be fused and a second image to be fused output by the first photoelectric sensor, and a third image to be fused output by the second photoelectric sensor; the brightness of the first image to be fused is greater than that of the second image to be fused, and the brightness of the first image to be fused is greater than that of the third image to be fused; therefore, the first image to be fused can more clearly represent the information of the stationary object, and the first image to be fused can represent the information in the same area through different brightness, so that the representative image corresponds to a higher dynamic range. Since the brightness of the second image to be fused and the third image to be fused are both lower than that of the first image to be fused, the second image to be fused and the third image to be fused are not easily blurred due to the moving area, and there is a corresponding position difference between the second image to be fused and the third image to be fused, more accurate depth information can be obtained, and the phenomenon of false false and missing false can be avoided. Finally, blurring is performed according to the depth information, and the depth information can be accurately represented by the representative image.

[0044] In some embodiments, obtaining multiple frames of images from the first image sequence for fusion to obtain a representative image includes: obtaining multiple frames of images from the first image sequence; determining feature points in each frame of the image to be fused; determining matching feature points representing the same information in different frames of the image to be fused; adjusting the feature points of each frame of the image to be fused according to the feature points and brightness of the adjusted preview image to obtain the adjusted multiple frames of the image to be fused. The feature point can be each pixel, or a pixel point obtained by sampling multiple pixels, or a pixel point selected according to a certain step size.

[0045] Step 106 , acquiring a plurality of continuous frames of images from the first image sequence based on the photographing moment of the photographing operation to obtain a second image sequence.

[0046] The second image sequence is a multi-frame image used to synthesize a dynamic image. Since the second image sequence is extracted from the first image sequence, the images in the second image sequence are all acquired based on images whose brightness is within the brightness threshold range. Optionally, the images in the second image sequence may all be images whose brightness is within the brightness threshold range, or may be adjusted based on images whose brightness is within the brightness threshold range.

[0047] When the second image sequence is composed of time-continuous images, pixels at the same position of the second image sequence change sequentially to avoid pixel jumps at the same position, thereby ensuring that pixel changes in the second image sequence are relatively small.

[0048] Optionally, based on the associated moment associated with the photographing moment of the photographing operation, multiple frames of images acquired at the associated moment may be determined from the first image sequence; and the multiple frames of images acquired at the associated moment may be determined as the second image sequence in time sequence, wherein the time sequence may be a forward sequence or a reverse sequence.

[0049] Step 108: Generate a dynamic image based on the representative image and the second image sequence.

[0050] A dynamic image, also called a live image or livephoto, is a collection of images that supports displaying a representative image and a second image sequence. When no dynamic display operation is detected, the dynamic image displays the representative image, and the fused representative image is equivalent to the cover of the dynamic image; when responding to a dynamic display operation, the dynamic image displays at least part of the second image sequence, and a dynamic display effect or a video playback effect can be formed.

[0051] Optionally, the dynamic display operation is determined based on the signal strength triggered by the electronic device. The signal strength includes but is not limited to the capacitance of the screen, the capacitance change and / or the capacitance change frequency, the level value generated by the processor based on the acquired data, etc. Optionally, the comparison between the signal strength and one or more strength thresholds can be used to determine whether to perform one or more operations, and can also be used to determine whether to perform a dynamic display operation.

[0052] Taking capacitance as an example, a dynamic display operation is caused by a touch event with a capacitance value greater than a preset capacitance threshold, and a static display operation is caused by a touch event with a capacitance value less than the preset capacitance threshold.

[0053] In some embodiments, a dynamic image is generated based on a representative image and a second image sequence, including: using the representative image as a cover image, and using the second image sequence as a display image of the cover image when a dynamic display operation is triggered, thereby, when the cover image triggers a dynamic display operation, the second image sequence can be displayed, so that the dynamic image displays an effect similar to that of a video.

[0054] In this embodiment, in response to a photographing operation, a first image sequence is acquired. Since the first image sequence contains at least two frames of images before and after the photographing operation, the first image sequence can represent information before and after the photographing moment, and the brightness of the images in the first image sequence are all within the brightness threshold range. On the one hand, more images can be screened, so as to ensure that the brightness and other attributes of the representative image are in a better state after acquiring multiple frames of images from the first image sequence and fusing them. On the other hand, since the brightness of the images in the first image sequence are all within the brightness threshold range, multiple consecutive frames of images are selected to obtain the second image sequence. After a dynamic image is generated based on the representative image and the second image sequence, the dynamic image will not have flickering or picture jump problems.

[0055] In some embodiments, the resolution of the images in the first image sequence is greater than the resolution of the photo preview display image.

[0056] The photo preview display image refers to an image that is processed by the image data collected by the electronic device and displayed on the screen of the electronic device at or before the photo is taken.

[0057] The resolution of each frame of the first image sequence is greater than the resolution of the photo preview display image and greater than the second resolution, so the images in the first image sequence contain more information than the photo preview display image. For example, the number of pixels in the images in the first image sequence is greater than the number of pixels in the photo preview display image, so that more detailed information can be accommodated and tiny objects or textures can be displayed.

[0058] The resolution of the photo preview display image is the resolution that the screen supports for displaying images. Since the preview image is mainly used for real-time display, its resolution is relatively low, which helps to improve the data processing efficiency of the screen driver chip to increase the refresh rate of the preview image, so that the user experience is smoother when framing. Optionally, the photo preview display image is displayed on the screen by driving the level change on the screen through the screen driver chip according to the resolution corresponding to the photo preview display image. Optionally, the photo preview display image is generated and displayed based on one or more of the pixel data, color depth, metadata, file format, dynamic range, and compression information in the preview image data.

[0059] Optionally, the resolution of the images in the first image sequence is a first resolution, the resolution of the photo preview display image is a second resolution, and the first resolution is greater than the second resolution.

[0060] In this embodiment, the resolution of each frame of the first image sequence is greater than the resolution of the photo preview display image and greater than the second resolution. Therefore, the images in the first image sequence contain more information than the photo preview display image, thereby ensuring the accuracy of the transmission of information such as brightness, and ensuring that the clarity or other image quality coefficients are high; and image preview is performed according to the resolution of the photo preview display image to adaptively adjust the resolution that the screen can support for preview, thereby completely transmitting information.

[0061] In some embodiments, multiple frames of images are obtained from the first image sequence and fused to obtain a representative image, including: obtaining multiple frames of candidate images from the first image sequence, the multiple frames of candidate images including multiple frames of images captured before the photographing operation; using inter-frame alignment information and brightness adjustment information of the multiple frames of images captured before the photographing operation, fusing the multiple frames of candidate images to obtain a representative image.

[0062] The candidate images include multiple frames of images captured before the photo operation. After being captured by the electronic device, the multiple frames of images captured before the photo operation can also serve as image preview and other functions, so there is a process for calculating inter-frame alignment information and brightness adjustment information.

[0063] The inter-frame alignment information is the result pre-calculated in the inter-frame alignment process of multiple frames of images captured before the photo operation, and the inter-frame alignment information is reused in this fusion process to obtain a representative image more efficiently. Optionally, the inter-frame alignment information can be displacement data of each pixel point or pixel block, through which the pixel distance for adjustment of each candidate image can be determined, so that the pixel points or pixel blocks in different candidate images can represent matching information.

[0064] The brightness adjustment information is the result pre-calculated in the brightness adjustment process of multiple frames of images captured before the photo operation, and the brightness adjustment information is reused in this fusion process to obtain a representative image more efficiently. Optionally, the brightness adjustment information can be a brightness adjustment ratio, so that the brightness in different image data is adaptively adjusted according to the ratio, for example, it is adjusted to a brightness within a brightness threshold range. Optionally, the brightness adjustment information can be a brightness adjustment value for each pixel, so that the brightness in different image data is efficiently adjusted according to the brightness adjustment value.

[0065] Optionally, during the acquisition process of one or more frames of images captured before the photographing operation, at least one of inter-frame alignment information and brightness adjustment information is determined to perform image adjustment, and the adjusted image is used as the one or more frames of images captured before the photographing operation.

[0066] Optionally, during the acquisition of one or more frames of images collected before the photo-taking operation, there is a process of calculating inter-frame alignment information to achieve inter-frame alignment of corresponding images based on the inter-frame alignment information; correspondingly, since the inter-frame alignment information has been calculated in the process of acquiring the first image sequence, when acquiring multiple frames of images from the first image sequence for fusion, the inter-frame alignment information can be reused to ensure processing efficiency.

[0067] Optionally, since the brightness of the images in the first image sequence is within the brightness threshold range, and the brightness of the images captured by the sensor is difficult to control, the images in the first image sequence have their own brightness adjustment process. Correspondingly, in the process of acquiring the first image sequence, the inter-frame alignment information and the brightness adjustment information are acquired and used to adjust one or more frames of images captured before the photographing operation, so that the one or more frames of images captured before the photographing operation are within the brightness threshold range.

[0068] Optionally, in the process of acquiring one or more frames of images collected before the photo operation, there is a process of calculating brightness adjustment information to achieve brightness adjustment of the corresponding image based on the brightness adjustment information; correspondingly, since the brightness adjustment information has been calculated in the process of acquiring the first image sequence, when acquiring multiple frames of images from the first image sequence for fusion, the brightness adjustment information can be reused to ensure processing efficiency.

[0069] Optionally, the inter-frame alignment information and brightness adjustment information collected before the photographing operation may be determined by adjusting the first image with reference to the second image. For example, the pixel position of the second image may be used as a reference to adjust the pixel position representing the same information in the first image, and the result information used in the adjustment process may be obtained as the inter-frame alignment information; at the same time, the brightness of the first image at the corresponding position may be adjusted with reference to the brightness of the second image, and the result information used in the adjustment process may be obtained as the brightness adjustment information.

[0070] Optionally, the brightness adjustment information collected before the photographing operation is determined by adjusting the first image with reference to the brightness of the second image; and the inter-frame alignment information collected before the photographing operation is determined by adjusting the second image with reference to the pixel position of the first image; for example, the pixel position of the first image can be used as a reference to adjust the pixel position representing the same information in the second image, and the information used in this adjustment process is obtained as the inter-frame alignment information; at the same time, the brightness of the first image at the corresponding position is adjusted with reference to the brightness of the second image, and the information used in this adjustment process is obtained as the brightness adjustment information.

[0071] In some embodiments, using the inter-frame alignment information and brightness adjustment information of multiple frames of images collected before the photo operation, multiple frames of candidate images are fused to obtain a representative image, including: determining feature points in adjacent frame images; determining matching feature points representing the same information in adjacent frame images; adjusting the feature points of adjacent frame images according to the feature points and brightness of the reference frame image to obtain adjusted adjacent frame images; the reference frame image is the frame image with the highest clarity among the adjacent frame images; based on the pixel correspondence relationship, the pixel values ​​contained in the adjusted adjacent frame images are fused to obtain a representative image. Among them, the feature point can be each pixel, or a pixel point obtained by sampling multiple pixels, or a pixel point selected according to a certain step size; and the adjacent frame images are at least two adjacent frame images at the time of acquisition.

[0072] In some embodiments, multiple frames of candidate images are fused using inter-frame alignment information and brightness adjustment information of multiple frames of images collected before the photographing operation to obtain a representative image, including: performing image fusion inter-frame alignment and brightness adjustment on the multiple frames of images collected before the photographing operation according to the inter-frame alignment information and brightness adjustment information of the multiple frames of images collected before the photographing operation.

[0073] In some embodiments, using the inter-frame alignment information and brightness adjustment information of the multiple frames of images collected before the photographing operation, the multiple frames of candidate images are fused to obtain a representative image, including: performing image fusion inter-frame alignment and brightness adjustment on the multiple frames of images collected before the photographing operation according to the inter-frame alignment information and brightness adjustment information of the multiple frames of images collected before the photographing operation. Optionally, the inter-frame alignment information and brightness adjustment information of the multiple frames of images may not be changed, may be adjusted in data content, or may be adjusted in format.

[0074] Optionally, the candidate image may also include multiple frames of images captured after the photographing operation, but in the process of fusing the multiple frames of candidate images based on the multiple frames of images captured after the photographing operation, there is also a process of calculating inter-frame alignment information and brightness adjustment information to obtain a representative image.

[0075] In this embodiment, in view of the situation that the multiple-frame candidate images include multiple-frame images captured before the photographing operation, the inter-frame alignment information and brightness adjustment information of the multiple-frame images captured before the photographing operation are used to perform inter-frame alignment and brightness adjustment. Since these inter-frame alignment information and brightness adjustment information may exist in advance, they can have higher processing efficiency.

[0076] In some embodiments, Figure 2 As shown, the method also includes:

[0077] Step 202: In response to the photo preview, store a first image with a first resolution.

[0078] Photo preview is a state in which an image is captured by a sensor of an electronic device and displayed based on the captured image. Optionally, in the case of photo preview, the electronic device captures an image to be processed at a first resolution through its own sensor, converts the image to be processed into a first image at a first resolution, and then stores the image at the first resolution. Optionally, in the case of photo preview, the electronic device captures a first image at a first resolution through its own sensor and stores the first image at the first resolution.

[0079] The first image of the first resolution is an image stored in the electronic device at the first resolution. Optionally, the first image of the first resolution can be stored in a cache, or in a virtual memory or flash memory.

[0080] Optionally, the first image of the first resolution may be one or more frames of images captured before the photographing operation, or may be acquired based on one or more frames of images captured before the photographing operation. That is, the first image of the first resolution belongs to a type of image in the first image sequence, or may be an image used to obtain the first image sequence. Since the first resolution is greater than the second resolution, the first image of the first resolution contains more information, so that the first image in the first image sequence has more information. For example, the number of pixels in the first image of the first resolution is greater than the number of pixels in the first image of the second resolution, so that more detail information can be accommodated, and tiny objects or textures can be displayed.

[0081] Step 204, downsampling and cropping the first image of the first resolution to obtain the first image of the second resolution; the first resolution is greater than the second resolution.

[0082] The second resolution is the resolution that the screen supports real-time display of images. In the case of real-time display, the second resolution is relatively low compared to the first image resolution, which helps to improve the data processing efficiency of the screen driver chip, so as to increase the refresh rate of the preview image and make the user experience smoother when framing.

[0083] Optionally, the image of the first resolution is a large-size image FullSize, and the size of the display image size image Preview is smaller than the large-size image. For example, the first resolution may be 4096*3072, and correspondingly, the second resolution may be 1440*1080.

[0084] In some embodiments, downsampling and cropping a first image of a first resolution to obtain a first image of a second resolution includes: downsampling the first image of the first resolution to obtain a downsampled first image; the resolution of the downsampled first image is between the first resolution and the second resolution; and cropping the downsampled first image according to the second resolution to obtain a first image of the second resolution. For example: downsampling a 4096*3072 image at a sampling rate of 2 to obtain a 2048*1536 image, and then cropping it to obtain a 1440*1080 image. Thus, by controlling the degree of downsampling not to be too high, ensuring less loss of details, and then using cropping, the downsampled first image can be adapted to a screen that supports the second resolution to ensure universality.

[0085] Step 206: display the first image at the second resolution.

[0086] In some embodiments, displaying a first image with a second resolution includes: controlling a screen driving chip to drive a level change on the screen according to the second resolution to display the first image with the second resolution on the screen.

[0087] In some embodiments, displaying a first image of a second resolution includes: generating and displaying the first image of the second resolution based on one or more of pixel data, color depth, metadata, file format, dynamic range, and compression information corresponding to the first image of the second resolution.

[0088] Optionally, the first image of the second resolution and the images in the first image sequence all have their own inter-frame alignment information and brightness adjustment information. The difference between the two is that in the preview stage, the inter-frame alignment information and brightness adjustment information of the first image of the second resolution have been determined; in the photographing stage, the inter-frame alignment information and brightness adjustment information of the images in the first image sequence are determined; therefore, for one or more frames of images captured before the photographing operation contained in the first image sequence, the inter-frame alignment information and brightness adjustment information of the multiple frames of images captured before the photographing operation are used to fuse the multiple frames of candidate images to obtain a representative image to ensure processing efficiency.

[0089] In this embodiment, the first resolution is greater than the second resolution, and thus the first image of the first resolution contains more information than the first image of the second resolution. In this case, storing the first image of the first resolution can ensure the accuracy of information transmission such as frame alignment and brightness, and can ensure high clarity or other image quality coefficients; and performing image preview based on the first image of the second resolution can adaptively adjust the resolution supported by the screen for preview, thereby completely transmitting information.

[0090] In some embodiments, Figure 3 As shown, in response to the photo preview, storing a first image with a first resolution includes:

[0091] Step 302: In response to the photo preview, a first image with a first resolution is acquired.

[0092] In some embodiments, the process of acquiring the first image of the first resolution involves the interaction between the camera application (Application, APP) and the hardware abstraction layer (HAL), and the interaction process involves the transmission and processing of a preview request, which includes stream attributes such as brightness (EV value), resolution and frame rate, and the spatial address of the image data, and the spatial address includes the buffer location; when processing the preview request, the hardware abstraction layer combines 3A and the driver with the image signal processor (Image Signal Processing, ISP) to generate the first image of the first resolution in YUV or other format; wherein 3A refers to automatic exposure control (Auto Exposure, AE), automatic focus control (Auto Focus, AF) and automatic white balance control (Auto White Balance, AWB).

[0093] Correspondingly, obtaining a first image with a first resolution includes: sending a preview request to a hardware abstraction layer through a camera application; the hardware abstraction layer generates and returns a first image with a first resolution in a corresponding format in combination with 3A and a driver in cooperation with an image signal processor according to the preview request sent by the terminal to the camera application, and forwards the first image with the first resolution to a cached data display area through the camera application, so that the electronic device obtains the first image with the first resolution.

[0094] Optionally, after receiving the preview request, the hardware abstraction layer configures the corresponding hardware settings according to the provided parameters, such as setting the correct exposure value, stream properties, etc. The preview request adjusts the autofocus, auto exposure, and auto white balance through interaction with the 3A module to ensure the best image quality. The data adjusted by 3A is sent to the image signal processor, which further processes the image data, including denoising, color correction, etc., and finally generates a first image of the first resolution in YUV format or other formats. Finally, the processed first image will be returned to the hardware abstraction layer, and the hardware abstraction layer will pass it to the application of the electronic device as the processed first image, thus laying the foundation for displaying the real-time image on the screen.

[0095] Step 304, when it is determined that the brightness of the first image exceeds the brightness threshold range, the brightness of the first image is adjusted based on the brightness of the second image to obtain the first image after the brightness adjustment, and the first image of the first resolution after the brightness adjustment is stored; wherein the second image is the previous frame image adjacent to the first image stored in the preview stage; and the brightness of the second image is within the brightness threshold range.

[0096] The second image is an image whose brightness is within the brightness threshold range, and thus the brightness of the second image is within the appropriate range, that is, it is not overexposed or underexposed. In the case where the second image is the previous frame image adjacent to the first image stored in the preview stage, the second image is acquired before the first image, that is, the acquisition time of the second image is earlier than the acquisition time of the first image, and thus, the brightness of the second image can be adjusted to be within the brightness threshold range, so that the preview is not just one frame, but a real-time storage preview process; optionally, one frame image whose brightness is within the brightness threshold range can be determined as the second image among multiple frames of images of the first resolution.

[0097] Optionally, if an image whose brightness is within a brightness threshold range is detected in the image collected by the sensor, the image whose brightness is within the brightness threshold range is used as the second image. Optionally, the second image can be determined based on a previous frame image of the first image in the buffer area.

[0098] In some embodiments, adjusting the brightness of the first image based on the brightness of the second image to obtain the brightness-adjusted first image includes: based on the brightness of the second image, performing multi-frame fusion on the first image and the second image to obtain the brightness-adjusted first image having the same brightness as the second image. Optionally, the multi-frame fusion process also involves processes such as inter-frame alignment, so as to facilitate the reuse of relevant information in subsequent shooting processes.

[0099] In some embodiments, the brightness of the first image is adjusted based on the brightness of the second image to obtain the first image with adjusted brightness, including: adjusting the brightness of the first image based on the brightness of the second image until the brightness of the first image and the brightness of the second image meet a matching condition; wherein meeting the matching condition includes but is not limited to the brightness of the first image being the same as the brightness of the second image or being within a preset range.

[0100] In some embodiments, the sequentially captured image data are used as the second image and the current first image of the first resolution; correspondingly, the brightness of the first image is adjusted based on the brightness of the second image to obtain the brightness-adjusted first image, including: sequentially adjusting the brightness of the current first image based on the brightness of the second image to obtain the brightness-adjusted first image; the brightness-adjusted first image is used as the second image corresponding to the next frame of the first image, and the next frame of the first image is at least one frame of image captured after the current first image.

[0101] Optionally, because in most shooting scenarios, the user will estimate whether the shooting environment is normal, when the user starts shooting, the brightness in the shooting environment is assumed to be within the brightness threshold range to ensure processing efficiency; and the default first captured image data is within the brightness threshold range, so the initial second image is within the brightness threshold range, and the subsequently captured image data is the initial first image; next, when the next first image is captured again, the initial first image after brightness adjustment is used as the second image of the next first image relative to the next first image, so that each preview image data is processed in sequence.

[0102] When the brightness of the first image exceeds the brightness threshold range, since the first image of the first resolution is adjusted based on the second image, the first image of the first resolution is not easy to be overexposed or too dark, and since the brightness is adjusted based on the first image, the information required to represent the first image can be retained, and there may be no or less frame loss, and it is not easy to have a picture or brightness jump, thereby ensuring a better user experience.

[0103] In some embodiments, the brightness of the first image exceeds the brightness threshold range, which means that the brightness of the first image is greater than a preset value of image overexposure;

[0104] In some embodiments, the brightness of the first image exceeding the brightness threshold range means that the brightness of the first image is less than a preset value of image underexposure.

[0105] Step 306: When it is determined that the brightness of the first image is within the brightness threshold range, directly store the first image of the first resolution.

[0106] Optionally, when the brightness of the first image or the brightness of the first image after brightness adjustment is within a brightness threshold range, the first image of the first resolution can be stored in a cache display data area, so that in response to a photo-taking operation, a first image sequence can be obtained from the cache display data area.

[0107] Optionally, when it is determined that the brightness of the first image is within the brightness threshold range, it does not involve brightness adjustment of the first image of the first resolution, nor does it involve adjustment of data such as variable names and variable categories, and it may only involve the process of data storage.

[0108] In this embodiment, the brightness of the second image is within the brightness threshold range, so that the second image can serve as the brightness reference frame of the first image, laying the foundation for brightness adjustment; on this basis, based on the brightness of the second image, the brightness of the first image is adjusted, so that the first image can approach the brightness threshold range, so that the brightness of the first image after brightness adjustment is not easy to be overexposed or underexposed; therefore, when previewing the image, it is not easy to become brighter or darker, and the preview screen can display the information carried by the first image to avoid screen jump. At the same time, the brightness of the first image is adjusted only when the brightness of the first image exceeds the brightness threshold range, which can reduce the number of first images that require brightness conditions, thereby ensuring relatively high processing efficiency. In addition, since the first image of the first resolution has brightness adjustment, the brightness adjustment information can be reused in the fusion process of the representative image, and the inter-frame alignment information can also be reused as required, so that the multi-frame fusion algorithm is less time-consuming and can obtain the representative image more efficiently. Moreover, the second image sequence is determined by using the image of the first resolution, which can make the resolution of the dynamic image relatively high to ensure the corresponding clarity is relatively high.

[0109] In some embodiments, the method further includes: storing the first image with a first resolution in a first area of ​​the cache display data area; and storing the first image with a second resolution in a second area of ​​the cache display data area.

[0110] Correspondingly, acquiring the first image sequence includes: acquiring the first image sequence from a first area of ​​the cache display data area.

[0111] The cached data display area is a cache area for image data, and the data in the cached data display area can be called under different conditions. Among them, the first image of the second resolution is called during image preview to generate a preview image on the screen; in response to a photo operation, the first image sequence is obtained from the first area of ​​the cached display data area, so that the inter-frame alignment information and brightness adjustment information in the preview process are multiplexed into the inter-frame alignment and brightness adjustment of the photo operation.

[0112] The first area and the second area are different cache areas, and are respectively used to store images of different resolutions. Optionally, the first area is used to store image data of a first resolution, and the image data of the first resolution at least includes a first image of the first resolution, and may also include a first image and a second image after brightness adjustment; the second area is used to store an image of a second resolution, such as the first image of the second resolution, for real-time display.

[0113] Optionally, since the first area and the second area are different buffer areas, data interference can be avoided when performing inter-frame alignment and brightness adjustment, and the corresponding representative image can be more efficiently fused. In addition, the data in different buffer areas can use the same identifier to determine the corresponding relationship between the two.

[0114] In some embodiments, obtaining the first image sequence from the first area of ​​the cache display data area includes: selecting first images of the first resolution according to a preset number from the first area of ​​the cache display data area, and arranging the selected images in chronological order to obtain the first image sequence. For example: selecting 10 first images of the first resolution from the first area of ​​the cache display data area, and arranging the 10 images in chronological order from the earliest to the last, to obtain the first image sequence.

[0115] In some embodiments, obtaining the first image sequence from the first area of ​​the cache display data area includes: after deleting the first acquired first image from the first image sequence, selecting the first image of the first resolution from the first area of ​​the cache display data area in real time, and using the selected image as the last acquired image in the first image sequence to achieve directional update. For example: after deleting the first acquired 1 frame of image from the first image sequence, selecting 1 first image of the first resolution from the first area of ​​the cache display data area, and storing the 1 image in the first image sequence.

[0116] In this embodiment, images with a first resolution and an image with a second resolution are stored in different partitions of the cache data display area, respectively, to avoid data interference between images with different resolutions and to facilitate the separate calling of data or information corresponding to the two images.

[0117] In some embodiments, acquiring multiple continuous image frames from the first image sequence based on the photographing moment of the photographing operation to obtain the second image sequence includes: acquiring multiple continuous image frames before the photographing operation from the first image sequence to obtain the second image sequence.

[0118] like Figure 4 As shown, the continuous multiple frame images before the shooting moment form a second image sequence, so that the second image sequence represents the information before the shooting moment, for example, multiple frame images before shutter.

[0119] In some embodiments, obtaining a plurality of consecutive frames of images before a photographing operation from a first image sequence to obtain a second image sequence includes: performing frame alignment and brightness adjustment according to frame alignment information and brightness adjustment information of the consecutive frames of images before the photographing operation during a preview process to obtain the second image sequence. Thus, the frame alignment information and the brightness adjustment information are multiplexed to improve processing efficiency.

[0120] In this embodiment, the dynamic image can reflect the information before the photo is taken. In addition, the continuous multiple frames of images before the photo operation are obtained from the first image sequence to obtain the second image sequence. Therefore, in this process, the information of the continuous multiple frames of images before the photo operation can be reused during the generation process.

[0121] In some embodiments, acquiring multiple continuous image frames from the first image sequence based on the photographing moment of the photographing operation to obtain the second image sequence includes: acquiring multiple continuous image frames after the photographing operation from the first image sequence to obtain the second image sequence.

[0122] like Figure 5 As shown, the second image sequence is a plurality of image frames after the capturing moment, that is, a plurality of image frames after the shutter.

[0123] In this embodiment, the second image sequence enables the dynamic image to reflect information after the photo-taking moment.

[0124] In some embodiments, multiple frames of continuous images are acquired from a first image sequence based on the photographing moment of a photographing operation to obtain a second image sequence, including: acquiring at least one frame of image before the photographing operation and at least one frame of image after the photographing operation from the first image sequence to obtain the second image sequence, wherein the images in the second image sequence are continuously captured images.

[0125] like Figure 6 As shown, there are multiple frames of images before and after the shooting time, for example: images shot before and after the shutter.

[0126] In this embodiment, the dynamic image can reflect the information before the photo is taken, and the dynamic image can reflect the information after the photo is taken. In addition, the continuous multiple frames of images before the photo operation are obtained from the first image sequence to obtain the second image sequence, so that in this process, the information of the continuous multiple frames of images before the photo operation can be reused during the generation process.

[0127] In an exemplary embodiment, Figure 7 As shown, the method specifically includes:

[0128] Step 701, acquiring a first image.

[0129] Step 702 , detecting whether the brightness of the first image is within a brightness threshold range, if so, executing step 703 , if not, executing step 704 .

[0130] Step 703, stores the first image in the first area of ​​the cache data display area, downsamples and crops the first image to obtain a first image of a second resolution, stores the first image of the second resolution in the second area of ​​the cache data display area, and then executes step 706.

[0131] Step 704, when the brightness of the first image exceeds the brightness threshold range, obtain the second image from the first area of ​​the cache data display area, adjust the brightness of the first image according to the brightness of the second image, and obtain the first image after brightness adjustment; downsample and crop the first image after brightness adjustment to obtain a first image of a second resolution; the second resolution is smaller than the first resolution, the brightness of the second image is within the brightness threshold range, and the second image is an adjacent previous frame image of the first image.

[0132] Step 705 , storing the first image after brightness adjustment in the first area of ​​the cache data display area; storing the first image with the second resolution in the second area of ​​the cache data display area, and then executing step 706 .

[0133] Step 706 , displaying the first image of the second resolution in the second area to implement image preview. If no photo-taking operation is detected, returning to step 701 .

[0134] Step 707, in response to a photographing operation, obtaining a first image sequence from a first area of ​​a cache display data area; the first image sequence includes one or more frames of images captured before the photographing operation, and one or more frames of images captured after the photographing operation; the brightness of the images in the first image sequence is within a brightness threshold range.

[0135] Step 708, obtaining multiple frames of candidate images from the first image sequence, the multiple frames of candidate images including multiple frames of images captured before the photographing operation; using the inter-frame alignment information and brightness adjustment information of the multiple frames of images captured before the photographing operation, the multiple frames of candidate images are fused to obtain a representative image.

[0136] Step 709: acquiring a plurality of continuous image frames from the first image sequence based on the photographing time of the photographing operation to obtain a second image sequence, wherein the second image sequence includes a plurality of continuous image frames before the photographing operation; or a plurality of continuous image frames after the photographing operation; or at least one image frame before the photographing operation and at least one image frame after the photographing operation.

[0137] Step 710: Generate a dynamic image based on the representative image and the second image sequence.

[0138] Therefore, the software process of LivePhoto is optimized. In the preview stage, a lightweight HDR algorithm or other method is used between the frames of the first preview image data and the second preview image data to adjust the brightness, so that overexposed or underexposed images can also be displayed normally on the preview screen, reducing the jump of multiple frame previews. In the photo stage, the inter-frame algorithm of the preview stage is reused to process the alignment information and brightness fusion information, reducing the amount of calculation of the photo algorithm and the time consumption of the algorithm processing. Moreover, in the live video fusion stage, because the preview display brightness is optimized, the picture jump caused by overexposed or underexposed scenes is solved, and the frame interval is prevented from being too long, exceeding 33ms, so that there will be no jump between the pictures before and after the photo is taken, ensuring that the live video is smoother and less prone to jamming. Moreover, the resolution of the first resolution image data for livephoto video fusion is higher than that of the preview display data, and because the first resolution is greater than the second resolution, the number of pixels is large; the file size is usually large; it is suitable for situations where detailed image details need to be viewed.

[0139] In an exemplary embodiment, Figure 8 As shown, based on the execution process of step 801 to step 806, the image preview process is described.

[0140] Wherein, step 701 includes:

[0141] 1. Send a preview request to the hardware abstraction layer through the camera application;

[0142] 2. The hardware abstraction layer generates and sends a first image with a corresponding format and a first resolution to the camera application in combination with 3A and the driver in accordance with the preview request sent by the terminal and with the image signal processor.

[0143] Then, step 703 includes:

[0144] 3. Forward the first image of the first resolution to the first area of ​​the cache data display area through the camera application to obtain the first image of the first resolution. Downsample and crop the first image of the first resolution to obtain the first image of the second resolution; the second resolution is smaller than the first resolution.

[0145] Step 704 specifically includes:

[0146] 4. Input the second image and the first image of the first resolution into the preview algorithm back-end processing module; the processing of the preview algorithm back-end processing module includes adjusting the brightness of the first image of the first resolution according to the brightness of the second image through a neural network model (such as AIgoEng) to obtain the first image after brightness adjustment; wherein the second image is a reference frame (RefFrame) for adjusting brightness, and the first image is a base frame (BaseFrame).

[0147] Step 705 specifically includes:

[0148] 5. When the preview algorithm back-end processing module outputs the first image after brightness adjustment, the first image after brightness adjustment is stored in the first area of ​​the cache data display area, such as the Fullsize partition, for easy reuse; the first image with the second resolution is stored in the second area of ​​the cache data display area, such as the Preview partition, for preview.

[0149] Step 706 specifically includes:

[0150] 6. Acquire the first image with the second resolution from the second area, and send the first image with the second resolution to the camera application for image preview; if no photo taking operation is detected, continue to acquire a new first image.

[0151] In an exemplary embodiment, Fig. 9 As shown, based on the execution process of step 707 to step 810, the image capturing process is described in detail.

[0152] Wherein, step 707 includes:

[0153] 1. Send a shooting request to the hardware abstraction layer through the camera application; the shooting request is triggered by the photo operation;

[0154] 2. The hardware abstraction layer generates and returns the first image sequence FN of the corresponding format to the camera application in combination with 3A and the driver in cooperation with the image signal processor according to the shooting request sent by the terminal;

[0155] 3. The first image sequence is forwarded to the cache data display area through the camera application to obtain multiple frame images F1, F2 and F3 in the first image sequence.

[0156] Step 708 includes:

[0157] 4. Determine the inter-frame alignment information and brightness adjustment information of the multiple-frame images F1, F2 and F3 in the first image sequence through a neural network model (such as AIgoEng) according to the inter-frame alignment information and brightness adjustment information of the first image after brightness adjustment in the first area (i.e., the Fullsize partition); align and brightness adjust the multiple-frame images F1, F2 and F3 according to the inter-frame alignment information and brightness adjustment information of the multiple-frame images F1, F2 and F3 in the first image sequence to obtain the adjusted multiple-frame display image data.

[0158] Step 708 includes:

[0159] 5. Through a neural network model (such as AIgoEng), the adjusted multi-frame display image data is fused to obtain a representative image.

[0160] Step 709 and step 710 include:

[0161] 6. Obtain a second image sequence before and after the shooting moment; determine the non-representative dynamic image associated with the representative image based on the multi-frame image data in the second image sequence, and store the representative image and its associated non-representative dynamic image in video form to obtain a dynamic image.

[0162] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0163] Based on the same inventive concept, the embodiment of the present application also provides an image processing device for implementing the above-mentioned image processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image processing device embodiments provided below can refer to the limitations on the image processing method above, and will not be repeated here.

[0164] In an exemplary embodiment, Fig.10 As shown, an image processing device is provided, comprising:

[0165] The acquisition module 1002 is configured to acquire a first image sequence in response to a photographing operation, wherein the first image sequence includes one or more frames of images acquired before the photographing operation and one or more frames of images acquired after the photographing operation; the brightness of the images in the first image sequence is within a brightness threshold range;

[0166] A fusion module 1004 is used to obtain multiple frames of images from the first image sequence and fuse them to obtain a representative image;

[0167] A selection module 1006 is used to obtain a plurality of continuous frames of images from the first image sequence based on the photographing moment of the photographing operation to obtain a second image sequence;

[0168] The generating module 1008 is configured to generate a dynamic image according to the representative image and the second image sequence.

[0169] In one of the embodiments, the resolution of the images in the first image sequence is greater than the resolution of the photo preview display image.

[0170] In one embodiment, the fusion module 1004 is used to:

[0171] Acquire multiple frames of candidate images from the first image sequence, where the multiple frames of candidate images include multiple frames of images acquired before the photographing operation;

[0172] The multiple frames of candidate images are fused using the inter-frame alignment information and brightness adjustment information of the multiple frames of images collected before the photographing operation to obtain a representative image.

[0173] In one embodiment, the acquisition module 1002 is used to:

[0174] In response to the photo preview, storing a first image with a first resolution;

[0175] Downsampling and cropping the first image with the first resolution to obtain a first image with a second resolution; the first resolution is greater than the second resolution;

[0176] The first image at the second resolution is displayed.

[0177] In one embodiment, the acquisition module 1002 is used to:

[0178] In response to the photo preview, acquiring a first image with a first resolution;

[0179] When it is determined that the brightness of the first image exceeds a brightness threshold range, the brightness of the first image is adjusted based on the brightness of the second image to obtain a brightness-adjusted first image, and the brightness-adjusted first image of the first resolution is stored; wherein the second image is a previous frame image adjacent to the first image stored in the preview stage; and the brightness of the second image is within the brightness threshold range;

[0180] In a case where it is determined that the brightness of the first image is within a brightness threshold range, the first image with the first resolution is directly stored.

[0181] In one embodiment, the acquisition module 1002 is used to:

[0182] Storing a first image with a first resolution in a first area of ​​the cache display data area;

[0183] storing a first image with a second resolution in a second area of ​​the cache display data area;

[0184] The acquiring the first image sequence includes: acquiring the first image sequence from a first area of ​​the cache display data area.

[0185] In one embodiment, the selection module 1006 is used to:

[0186] A plurality of continuous frames of images before the photographing operation are acquired from the first image sequence to obtain a second image sequence.

[0187] In one embodiment, the selection module 1006 is used to:

[0188] A plurality of continuous frames of images after the photographing operation are acquired from the first image sequence to obtain a second image sequence.

[0189] In one embodiment, the selection module 1006 is used to:

[0190] At least one frame of image before the photographing operation and at least one frame of image after the photographing operation are acquired from the first image sequence to obtain a second image sequence, wherein the images in the second image sequence are continuously acquired images.

[0191] Each module in the above image processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in an electronic device in the form of hardware, or can be stored in a memory in an electronic device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0192] In an exemplary embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.11As shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, an image processing method is implemented. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the electronic device casing, or an external keyboard, touchpad or mouse.

[0193] Those skilled in the art will understand that Fig.11 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0194] In one embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0195] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0196] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0197] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0198] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0199] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0200] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: The method comprises: In response to a photographing operation, a first image sequence is acquired, wherein the first image sequence includes one or more frames of images acquired before the photographing operation and one or more frames of images acquired after the photographing operation; the brightness of the images in the first image sequence is within a brightness threshold range; Acquire multiple frames of images from the first image sequence and fuse them to obtain a representative image; Acquire a plurality of continuous frames of images from the first image sequence based on the photographing moment of the photographing operation to obtain a second image sequence; A dynamic image is generated based on the representative image and the second image sequence.

2. The method according to claim 1, characterized in that The resolution of the images in the first image sequence is greater than the resolution of the photo preview display image.

3. The method according to claim 1, characterized in that The acquiring multiple frames of images from the first image sequence and fusing them to obtain a representative image includes: Acquire multiple frames of candidate images from the first image sequence, where the multiple frames of candidate images include multiple frames of images acquired before the photographing operation; The multiple frames of candidate images are fused using the inter-frame alignment information and brightness adjustment information of the multiple frames of images collected before the photographing operation to obtain a representative image.

4. The method according to claim 1, characterized in that: The method further comprises: In response to the photo preview, storing a first image with a first resolution; Downsampling and cropping the first image with the first resolution to obtain a first image with a second resolution; the first resolution is greater than the second resolution; The first image at the second resolution is displayed.

5. The method according to claim 4, characterized in that The storing of the first image with the first resolution in response to the photo preview comprises: In response to the photo preview, acquiring a first image with a first resolution; When it is determined that the brightness of the first image exceeds a brightness threshold range, the brightness of the first image is adjusted based on the brightness of the second image to obtain a brightness-adjusted first image, and the brightness-adjusted first image of the first resolution is stored; wherein the second image is a previous frame image adjacent to the first image stored in the preview stage; and the brightness of the second image is within the brightness threshold range; In a case where it is determined that the brightness of the first image is within a brightness threshold range, the first image with the first resolution is directly stored.

6. The method according to claim 4, characterized in that The method further comprises: Storing a first image with a first resolution in a first area of ​​the cache display data area; storing a first image with a second resolution in a second area of ​​the cache display data area; The acquiring the first image sequence includes: acquiring the first image sequence from a first area of ​​the cache display data area.

7. The method according to claim 1, characterized in that The acquiring a plurality of continuous frames of images from the first image sequence based on the photographing moment of the photographing operation to obtain a second image sequence includes: A plurality of continuous frames of images before the photographing operation are acquired from the first image sequence to obtain a second image sequence.

8. The method according to claim 1, characterized in that The acquiring a plurality of continuous frames of images from the first image sequence based on the photographing moment of the photographing operation to obtain a second image sequence includes: A plurality of continuous frames of images after the photographing operation are acquired from the first image sequence to obtain a second image sequence.

9. The method according to claim 1, characterized in that: The acquiring a plurality of continuous frames of images from the first image sequence based on the photographing moment of the photographing operation to obtain a second image sequence includes: At least one frame of image before the photographing operation and at least one frame of image after the photographing operation are acquired from the first image sequence to obtain a second image sequence, wherein the images in the second image sequence are continuously acquired images.

10. An image processing device, characterized in that: The device comprises: an acquisition module, configured to acquire a first image sequence in response to a photographing operation, wherein the first image sequence includes one or more frames of images acquired before the photographing operation and one or more frames of images acquired after the photographing operation; the brightness of the images in the first image sequence is within a brightness threshold range; A fusion module, used for acquiring multiple frames of images from the first image sequence and fusing them to obtain a representative image; A selection module, configured to obtain a plurality of continuous frames of images from the first image sequence based on the photographing moment of the photographing operation to obtain a second image sequence; A generating module is used to generate a dynamic image according to the representative image and the second image sequence.

11. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images