Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides an image processing method, device, electronic equipment and storage medium, the method comprising: determining a feature image corresponding to a target spectral range in a first image; preprocessing a second image to obtain a preprocessed image; wherein the second image and the first image are images corresponding to the same scene; the first image has more spectral channels than the second image; and fusing the preprocessed image and the feature image to obtain a target image. The technical solution of the present disclosure can improve the natural effect of image beautification.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of imaging technology, beautification functions are increasingly widely used in areas such as selfies and video calls on electronic devices. Beautification functions include various dimensions such as skin smoothing, whitening, rosy complexion, and eye brightening. Skin smoothing plays a crucial role in the entire beautification algorithm process. It filters out minor imperfections in the image, making color transitions smoother, thus making imperfections less noticeable and creating a natural beautification effect. Summary of the Invention

[0003] This disclosure provides an image processing method, apparatus, electronic device, and storage medium.

[0004] According to a first aspect of this disclosure, an image processing method is provided, the method comprising:

[0005] Determine the feature image corresponding to the target spectral band in the first image;

[0006] The second image is preprocessed to obtain a preprocessed image; wherein the second image and the first image are images corresponding to the same scene; the first image has more spectral channels than the second image;

[0007] The target image is obtained by fusing the preprocessed image and the feature image.

[0008] In some embodiments, determining the feature image corresponding to the target spectral band in the first image includes:

[0009] Based on a preset operation performed on the electronic device, the target feature in the first image to which the preset operation is directed is determined;

[0010] Based on the target feature, the spectral band in which the target feature is located is determined as the target spectral band; wherein, the image indicated by the target spectral band is the feature image.

[0011] In some embodiments, the method further includes:

[0012] Feature extraction is performed on the first image to obtain preset features;

[0013] Determining the feature image corresponding to the target spectral band in the first image includes:

[0014] The target feature is determined from the preset features.

[0015] In some embodiments, fusing the preprocessed image and the feature image to obtain the target image includes:

[0016] Align the preprocessed image and the feature image to obtain an aligned preprocessed image and an aligned feature image;

[0017] The aligned feature image is mapped onto the aligned preprocessed image to obtain the target image.

[0018] In some embodiments, the method further includes:

[0019] The feature image is blurred.

[0020] The process of fusing the preprocessed image and the feature image to obtain the target image includes:

[0021] The target image is obtained by fusing the preprocessed image and the feature image after blurring.

[0022] In some embodiments, the field of view of the first image acquisition module that acquires the first image is greater than or equal to the field of view of the second image acquisition module that acquires the second image.

[0023] In some embodiments, the preprocessing includes at least one of the following:

[0024] Filtering algorithm processing, filter effect algorithm processing, sharpening algorithm processing.

[0025] According to a second aspect of this disclosure, an image processing apparatus is provided, characterized in that the apparatus comprises:

[0026] The determination module is used to determine the feature image corresponding to the target spectral band in the first image;

[0027] The preprocessing module is used to preprocess the target region in the second image to obtain a preprocessed image; wherein the second image and the first image are images corresponding to the same scene; the first image has more spectral channels than the second image;

[0028] The fusion module is used to fuse the preprocessed image and the feature image to obtain the target image.

[0029] In some embodiments, the determining module is configured to:

[0030] Based on a preset operation performed on the electronic device, the target feature in the first image to which the preset operation is directed is determined;

[0031] Based on the target feature, the spectral band in which the target feature is located is determined as the target spectral band; wherein, the image indicated by the target spectral band is the feature image.

[0032] In some embodiments, the apparatus further includes:

[0033] The extraction module is used to extract features from the first image to obtain preset features;

[0034] The determining module is used for:

[0035] The target feature is determined from the preset features.

[0036] In some embodiments, the fusion module is configured to:

[0037] Align the preprocessed image and the feature image to obtain an aligned preprocessed image and an aligned feature image;

[0038] The aligned feature image is mapped onto the aligned preprocessed wave image to obtain the target image.

[0039] In some embodiments, the device includes:

[0040] A blur module is used to blur the feature image;

[0041] The fusion module is used for:

[0042] The target image is obtained by fusing the preprocessed image and the feature image after blurring.

[0043] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0044] processor;

[0045] Memory used to store processor-executable instructions;

[0046] The processor is configured to execute the steps of the method described in the first aspect embodiment.

[0047] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, wherein when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the steps of the method described in the first aspect of the disclosure.

[0048] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0049] As can be seen from the above embodiments, this disclosure, by determining the feature image corresponding to the target spectral band in the first image, obtains the target features that need to be retained in the first image, and fuses the preprocessed image and the feature image, thus achieving the retention of the required target features while beautifying the second image, making the beautification effect more natural and meeting the aesthetic individual needs of different users. Moreover, since the first image has more spectral channels than the second image, more information about the subject can be obtained from the first image, making the target features retained in the target image closer to the real subject, further improving the natural effect of the target image beautification.

[0050] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0051] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0052] Figure 1 This is a flowchart illustrating an image processing method.

[0053] Figure 2 This is one of the flowcharts illustrating an image processing method according to an exemplary embodiment;

[0054] Figure 3 This is a second image shown according to an exemplary embodiment;

[0055] Figure 4 This is one of the first images shown according to an exemplary embodiment;

[0056] Figure 5 This is a second of the first images shown according to an exemplary embodiment;

[0057] Figure 6 This is a second flowchart illustrating an image processing method according to an exemplary embodiment;

[0058] Figure 7 This is a schematic diagram of the structure of an image processing apparatus according to an exemplary embodiment;

[0059] Figure 8 This is a block diagram illustrating the structural composition of an electronic device according to an exemplary embodiment. Detailed Implementation

[0060] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses consistent with some aspects of this disclosure as detailed in the appended claims.

[0061] Take beautifying facial images as an example. Figure 1 As shown, the following method is used to process face images:

[0062] Step S10: Obtain the input image;

[0063] Step S20: Identify the skin region in the input image, that is, use the skin region detection algorithm to process the input image and distinguish the skin region and non-skin region in the input image. Non-skin regions generally include areas such as eyes, nostrils, and eyebrows.

[0064] Step S30: Apply a filtering algorithm to the skin area to obtain the filtered image. Filtering algorithms generally have two functions: smoothing or blurring the image, and removing noise. In skin smoothing algorithms, their main function is to filter out minor imperfections and smooth color transitions. The main idea is to take the weighted average of the center pixel color and the surrounding pixel colors, and then update the color value of the center pixel.

[0065] Step S40: Fuse the filtered image and the input image, replacing the skin region in the input image with the processed image to obtain the output image;

[0066] Step S50: Display the output image. While this image processing method can also perform skin smoothing for different skin features, it removes almost all blemishes from the face, failing to meet the personalized needs of different users. To further improve the image processing effect, the embodiments of this disclosure propose the following technical solutions.

[0067] Figure 2 An illustrative flowchart of an image processing method is shown, such as... Figure 2 As shown, a first aspect of this disclosure provides an image processing method, the method comprising:

[0068] Step S110: Based on the feature image corresponding to the target spectral band in the first image;

[0069] Step S120: Preprocess the second image to obtain a preprocessed image; wherein the second image and the first image are images corresponding to the same scene; the first image has more spectral channels than the second image;

[0070] Step S130: Fuse the preprocessed image and the feature image to obtain the target image.

[0071] The image processing method of this disclosure is applied to shooting scenarios including but not limited to portrait scenes, food scenes, landscape scenes, or plant scenes.

[0072] In step S110, the first image is the image acquired by the first image acquisition module, and the second image is the image acquired by the second image acquisition module.

[0073] The first image acquisition module and the second image acquisition module can both be the front-facing camera of an electronic device, or they can be the rear-facing camera of an electronic device.

[0074] The first image acquisition module and the second image acquisition module constitute a dual-camera imaging system for electronic devices.

[0075] The spectral bands of the first image include, but are not limited to, near-infrared, ultraviolet, far-infrared, or visible light ranges.

[0076] Generally, the spectral bands of the second image include the visible light range.

[0077] In some embodiments, the spectral channels of the second image include a red light channel, a blue light channel, and a green light channel.

[0078] The second image is an RGB image.

[0079] Generally, the second image acquisition module can acquire information such as the color, intensity, and spatial relative position of the objects being photographed in the scene.

[0080] The second image is more in line with human visual perception, but it is easily affected by insufficient ambient light, fog, and other adverse weather conditions, causing it to fail to capture or weaken some of the subject's detailed features. The first image, on the other hand, has more spectral channels and can capture more detailed features of the subject. For example, the first image acquisition module may include a spectral imaging chip or sensor. The first image acquisition module can acquire more information about the subject's physical and spectral properties.

[0081] In a non-limiting sense, physical properties include color, brightness, or shape. Spectral properties can be the properties of an object to absorb, reflect, or radiate light of different wavelengths.

[0082] The first image can provide more and more accurate information about the subject, making the image features in the first image closer to the real subject, which is beneficial to further improve the natural effect of the target image beautification in step S130.

[0083] Generally, the first image includes images of preset features corresponding to multiple spectral bands. One or more preset features are located in the image corresponding to one spectral band, and another or more preset features are located in the image corresponding to another spectral band. The feature image corresponding to the target spectral band is the image corresponding to the preset features (i.e., target features) that are to be retained.

[0084] The target image retains the target features from the first image.

[0085] Figure 3 The second image is shown as an example. Figure 4 The effect of the first image at a spectral band of 556 nm is shown as an example. Figure 5 The effect of the first image at a spectral band of 625 nm is shown as an example. (Comparison) Figure 3 and Figure 4 , Figure 5 It can be seen that, Figure 1 Facial details, such as acne, are not clearly visible, but Figure 4 It can be displayed more clearly in the middle, and Figure 5 It can also display Figure 3 and Figure 4 The mole features are not obvious in the middle. Figure 5 (At the middle circle). Therefore, different detailed features can be obtained through different spectral channels of the second image.

[0086] Without limitation, the first image can be a multispectral image or a hyperspectral image.

[0087] The first image can have several, a dozen, hundreds, or thousands of spectral channels or more.

[0088] Target features or preset features refer to blemishes that differ significantly from the target area in the image of the subject. These differences include, but are not limited to, color or brightness variations. For example, if the subject is a face, blemishes refer to features within the skin area that differ significantly from the skin area itself, such as acne or birthmarks.

[0089] In some embodiments, the target feature or preset feature is a facial feature.

[0090] In some embodiments, the feature image is a feature mask.

[0091] Unrestricted, the target area is the skin area.

[0092] In step S120, the preprocessing is used at least to remove or fade preset features on the image of the subject.

[0093] In some embodiments, the preprocessing includes at least one of the following:

[0094] Filtering algorithm processing, filter effect algorithm processing, sharpening algorithm processing.

[0095] Without limitation, the filtering algorithm includes at least one of the following: box filtering (such as mean filtering), statistical sorting filtering (such as median filtering), Gaussian filtering, and edge-preserving filtering (such as bilateral filtering). The filtering algorithm can remove noise and blemishes from the target region of the second image, resulting in a filtered image with smooth pixel changes and natural transitions.

[0096] Filter algorithms can include Canny (a multi-level edge detection algorithm), Structure Tensor (a region-based image segmentation algorithm), Sobel (an edge detection algorithm), etc., but are not limited to these.

[0097] The sharpening algorithm can be the Laplacian image sharpening algorithm, but it is not limited to this.

[0098] In some embodiments, the second image is preprocessed to obtain a preprocessed image, including:

[0099] The target region in the second image is filtered to obtain a filtered image.

[0100] When the scene includes a human face, the image processing method of this disclosure embodiment can be used for facial smoothing and beautification. The feature image corresponding to the target spectral band can be a feature image of one or more facial features. For example, the feature image can be an image with freckles, moles, or acne, etc.

[0101] The target area can be a skin area, including at least one of the following: facial skin, neck skin, or limb skin.

[0102] In some embodiments, determining the feature image corresponding to the target spectral band in the first image includes:

[0103] The image corresponding to the spectral segment where the preset feature at a preset position in the first image is located is determined to be the feature image corresponding to the target spectral segment, and / or, the image corresponding to the spectral segment where the preset feature in the first image has a coverage area greater than or equal to a preset area is determined to be the feature image corresponding to the target spectral segment.

[0104] In practical applications, the relative positional relationship between distinctive features and preset features can be used to determine whether a preset feature is in a preset position, thereby determining whether the preset feature is a target feature. For example, for a first image including a face, distinctive features can be facial features such as eyebrows, eyes, nose, mouth, and ears.

[0105] For example, if both the first and second images include a face, the eyebrow features of the face can be identified. Based on the coordinates of the eyebrow features in the first image and the coordinates of a preset feature in the first image, the relative positional relationship between the preset feature and the eyebrow features is determined. When the preset relationship is satisfied, the preset feature is located at a preset position, and this preset feature is the target feature. For example, the preset position could be located between the two eyebrows.

[0106] Without limitation, preset features or target features can be extracted from the first image using methods such as gray-level difference statistics or gray-level co-occurrence matrix.

[0107] Taking the first image as a face image as an example, most moles on the face are areas that are dark in the center and bright around the edges. Based on this feature, the difference of Gaussian algorithm can be used to detect moles on the face.

[0108] In practical applications, neural network models can also be used to obtain target features from the first image. The training data for these models includes historical data related to portrait retouching from image processing programs. This historical data reflects the user's aesthetic preferences, and training the neural network model with this data helps the model select target features that better reflect the user's individual aesthetic preferences.

[0109] In some embodiments, the method includes: identifying a target region in the second image. Without limitation, a skin probability detection algorithm can be used to identify the target region in the second image.

[0110] In step S130, compared to the second image, the target image not only smooths and beautifies the skin of the second image to obtain a target area with smooth pixel changes and natural transitions, but also retains the target features in the feature image. This achieves the goal of retaining the required target features while beautifying the second image, making the beautification effect more natural and meeting the aesthetic and individual needs of different users.

[0111] For example, when the subject includes a human face, in some applications, the face may have freckles. During image processing, it's often undesirable to remove the freckles, otherwise the characteristic of the freckle makeup is lost. Using the image processing method of this disclosure, a first image acquisition module can more accurately acquire a first image containing freckles, and the freckles can be identified as target features to obtain a feature mask. The skin area (i.e., the target area) of the second image is filtered to obtain a filtered image after skin smoothing and beautification. Finally, the target image obtained by fusing the filtered image and the feature mask retains the effect of the freckle makeup while also smoothing and beautifying other areas of the skin. This achieves the goal of removing facial blemishes while effectively preserving personal characteristics or makeup effects. This intelligent beautification method meets the personalized needs of users.

[0112] According to some other optional embodiments, determining the feature image corresponding to the target spectral band in the first image includes:

[0113] Based on a preset operation performed on the electronic device, the target feature in the first image to which the preset operation is directed is determined;

[0114] Based on the target feature, the spectral band in which the target feature is located is determined as the target spectral band; wherein, the image indicated by the target spectral band is the feature image.

[0115] Without limitation, the preset operation can be a touch operation performed by the user on the display screen of the electronic device. For example, the preset operation is an operation such as swiping, long pressing, dragging, or clicking.

[0116] Electronic devices include, but are not limited to, mobile phones, televisions, tablets, laptops, or wearable devices.

[0117] In practical applications, the image processing APP (application) interface on the electronic device display screen shows multiple preset feature images corresponding to multiple spectral bands. Based on the user's preset operation on the preset feature images on the display screen, the selected preset feature is determined as the target feature, and the feature image corresponding to the target spectral band is obtained.

[0118] According to some other alternative embodiments, the method further includes:

[0119] Feature extraction is performed on the first image to obtain preset features;

[0120] Determining the feature image corresponding to the target spectral band in the first image includes:

[0121] The target feature is determined from the preset features.

[0122] Without limitation, preset features can be extracted from the first image using feature detection methods. For example, preset features can be extracted from the first image using methods such as gray-level difference statistics or gray-level co-occurrence matrix.

[0123] The target feature is some or all of the preset features. The preset features may include the detailed features mentioned above, such as freckles, moles, or acne.

[0124] Let mask n represent the preset feature, where n is a positive integer not less than 0. If the preset features include mask1, mask2, or mask3, the target feature can be mask2, or the target features can be both mask1 and mask2.

[0125] According to some other optional embodiments, fusing the preprocessed image and the feature image to obtain the target image includes:

[0126] Align the preprocessed image and the feature image to obtain an aligned preprocessed image and an aligned feature image;

[0127] The aligned feature image is mapped onto the aligned preprocessed image to obtain the target image.

[0128] In practical applications, the mapping operation can map the target features in the aligned feature image to the aligned preprocessed image to obtain the target image.

[0129] Considering that the spatial positions of the first and second image acquisition modules will not completely overlap, the first and second images obtained for the same subject in the same scene will not be exactly the same. An alignment operation is needed to align the positional information of the subject in the first and second images so that the coordinates of the same feature of the subject are the same in both images.

[0130] Generally, alignment operations can involve processes such as cropping images and aligning coordinates.

[0131] According to some other alternative embodiments, the field of view of the first image acquisition module that acquires the first image is greater than or equal to the field of view of the second image acquisition module that acquires the second image.

[0132] The field of view of the first image acquisition module is greater than or equal to (close to) the field of view of the second image acquisition module, which can ensure that the first image contains all the contents of the second image, thus ensuring the comprehensiveness of the information in the first image and reducing the possibility of missing preset feature extraction.

[0133] However, because the field of view of the first image acquisition module is larger, the first image may contain more image information. Compared to the second image, the first image includes redundant information. Therefore, the redundant information in the first image can be removed by cropping, and then coordinate alignment can be performed to achieve alignment between the first and second images.

[0134] In some embodiments, aligning the preprocessed image and the feature image to obtain an aligned preprocessed image and an aligned feature image includes:

[0135] The first coordinates of the correction feature in the first image and the second coordinates of the correction feature in the second image are obtained respectively.

[0136] If the first coordinate and the second coordinate are different, transform the first coordinate into the target coordinate so that the target coordinate is equal to the second coordinate, or transform the second coordinate into the target coordinate so that the target coordinate is equal to the first coordinate.

[0137] Correction features are features present in both the first and second images. Examples include moles or eyes.

[0138] In practical applications, the first and second images can also be aligned using calibration methods.

[0139] According to some other alternative embodiments, the method further includes:

[0140] The feature image is blurred.

[0141] The process of fusing the preprocessed image and the feature image to obtain the target image includes:

[0142] The target image is obtained by fusing the preprocessed image and the feature image after blurring.

[0143] Without limitation, the blurring process can be Gaussian blur.

[0144] Blurring the feature images before fusion can make the fusion of the preprocessed images and feature images smoother and more natural, improving the fusion effect and enhancing the display effect of the target image.

[0145] In some embodiments, the method further includes:

[0146] The target region is blurred.

[0147] The preprocessing of the target region in the second image to obtain a preprocessed image includes:

[0148] The blurred target region is then filtered to obtain a filtered image (i.e., a fish hammer image).

[0149] The blurring of the target region can also be done using Gaussian blur.

[0150] Blurring the target region can further improve the fusion effect, making the fusion of the filtered image and the feature image smoother and more natural.

[0151] In practical applications, the target region and feature images can be blurred to further ensure the fusion effect.

[0152] In a specific example, such as Figure 6 As shown, the image processing method is applied to portrait beautification, and the electronic device is a mobile phone, including:

[0153] Step S210: Acquire the required input spectral image (i.e., the first image) and RGB image (i.e., the second image). The spectral image can be acquired using a spectral imaging device. The RGB image can be acquired using a traditional RGB imaging device. Both imaging devices (i.e., image acquisition modules) acquire images of the same human figure.

[0154] Step S220: Extract features from the spectral image to obtain preset features; determine the target features from the preset features to obtain the feature mask. This step corresponds to... Figure 6 Feature extraction algorithms are used in this context. Different facial features, such as freckles, moles, and acne, are clearly imaged at different wavelengths. Figure 4 and Figure 5 As shown. For imaging at a specific wavelength, feature detection methods can be used to extract facial features of interest and generate a corresponding facial feature mask matrix [mask1, mask2, mask3, ...], that is, to generate preset features.

[0155] Add selectable preset features to beauty apps on smartphones, such as freckles, moles, and acne. When users open the beauty app, they can select which facial features to remove or retain based on their aesthetic needs through preset operations. The retained preset features form a feature mask.

[0156] Step S230: Filter the target region in the second image to obtain a filtered image. This step corresponds to... Figure 6 The algorithm describes skin region detection and filtering algorithms. The skin probability detection algorithm identifies skin-colored regions (mask_skin) in the second image. For skin-colored regions of the same size as the original image (second image), Gaussian blur is applied before filtering.

[0157] Step S240: Fuse the filtered image and the feature mask to obtain the target image. In practical applications, the spectral image and RGB image can be aligned using a calibration method to obtain an aligned filtered image and an aligned feature mask. Then, the target features in the aligned feature mask are mapped onto the aligned filtered image to obtain the target image.

[0158] Step S250: Output image (i.e., output target image).

[0159] The image processing method of this disclosure utilizes a spectral imaging chip to simultaneously acquire more spectral characteristics, such as imaging and spectral data, providing a new means to identify facial features with different spectral characteristics. Furthermore, the APP layer can provide options for retaining or removing different facial features (i.e., preset features), enhancing the intelligence of the image processing method.

[0160] A second aspect of this disclosure provides an image processing apparatus, such as... Figure 7 As shown, the device 300 includes:

[0161] The determining module 310 is used to determine the feature image corresponding to the target spectral band in the first image;

[0162] The preprocessing module 320 is used to preprocess the target region in the second image to obtain a preprocessed image; wherein the second image and the first image are images corresponding to the same scene; the first image has more spectral channels than the second image.

[0163] The fusion module 330 is used to fuse the preprocessed image and the feature image to obtain the target image.

[0164] According to some optional embodiments, the determining module is configured to:

[0165] Based on a preset operation performed on the electronic device, the target feature in the first image to which the preset operation is directed is determined;

[0166] Based on the target feature, the spectral band in which the target feature is located is determined as the target spectral band; wherein, the image indicated by the target spectral band is the feature image.

[0167] According to some optional embodiments,

[0168] The device further includes:

[0169] The extraction module is used to extract features from the first image to obtain preset features;

[0170] The determining module is used for:

[0171] The target feature is determined from the preset features. Electronic device.

[0172] According to some optional embodiments, the fusion module is used for:

[0173] Align the preprocessed image and the feature image to obtain an aligned preprocessed image and an aligned feature image;

[0174] The aligned feature image is mapped onto the aligned preprocessed image to obtain the target image.

[0175] According to some alternative embodiments, the device includes:

[0176] A blur module is used to blur the feature image;

[0177] The fusion module is used for:

[0178] The target image is obtained by fusing the preprocessed image and the feature image after blurring.

[0179] According to some optional embodiments, the field of view of the first image acquisition module that acquires the first image is greater than or equal to the field of view of the second image acquisition module that acquires the second image.

[0180] According to some alternative embodiments, the spectral channels of the second image include a red light channel, a blue light channel, and a green light channel.

[0181] A third aspect of this disclosure provides an electronic device, including:

[0182] processor;

[0183] Memory used to store processor-executable instructions;

[0184] The processor is configured to execute the steps of the method described in the first aspect embodiment.

[0185] A fourth aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is able to perform the steps of the method described in the first embodiment.

[0186] In an exemplary embodiment, the multiple modules in the image processing apparatus may be implemented by one or more central processing units (CPUs), graphics processing units (GPUs), baseband processors (BPs), application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0187] Figure 8 This is a block diagram illustrating an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0188] Reference Figure 8 The electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0189] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0190] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0191] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0192] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0193] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0194] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0195] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 can detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0196] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0197] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0198] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0199] The methods disclosed in the several method embodiments provided in this disclosure can be arbitrarily combined without conflict to obtain new method embodiments.

[0200] The features disclosed in the several device embodiments provided in this disclosure can be arbitrarily combined without conflict to obtain new product embodiments.

[0201] The features disclosed in the several method or device embodiments provided in this disclosure can be arbitrarily combined without conflict to obtain new method embodiments or product embodiments.

[0202] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0203] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Feature extraction is performed on the first image to obtain multiple preset features corresponding to the target region in the first image; Based on a preset operation applied to the electronic device, a target feature to be retained in the first image pointed to by the preset operation is determined from the plurality of preset features, and the spectral segment where the target feature is located is determined as the target spectral segment, thereby obtaining a feature image corresponding to the target spectral segment in the first image; The target region in the second image is preprocessed to obtain a preprocessed image; wherein the second image and the first image are images corresponding to the same scene; the first image has more spectral channels than the second image, and the preprocessing is at least used to remove or fade other preset features on the second image besides the target feature; The preprocessed image and the feature image are fused to obtain the target image; wherein the target image retains the target features and removes the other preset features.

2. The method according to claim 1, characterized in that, The process of fusing the preprocessed image and the feature image to obtain the target image includes: Align the preprocessed image and the feature image to obtain an aligned preprocessed image and an aligned feature image; The aligned feature image is mapped onto the aligned preprocessed image to obtain the target image.

3. The method according to claim 1, characterized in that, The method further includes: The feature image is blurred. The process of fusing the preprocessed image and the feature image to obtain the target image includes: The target image is obtained by fusing the preprocessed image and the feature image after blurring.

4. The method according to claim 1, characterized in that, The field of view of the first image acquisition module that acquires the first image is greater than or equal to the field of view of the second image acquisition module that acquires the second image.

5. The method according to claim 1, characterized in that, The preprocessing includes at least one of the following: Filtering algorithm processing, filter effect algorithm processing, sharpening algorithm processing.

6. An image processing apparatus, characterized in that, The device includes: A determination module is used to extract features from a first image to obtain multiple preset features corresponding to a target region in the first image; based on a preset operation performed on an electronic device, a target feature to be retained in the first image pointed to by the preset operation is determined from the multiple preset features, and the spectral band where the target feature is located is determined as the target spectral band, thereby obtaining a feature image corresponding to the target spectral band in the first image; A preprocessing module is used to preprocess the target region in the second image to obtain a preprocessed image; wherein the second image and the first image are images corresponding to the same scene; the first image has more spectral channels than the second image, and the preprocessing is used at least to remove or fade other preset features on the second image besides the target feature; The fusion module is used to fuse the preprocessed image and the feature image to obtain a target image; wherein the target image retains the target features and removes the other preset features.

7. The apparatus according to claim 6, characterized in that, The fusion module is used for: Align the preprocessed image and the feature image to obtain an aligned preprocessed image and an aligned feature image; The aligned feature image is mapped onto the aligned preprocessed image to obtain the target image.

8. The apparatus according to claim 6, characterized in that, The device includes: A blur module is used to blur the feature image; The fusion module is used for: The target image is obtained by fusing the preprocessed image and the feature image after blurring.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the steps of the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the steps of the method according to any one of claims 1 to 5.

Images