Image processing method, device, handheld terminal and computer-readable storage medium

Abstract

The present application relates to an image processing method, device, handheld terminal and computer-readable storage medium. An electronic device obtains a multispectral image containing a portrait area; then, based on the difference in the color channel values ​​of each pixel in the multispectral image, the multispectral image is segmented to obtain a background area other than the portrait area in the multispectral image; finally, a blurring operation is performed on the background area. The above method is used to accurately segment the portrait area and the background area in the multispectral image; based on the above image segmentation, the electronic device performs a blurring operation on the background area, which can avoid the loss of portrait information caused by inaccurate segmentation and improve the background blurring effect of the multispectral image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, handheld terminal and computer-readable storage medium. Background Art

[0002] When users use mobile phones and other terminal devices, the terminal devices often need to process specific areas in the image; in this case, these areas need to be segmented from the image. For example, when a user wants to blur the background of an image containing a portrait, the portrait area and the background area in the image need to be segmented and the background area needs to be blurred.

[0003] In traditional methods, the terminal device can recognize the RGB image through machine learning models and other methods, and then segment the recognized portrait area.

[0004] However, when the colors of the portrait area and the background area in the RGB image are similar, the above method will lead to inaccurate image segmentation results and reduce the background blur effect. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, device, handheld terminal and computer-readable storage medium, which can enhance the background blur effect of an image.

[0006] In a first aspect, an image processing method is provided, the method comprising:

[0007] Acquire a multispectral image containing a portrait area; wherein the multispectral image includes color channel values ​​of each pixel in multiple spectral bands;

[0008] According to the difference of the color channel value of each pixel in the multispectral image, the multispectral image is segmented to obtain the background area except the portrait area in the multispectral image;

[0009] Blur the background area.

[0010] In a second aspect, an image processing device includes:

[0011] An acquisition module is used to acquire a multispectral image containing a portrait area; wherein the multispectral image includes color channel values ​​of each pixel in multiple spectral bands;

[0012] A segmentation module is used to segment the multispectral image according to the difference of the color channel value of each pixel in the multispectral image, so as to obtain the background area except the portrait area in the multispectral image;

[0013] The blur module is used to blur the background area.

[0014] In a third aspect, a handheld terminal includes a processor and a multispectral camera, wherein the multispectral camera is used to acquire a multispectral image, and the processor is used to:

[0015] Acquire a multispectral image containing a portrait area; wherein the multispectral image includes color channel values ​​of each pixel in multiple spectral bands;

[0016] According to the difference of the color channel value of each pixel in the multispectral image, the multispectral image is segmented to obtain the background area except the portrait area in the multispectral image;

[0017] A blurring operation is performed on the background area.

[0018] In a fourth aspect, an electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the image processing method in the first aspect.

[0019] In a fifth aspect, a computer-readable storage medium stores a computer program, which implements the steps of the image processing method in the first aspect when executed by a processor.

[0020] In the above-mentioned image processing method, device, handheld terminal and computer-readable storage medium, the electronic device obtains a multispectral image containing a portrait area; then, according to the difference in the color channel values ​​of each pixel in the multispectral image, the multispectral image is segmented to obtain the background area excluding the portrait area in the multispectral image; finally, the background area is blurred. Since the multispectral image includes the color channel values ​​of each pixel in multiple spectral bands, when the electronic device segments the multispectral image by the difference in the color channel values ​​of each pixel, the multiple color channel values ​​can more clearly represent the difference between the portrait area and the background area with similar colors, thereby obtaining a more accurate image segmentation effect; based on the above-mentioned image segmentation, the electronic device performs a blurring operation on the background area, which can avoid the loss of portrait information caused by inaccurate segmentation and improve the background blurring effect of the multispectral image. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A diagram showing an application environment of an image processing method in an embodiment;

[0023] Figure 2 is a flowchart of an image processing method in one embodiment;

[0024] Figure 3 is a schematic diagram of an image processing method in one embodiment;

[0025] Figure 4 is a flowchart of an image processing method in another embodiment;

[0026] Figure 5 is a schematic diagram of an image processing method in another embodiment;

[0027] Figure 6 is a flowchart of an image processing method in another embodiment;

[0028] Figure 7 is a flowchart of an image processing method in another embodiment;

[0029] Figure 8 is a flowchart of an image processing method in another embodiment;

[0030] Fig. 9 is a structural block diagram of an image processing device in one embodiment;

[0031] Fig.10 is a structural block diagram of an image processing device in another embodiment;

[0032] Fig.11 is a structural block diagram of an image processing device in another embodiment;

[0033] Fig.12 FIG. 4 is a structural block diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first client may be referred to as a second client, and similarly, a second client may be referred to as a first client. Both the first client and the second client are clients, but they are not the same client.

[0036] Figure 1 FIG. 1 is a schematic diagram of an application environment of an image processing method in an embodiment. Figure 1 As shown, the application environment includes an electronic device 100, and the electronic device 100 may include a multispectral camera 110, and the multispectral camera 110 may be used to collect multispectral images. The electronic device 100 may be a handheld terminal device including a mobile phone, a tablet computer, or any terminal device including a laptop computer, a desktop computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a vehicle-mounted computer, a wearable device, etc.

[0037] Figure 2 FIG. 1 is a flowchart of an image processing method in an embodiment. The image processing method in this embodiment is executed on Figure 1 The electronic device in the example is used for description. Figure 2 As shown, the image recognition method includes:

[0038] S101, obtaining a multispectral image including a portrait area; wherein the multispectral image includes color channel values ​​of each pixel in multiple spectral bands.

[0039] The application scenario of the above image processing method can be that the user processes the image during the photo shooting process, or the user processes the stored multispectral image, which is not limited here. The electronic device can call the stored multispectral image, or call the multispectral camera set in the electronic device to collect the multispectral image, which is not limited here. Optionally, the electronic device can be a handheld terminal, and the handheld terminal can receive the photo shooting instruction input by the user; then in response to the photo shooting instruction, the multispectral camera in the handheld terminal is called to collect the multispectral image containing the portrait area.

[0040] The multi-spectral camera may be provided with a plurality of repeated filter units on a plane, and each filter unit corresponds to a plurality of color channels. At least four color channels are provided in the multi-spectral camera, and the number of the color channels may also be eight or nine, etc., which is not limited here. The spectral bands corresponding to the multiple color channels are different. For example, the spectral range between 400 nanometers and 1100 nanometers in wavelength may be divided into nine spectral bands to obtain nine color channels; Figure 3The figure shows a filter unit including 9 color channels. The spectral bandwidths corresponding to the above-mentioned multiple color channels may be the same or different. When an electronic device calls a multispectral camera to collect a multispectral image, the light reflected in the scene to be shot is processed by the filter units corresponding to the multiple color channels, and the color channel values ​​corresponding to the multiple color channels can be obtained. For example, when the multispectral camera includes 9 color channel values, each pixel includes 9 color channel values, which has richer spectral information than RGB images. The multispectral image obtained by the electronic device may include multiple color channel values ​​corresponding to each pixel.

[0041] The multispectral image may include a portrait area, and the portrait area may correspond to one portrait or multiple portraits, which is not limited here. In addition, the multispectral image may also include other objects, which are not limited here.

[0042] S102: Segment the multispectral image according to the difference in color channel values ​​of pixels in the multispectral image to obtain a background area other than the portrait area in the multispectral image.

[0043] In a multispectral image, there are differences in the color channel values ​​corresponding to the portrait area and the background area. In particular, when the portrait area and the background area are similar in color, since the color channel values ​​contain richer pixel information than the RGB image, the portrait area and the background area can be segmented by the differences between the above multiple color channel values. The above differences can be represented by the absolute change in the color channel values ​​or by the relative change in the color channel values, which is not limited here.

[0044] The electronic device can perform segmentation processing on the multispectral image according to the difference in color channel values ​​of each pixel; wherein, the electronic device can compare the color channel values ​​of adjacent pixels to determine the difference in color channel values ​​of adjacent pixels, or determine the spectral characteristics of the pixel according to the color channel value of each pixel, and then further determine the difference between the spectral characteristics of adjacent pixels, which is not limited here.

[0045] The above spectral features can be used to characterize the reflection characteristics of the light of the acquisition object corresponding to the pixel point in different spectral bands. The above spectral features can be the color channel values ​​of the pixel, or they can be color parameters obtained after interpolation of the color channel values. For example, if the pixel includes 9 color channel values, the electronic device can interpolate the above 9 color channel values ​​in pairs to obtain 17 color parameters. The specific form of the spectral features of the pixel is not limited here.

[0046] When calculating the difference in color channel values ​​between two adjacent pixels, the electronic device may calculate the difference in each color channel value separately according to the order of spectral arrangement; or it may be a difference value obtained by fitting the difference in each color channel value; the calculation method of the above difference is not limited here.

[0047] S103: Perform blurring operation on the background area.

[0048] After segmenting the multispectral image, the electronic device can obtain the portrait area and the background area in the multispectral image. Further, the electronic device can perform a blurring operation on the background area.

[0049] The above blurring operation can make the depth of field of the background area shallower, so that the focus of the multi-spectral image is concentrated on the portrait area. In the above image blurring process, the electronic device can use the average value of the color channel values ​​of the surrounding pixels corresponding to each pixel in the background area as the color channel value of the pixel to reduce the difference in color channel values ​​between adjacent pixels to achieve the effect of image blurring; or, the electronic device can also use a Gaussian blur algorithm to blur the above background area, which is not limited here.

[0050] The electronic device can use the segmentation boundary between the portrait area and the background area as a reference to blur the image of the background area, or use the above-mentioned segmentation boundary as a reference to set a transition area between the portrait area and the background area to be blurred, and then use different blurring weights to blur the transition area and the background area to be blurred respectively; the method of the above-mentioned blurring operation is not limited here.

[0051] In the above-mentioned image processing method, the electronic device obtains a multispectral image including a portrait area; then, according to the difference in the color channel values ​​of each pixel in the multispectral image, the multispectral image is segmented to obtain the background area excluding the portrait area in the multispectral image; finally, the background area is blurred. Since the multispectral image includes the color channel values ​​of each pixel in multiple spectral bands, when the electronic device segments the multispectral image by the difference in the color channel values ​​of each pixel, the multiple color channel values ​​can more clearly represent the difference between the portrait area and the background area with similar colors, thereby obtaining a more accurate image segmentation effect; based on the above-mentioned image segmentation, the electronic device performs a blurring operation on the background area, which can avoid the loss of portrait information caused by inaccurate segmentation and improve the background blurring effect of the multispectral image.

[0052] Figure 4 FIG. 1 is a flow chart of an image processing method in another embodiment. This embodiment relates to a method for electronic equipment to perform segmentation processing on a multispectral image. Based on the above embodiment, Figure 4 As shown, the above S102 includes:

[0053] S201 , respectively performing fitting processing on multiple color channel values ​​of each pixel to obtain a spectral curve of the pixel.

[0054] The electronic device can perform fitting processing on the color channel value of each pixel to obtain the spectral curve of each pixel. Figure 5 , which is an example of a spectrum curve, the horizontal axis represents the spectrum wavelength, for example, 400 nanometers to 1100 nanometers, and the vertical axis represents the intensity of the received light.

[0055] The electronic device may adopt a polynomial fitting method to determine the spectral curve corresponding to the color channel value, or may adopt a Gaussian fitting method to obtain the spectral curve. The fitting method of the spectral curve is not limited herein.

[0056] S202: Segment the multispectral image according to the difference between the spectral curves of adjacent pixels in the multispectral image.

[0057] Furthermore, the electronic device may segment the multi-spectral image by calculating the difference between spectral curves corresponding to two adjacent pixels.

[0058] When calculating the difference between spectral curves corresponding to two pixels, the electronic device can map the two spectral curves into the same feature space, and determine the degree of difference between the two spectral curves by calculating the Euclidean distance between the two spectral curves; or, the electronic device can also sample each spectral curve at a preset interval, input the sampling value corresponding to each spectral curve into a preset model, and output the difference value of the two spectral curves through the preset model; the calculation method of the above difference is not limited here.

[0059] The electronic device may consider that the greater the difference between the spectral curves of two adjacent pixels, the greater the possibility that the two pixels belong to the same image area, and thus the segmentation boundary in the multispectral image may be determined based on the comparison result of the difference between the spectral curves, and the multispectral image may be segmented.

[0060] In the above-mentioned image processing method, the electronic device can more accurately determine whether the spectral characteristics of two adjacent pixels are the same by obtaining the spectral curve of each pixel, thereby improving the segmentation accuracy of the multi-spectral image.

[0061] In the following embodiments, specific segmentation methods for segmenting a multispectral image according to differences in spectral curves of adjacent pixels are respectively introduced.

[0062] Figure 6FIG. 1 is a flow chart of an image processing method in another embodiment. This embodiment relates to a method for electronic equipment to perform segmentation processing on a multispectral image. Based on the above embodiment, Figure 6 As shown, the above S202 includes:

[0063] S301, calculating the difference value of the spectrum curves corresponding to two adjacent pixels.

[0064] The electronic device may calculate the mean square error of the spectral curves corresponding to two adjacent pixels, and then determine the mean square error as the difference value of the spectral curves corresponding to the two adjacent pixels.

[0065] S302: Mark two pixels whose difference values ​​are smaller than a preset threshold as pixels of the same object.

[0066] A difference value threshold may be preset in the electronic device. When the difference value of two spectral curves is less than the preset threshold, the electronic device may consider that the two pixels belong to the same object and may mark the two pixels as pixels of the same object. For example, the electronic device may take a pixel on the boundary of a multi-spectral image as a starting point, mark the pixel as 1, and determine the adjacent pixels of the pixel; further, the electronic device may respectively calculate the difference value of the spectral curve corresponding to the pixel and the adjacent pixel, and if the difference value is less than the preset threshold, mark the adjacent pixel as 1; if the difference value of the spectral curve corresponding to the adjacent pixel and the pixel is greater than or equal to the preset threshold, the pixel may be marked as 2, indicating that the adjacent pixel is a pixel of another object; each pixel is traversed in the above manner to determine the marking result of each pixel.

[0067] S303: Determine the segmentation boundary between the portrait area and the background area in the multispectral image according to the marking result, and use the segmentation boundary to perform segmentation processing on the multispectral image.

[0068] Furthermore, the electronic device can group pixels with similar spectral curves to form an area based on the labeling result, thereby determining the segmentation boundary of the portrait area and the background area. The electronic device can determine adjacent pixels with different labels as pixels on the segmentation boundary to obtain the segmentation boundary; optionally, based on the labeling result, the pixels labeled as the same object can be connected to obtain a connected area; then, the boundary of the connected area is denoised and smoothed to determine it as the segmentation boundary of the portrait area and the background area; the method for determining the above segmentation boundary is not limited here.

[0069] The electronic device can use the above segmentation boundary to perform segmentation processing on the image to obtain the portrait area and the background area of ​​the multispectral image.

[0070] In the above-mentioned image processing method, the electronic device can quickly and accurately determine the segmentation boundary of the portrait area and the background area by marking two pixels whose difference value is less than a preset threshold as pixels of the same object, thereby completing the segmentation processing of the multispectral image and improving the accuracy and efficiency of background blur processing of the multispectral image.

[0071] Figure 7 FIG. 1 is a flow chart of an image processing method in another embodiment. This embodiment relates to a method for electronic equipment to perform segmentation processing on a multispectral image. Based on the above embodiment, Figure 7 As shown, the above S202 includes:

[0072] S401. Determine a target pixel belonging to a portrait area in a multispectral image according to a preset reference spectral curve; a first difference value between a spectral curve of the target pixel and the reference spectral curve is less than a first difference threshold.

[0073] The electronic device can set a reference spectral curve of the portrait, and then calculate the first difference value between each spectral curve and the reference spectral curve; then, compare the first difference value with the first difference threshold, and when the first difference value is less than the first difference threshold, determine the pixel corresponding to the first difference value as the target pixel belonging to the portrait area. The spectral range corresponding to the multi-spectral image can include spectral bands such as infrared light and near-infrared light. Since the depth of field and other features corresponding to the portrait area and the background area are different, it is possible to distinguish which pixel may belong to the portrait area and which pixel may belong to the background area based on the difference in the spectral curves.

[0074] In another implementation, the electronic device may also preset a spectral curve distribution of the portrait area, and the spectral curve distribution may include spectral curves of multiple pixels, and the spectral curves of the multiple pixels may include spectral curves corresponding to different parts of the portrait, such as hair, skin, etc. Further, the electronic device may compare the spectral curve distribution with the spectral curves of each pixel in the multispectral image to determine a target pixel area in the multispectral image that may belong to the portrait area.

[0075] S402: Determine adjacent pixels of the target pixel, and calculate a second difference value between a spectral curve of the target pixel and a spectral curve of the adjacent pixels.

[0076] Based on the above steps, the electronic device can determine the approximate range of the multispectral image that may belong to the portrait area, that is, the target pixel area that may belong to the portrait area. Further, the electronic device can accurately determine the segmentation boundary between the portrait area and the background area based on the difference in the spectral curve between the target pixel and the adjacent pixels.

[0077] The electronic device may determine the second difference value of the spectral curve between each target pixel and the corresponding adjacent pixel, or may extract part of the target pixels located on the area contour of the target pixel area in the target pixel area, and then calculate the second difference value of the spectral curve between the part of the target pixels and the corresponding adjacent pixels; the method for obtaining the above-mentioned second difference value is not limited herein.

[0078] S403: Determine a segmentation boundary between the portrait area and the background area according to the second difference value, and use the segmentation boundary to perform segmentation processing on the multispectral image.

[0079] The electronic device can determine the target pixel whose second difference value is less than the preset threshold and the corresponding adjacent pixels as pixels on the segmentation boundary, thereby obtaining the segmentation boundary of the portrait area and the background area. Further, the electronic device uses the segmentation boundary to segment the multispectral image to obtain the portrait area and the background area of ​​the multispectral image.

[0080] In the above-mentioned image processing method, the electronic device determines the target pixels that may belong to the portrait area in the multispectral image by referring to the spectral curve, and then determines the segmentation boundary according to the difference value between the target pixel and the adjacent pixels. This can reduce the number of pixels for calculating the spectral curve difference, reduce the amount of processing data for image processing, and thus improve the efficiency of image processing.

[0081] Figure 8 FIG. 1 is a flow chart of an image processing method in another embodiment. This embodiment relates to a method for electronic equipment to perform segmentation processing on a multispectral image. Based on the above embodiment, Figure 8 As shown, the above S202 includes:

[0082] S501: Input the spectral curve of each pixel in the multispectral image into a preset image segmentation model to obtain the segmentation boundary of the portrait area and the background area in the multispectral image.

[0083] S502: Use segmentation boundaries to perform segmentation processing on the multispectral image.

[0084] The above-mentioned image segmentation model is obtained by training with a multispectral image sample set, and the multispectral image samples include the spectral curve of each pixel in the historical multispectral image, and the segmentation boundaries of the portrait area and the background area in the historical multispectral image.

[0085] The process of establishing the above-mentioned image segmentation model may include: taking the spectral curve of each pixel in the historical multispectral image as a reference input of the initial image segmentation model, taking the segmentation boundary of the portrait area and the background area in the historical multispectral image as a reference output of the initial image segmentation model, training the initial image segmentation model according to a preset loss function, and obtaining the image segmentation model.

[0086] In the above-mentioned image processing method, the electronic device can more accurately determine the segmentation boundary of the portrait area and the background area in the multispectral image through the image segmentation model, thereby improving the accuracy of image segmentation and further improving the background blur effect of the multispectral image.

[0087] It should be understood that although Figure 2-8 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-8 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0088] Fig. 9 FIG. 1 is a structural block diagram of an image processing device according to an embodiment. Fig. 9 As shown, the above device comprises:

[0089] The acquisition module 10 is used to acquire a multispectral image containing a portrait area; wherein the multispectral image includes color channel values ​​of each pixel in multiple spectral bands;

[0090] A segmentation module 20, configured to segment the multispectral image according to the difference in color channel values ​​of each pixel in the multispectral image, to obtain a background area other than the portrait area in the multispectral image;

[0091] The blur module 30 is used to perform blur processing operation on the background area.

[0092] In one embodiment, based on the above embodiment, Fig.10 As shown, the segmentation module 20 includes:

[0093] A fitting unit 201 is used to perform fitting processing on the multiple color channel values ​​of each pixel to obtain a spectral curve of the pixel;

[0094] The processing unit 202 is configured to perform segmentation processing on the multispectral image according to differences in spectral curves of adjacent pixels in the multispectral image.

[0095] In one embodiment, based on the above embodiment, the processing unit 202 is specifically used to: calculate the difference value of the spectral curves corresponding to two adjacent pixels; mark the two pixels whose difference value is less than a preset threshold as pixels of the same object; determine the segmentation boundary of the portrait area and the background area in the multispectral image according to the marking result, and use the segmentation boundary to segment the multispectral image.

[0096] In one embodiment, based on the above embodiment, the processing unit 202 is specifically configured to: determine the mean square error of the spectral curves of the two adjacent pixels as a difference value of the spectral curves corresponding to the two adjacent pixels.

[0097] In one embodiment, based on the above embodiment, the processing unit 202 is specifically used to: perform region connectivity processing on pixels marked as the same object according to the marking result to obtain a connected region; perform denoising and smoothing processing on the boundary of the connected region to determine it as the segmentation boundary of the portrait region and the background region.

[0098] In one embodiment, based on the above embodiment, the processing unit 202 is specifically used to: determine the target pixel belonging to the portrait area in the multispectral image according to a preset reference spectral curve; the first difference value between the spectral curve of the target pixel and the reference spectral curve is less than a first difference threshold; determine the adjacent pixels of the target pixel, and calculate the second difference value between the spectral curve of the target pixel and the spectral curve of the adjacent pixels; determine the segmentation boundary of the portrait area and the background area according to the second difference value, and use the segmentation boundary to segment the multispectral image.

[0099] In one embodiment, based on the above embodiment, the processing unit 202 is specifically used to: input the spectral curve of each pixel in the multispectral image into a preset image segmentation model to obtain the segmentation boundary of the portrait area and the background area in the multispectral image; wherein the image segmentation model is obtained by training with a multispectral image sample set, and the multispectral image sample includes the spectral curve of each pixel in the historical multispectral image, and the segmentation boundary of the portrait area and the background area in the historical multispectral image; and the multispectral image is segmented using the segmentation boundary.

[0100] In one embodiment, based on the above embodiment, Fig.11As shown, the above-mentioned device also includes a training module 40, which is used to: use the spectral curve of each pixel in the historical multispectral image as a reference input of the initial image segmentation model, use the segmentation boundary of the portrait area and the background area in the historical multispectral image as a reference output of the initial image segmentation model, train the initial image segmentation model according to a preset loss function, and obtain the image segmentation model.

[0101] In one embodiment, based on the above embodiment, the acquisition module 10 is specifically used to: receive a photo taking instruction input by a user; in response to the photo taking instruction, call a multispectral camera in a handheld terminal to collect the multispectral image containing the portrait area.

[0102] The division of the modules in the above-mentioned image processing device is only for illustration. In other embodiments, the image processing device may be divided into different modules as needed to complete all or part of the functions of the above-mentioned image processing device.

[0103] For the specific definition of the image processing device, please refer to the definition of the image processing method above, which will not be repeated here. Each module in the above image processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0104] In one embodiment, a handheld terminal is provided, comprising a processor and a multispectral camera, wherein the multispectral camera is used to acquire a multispectral image, and the processor is used to:

[0105] Acquire a multispectral image containing a portrait area; wherein the multispectral image includes color channel values ​​of each pixel in multiple spectral bands;

[0106] According to the difference of the color channel value of each pixel in the multispectral image, the multispectral image is segmented to obtain the background area except the portrait area in the multispectral image;

[0107] A blurring operation is performed on the background area.

[0108] Fig.12Schematic diagram of the internal structure of an electronic device in an embodiment. The electronic device can be any terminal device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a car computer, a wearable device, etc. The electronic device includes a processor and a memory connected via a system bus. Among them, the processor may include one or more processing units. The processor may be a CPU (Central Processing Unit) or a DSP (Digital Signal Processing), etc. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement an image processing method provided in each of the following embodiments. The internal memory provides a cache operating environment for the operating system computer program in the non-volatile storage medium.

[0109] The implementation of each module in the image processing device provided in the embodiment of the present application may be in the form of a computer program. The computer program may be run on a terminal or a server. The program module constituted by the computer program may be stored in a memory of an electronic device. When the computer program is executed by a processor, the steps of the method described in the embodiment of the present application are implemented.

[0110] The embodiment of the present application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions are used as steps of the image processing method.

[0111] An embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the image processing method.

[0112] Any reference to memory, storage, database or other medium used in this application may include non-volatile and / or volatile memory. Non-volatile memory may include ROM (Read-Only Memory), PROM (Programmable Read-only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-only Memory) or flash memory. Volatile memory may include RAM (Random Access Memory), which is used as an external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), SDRAM (Synchronous Dynamic Random Access Memory), Double Data Rate DDRSDRAM (Double Data Rate Synchronous Dynamic Random Access memory), ESDRAM (Enhanced Synchronous Dynamic Random Access memory), SLDRAM (Sync Link Dynamic Random Access Memory), RDRAM (Rambus Dynamic Random Access Memory), and DRDRAM (Direct Rambus Dynamic Random Access Memory).

[0113] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a multispectral image containing a portrait area; wherein the multispectral image includes color channel values ​​of each pixel in multiple spectral bands; According to the difference in color channel values ​​of adjacent pixels in the multispectral image, segmenting the portrait area and the background area of ​​the multispectral image to obtain the background area excluding the portrait area in the multispectral image; A blurring operation is performed on the background area.

2. The method according to claim 1, characterized in that The segmentation process of the portrait area and the background area of ​​the multispectral image according to the difference in color channel values ​​of adjacent pixels in the multispectral image includes: Performing fitting processing on the multiple color channel values ​​of each pixel respectively to obtain a spectral curve of the pixel; The multispectral image is segmented according to the difference between the spectral curves of adjacent pixels in the multispectral image.

3. The method according to claim 2, characterized in that The segmentation process of the multispectral image according to the difference of spectral curves of adjacent pixels in the multispectral image comprises: Calculate the difference value of the spectral curves corresponding to two adjacent pixels; Marking the two pixels whose difference values ​​are smaller than a preset threshold as pixels of the same object; According to the marking result, a segmentation boundary between a portrait area and a background area in the multispectral image is determined, and the multispectral image is segmented using the segmentation boundary.

4. The method according to claim 3, characterized in that: The calculating the difference value of the spectral curves corresponding to two adjacent pixels includes: The mean square error of the spectral curves of the two adjacent pixels is determined as a difference value of the spectral curves corresponding to the two adjacent pixels.

5. The method according to claim 3, characterized in that: Determining the segmentation boundary of the portrait area and the background area in the multispectral image according to the marking result includes: According to the labeling result, pixels labeled as the same object are subjected to region connectivity processing to obtain a connected region; De-noising and smoothing are performed on the boundary of the connected area to determine it as the segmentation boundary of the portrait area and the background area.

6. The method according to claim 2, characterized in that The segmentation process of the multispectral image according to the difference of spectral curves of adjacent pixels in the multispectral image comprises: According to a preset reference spectral curve, a target pixel belonging to a portrait area is determined in the multispectral image; a first difference value between the spectral curve of the target pixel and the reference spectral curve is less than a first difference threshold; Determining adjacent pixels of the target pixel, and calculating a second difference value between a spectral curve of the target pixel and a spectral curve of the adjacent pixels; A segmentation boundary between the portrait area and the background area is determined according to the second difference value, and the segmentation boundary is used to perform segmentation processing on the multispectral image.

7. The method according to claim 2, characterized in that The segmentation process of the multispectral image according to the difference of spectral curves of adjacent pixels in the multispectral image comprises: Inputting the spectral curve of each pixel in the multispectral image into a preset image segmentation model to obtain the segmentation boundary of the portrait area and the background area in the multispectral image; wherein the image segmentation model is obtained by training with a multispectral image sample set, and the multispectral image sample includes the spectral curve of each pixel in the historical multispectral image, and the segmentation boundary of the portrait area and the background area in the historical multispectral image; The multispectral image is segmented using the segmentation boundary.

8. The method according to claim 7, characterized in that The method further comprises: The spectral curve of each pixel in the historical multispectral image is used as a reference input of the initial image segmentation model, the segmentation boundary of the portrait area and the background area in the historical multispectral image is used as a reference output of the initial image segmentation model, and the initial image segmentation model is trained according to a preset loss function to obtain the image segmentation model.

9. The method according to any one of claims 1 to 8, characterized in that: The step of acquiring a multispectral image containing a portrait area includes: Receive a photo taking command input by a user; In response to the photographing instruction, a multispectral camera in the handheld terminal is called to collect the multispectral image containing the portrait area.

10. An image processing device, characterized in that: include: An acquisition module, used to acquire a multispectral image containing a portrait area; wherein the multispectral image includes color channel values ​​of each pixel in multiple spectral bands; a segmentation module, configured to segment the portrait area and the background area of ​​the multispectral image according to the difference in color channel values ​​of adjacent pixels in the multispectral image, so as to obtain the background area excluding the portrait area in the multispectral image; The blur module is used to perform blur processing operations on the background area.

11. A handheld terminal, characterized in that: The invention comprises a processor and a multispectral camera, wherein the multispectral camera is used to obtain a multispectral image, and the processor is used to: Acquire a multispectral image containing a portrait area; wherein the multispectral image includes color channel values ​​of each pixel in multiple spectral bands; According to the difference in color channel values ​​of adjacent pixels in the multispectral image, segmenting the portrait area and the background area of ​​the multispectral image to obtain the background area excluding the portrait area in the multispectral image; A blurring operation is performed on the background area.

12. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the computer program is executed by the processor, the processor is caused to perform the steps of the image processing method according to any one of claims 1 to 9.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images