Image processing method and device and electronic equipment

Abstract

The invention provides an image processing method and apparatus, and an electronic device. The method comprises the steps of obtaining a first image; performing down-sampling processing on the first image to obtain a down-sampled image; carrying out average pooling processing on pixels in the down-sampling image to obtain a first pooling image; determining a plurality of pixel merging weights corresponding to pixels in the down-sampling image; and based on the plurality of pixel combination weights, carrying out pixel combination processing on the down-sampling image and the first pooling image to obtain a second image.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular to an image processing method, device and electronic device. Background Art

[0002] Image downsampling can adjust the resolution of a video or image so that the video or image can reach a suitable size.

[0003] At present, electronic devices can obtain a downsampled image corresponding to an image based on the nearest neighbor interpolation method. For example, the electronic device can select the pixel closest to the sampling point as a new pixel in the downsampled image to obtain the downsampled image. However, when downsampling an image according to the above method, a lot of image information will be lost, resulting in a lower definition of the downsampled image. Summary of the invention

[0004] The present disclosure provides an image processing method, device and electronic device, which are used to solve one or more technical problems in the prior art.

[0005] In a first aspect, the present disclosure provides an image processing method, the method comprising:

[0006] acquiring a first image;

[0007] Performing downsampling processing on the first image to obtain a downsampled image;

[0008] Performing average pooling processing on pixels in the downsampled image to obtain a first pooled image;

[0009] Determining a plurality of pixel merging weights corresponding to pixels in the downsampled image;

[0010] Based on the multiple pixel merging weights, pixel merging processing is performed on the downsampled image and the first pooled image to obtain a second image.

[0011] In a second aspect, the present disclosure provides an image processing device, the image processing device comprising an acquisition module, a sampling module, a pooling module, a determination module and a processing module, wherein:

[0012] The acquisition module is used to acquire a first image;

[0013] The sampling module is used to perform down-sampling processing on the first image to obtain a down-sampled image;

[0014] The pooling module is used to perform average pooling processing on the pixels in the downsampled image to obtain a first pooled image;

[0015] The determination module is used to determine a plurality of pixel merging weights corresponding to pixels in the downsampled image;

[0016] The processing module is used to perform pixel merging processing on the down-sampled image and the first pooled image based on the multiple pixel merging weights to obtain a second image.

[0017] In a third aspect, an embodiment of the present disclosure provides an electronic device including: a processor and a memory;

[0018] The memory stores computer-executable instructions;

[0019] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the image processing method as described in the first aspect and various possible aspects of the first aspect.

[0020] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the image processing method as described in the first aspect and various possible aspects of the first aspect are implemented.

[0021] The present disclosure provides an image processing method, device and electronic device, wherein the electronic device can obtain a first image, and perform downsampling processing on the first image to obtain a downsampled image, the electronic device performs average pooling processing on the pixels in the downsampled image to obtain a first pooled image, and determines a plurality of pixel merging weights corresponding to the pixels in the downsampled image, and based on the plurality of pixel merging weights, performs pixel merging processing on the downsampled image and the first pooled image to obtain a second image. In the above method, since more image information in the first image can be retained in the first pooled image, and the second image is obtained by performing pixel merging of the downsampled image and the first pooled image, more image information of the first image can be retained in the second image, and the resolution of the second image is smaller than that of the first image, so that the image clarity after image downsampling can be improved on the basis of reducing the image resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0024] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0025] Figure 3 A schematic diagram of a process for determining an image block provided by an embodiment of the present disclosure;

[0026] Figure 4 A schematic diagram of a process for determining a first pooled image provided by an embodiment of the present disclosure;

[0027] Figure 5 A schematic diagram of a process for determining variance provided by an embodiment of the present disclosure;

[0028] Figure 6 A schematic diagram of determining a third pooled image provided by an embodiment of the present disclosure;

[0029] Figure 7 A schematic diagram of a process for determining covariance provided in an embodiment of the present disclosure;

[0030] Figure 8 A schematic diagram of a method for determining a second image provided by an embodiment of the present disclosure;

[0031] Fig. 9 A schematic diagram of a process for determining a target pixel merging weight provided in an embodiment of the present disclosure;

[0032] Fig.10 A schematic diagram of a filled average pooling process provided by an embodiment of the present disclosure;

[0033] Fig.11 A schematic diagram of a process for determining a second image provided by an embodiment of the present disclosure;

[0034] Fig.12 A schematic diagram of a lightweight downsampling method provided in an embodiment of the present disclosure;

[0035] Fig.13 A schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure; and

[0036] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0038] To facilitate understanding, the concepts involved in the embodiments of the present disclosure are explained below.

[0039] Electronic device: is a device with wireless transceiver function. Electronic devices can be deployed on land, including indoors or outdoors, handheld, wearable or vehicle-mounted. The electronic device can be a mobile phone, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) electronic device, an augmented reality (AR) electronic device, a wireless terminal in industrial control, a vehicle-mounted electronic device, a wireless terminal in self-driving, a wireless electronic device in remote medical care, a wireless electronic device in smart grid, a wireless electronic device in transportation safety, a wireless electronic device in smart city, a wireless electronic device in smart home, a wearable electronic device, etc. The electronic device involved in the embodiments of the present disclosure can also be called a terminal, a user equipment (UE), an access electronic device, a vehicle-mounted terminal, an industrial control terminal, a UE unit, a UE station, a mobile station, a mobile station, a remote station, a remote electronic device, a mobile device, a UE electronic device, a wireless communication device, a UE agent or a UE device, etc. The electronic device can also be fixed or mobile.

[0040] Next, combine Figure 1 , the application scenarios of the embodiments of the present disclosure are described.

[0041] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure. Figure 1 , including: an electronic device. The display page of the electronic device includes image A, and the resolution of image A is: 3840*2160. The electronic device can downsample image A to obtain image a, wherein the resolution of image a is 1902*1080. Figure 1 In the illustrated embodiment, the objects in image A and image a remain unchanged (e.g., if image A includes an apple, image B also includes an apple), but the number of pixels in image a is smaller than the number of pixels in image A. Thus, the image can be made to fit the size of the display area through downsampling.

[0042] It should be noted that Figure 1 The application scenarios of the embodiments of the present disclosure are merely illustrative and are not intended to limit the application scenarios of the embodiments of the present disclosure.

[0043] In the related art, the electronic device can obtain a downsampled image corresponding to the image based on the nearest neighbor interpolation method. For example, if the resolution of the original image is 800*800, and the downsampling ratio is 2 times, the electronic device can set 400*400 sampling points in the original image, and use the nearest pixel of each sampling point as the new pixel in the downsampled image, so that the downsampled image corresponding to the original image (resolution of 400*400) can be obtained. However, in the above method, the electronic device can retain the pixel closest to the sampling point, so that the 400*400 pixels in the original image will be lost in the downsampled image obtained by the electronic device, resulting in a large loss of image information in the downsampled image, and the clarity of the downsampled image is poor.

[0044] In order to solve the technical problems in the related art, the embodiment of the present disclosure provides an image processing method, in which an electronic device can obtain a first image, and perform downsampling processing on the first image to obtain a downsampled image, determine N image blocks in the downsampled image, and determine the average pixel value corresponding to multiple pixels in each image block, and generate a first pooled image based on the average pixel value corresponding to each image block, wherein the first pooled image includes N pixels, and the pixel values ​​of the N pixels correspond to the average pixel values ​​of the N image blocks one by one, and the electronic device can determine multiple pixel merging weights corresponding to the pixels in the downsampled image, and perform pixel merging processing on the downsampled image and the first pooled image based on the multiple pixel merging weights to obtain a second image. In this way, since each pixel in the first pooled image is determined by multiple pixels in the downsampled image, the electronic device can retain more image information of the first image in the second image obtained based on the first pooled image and the downsampled image, which can not only reduce the resolution of the second image, but also improve the clarity of the second image.

[0045] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0046] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure. Figure 2 , the method may include:

[0047] S201: Acquire a first image.

[0048] The execution subject of the embodiment of the present disclosure may be an electronic device, or an image processing device provided in the electronic device. The image processing device may be implemented based on software, or the image processing device may be implemented based on a combination of software and hardware. Optionally, the electronic device may be a device with on-end computing capability.

[0049] Among them, the first image may be an image to be downsampled. For example, the first image may be an image with a higher resolution. For example, in an application scenario, when the image acquired by the electronic device does not match the screen of the electronic device, the electronic device may downsample the image so that the downsampled image matches the screen of the electronic device, and therefore, the image may be the first image. For example, if the resolution of the image acquired by the electronic device is 800*800 and the screen resolution of the electronic device is 400*400, the electronic device may determine the image as the first image and downsample the image by 2 times.

[0050] It should be noted that the electronic device may receive the first image sent by other devices, the electronic device may also obtain the first image, and the electronic device may also obtain the first image based on any feasible implementation method, which is not limited in the embodiments of the present disclosure.

[0051] S202: Perform down-sampling processing on the first image to obtain a down-sampled image.

[0052] Among them, the downsampled image can be an image obtained by reducing the resolution of the first image. The electronic device can obtain the downsampled image based on the following feasible implementation method: obtain the downsampling ratio, and based on the downsampling ratio, perform average downsampling processing on the first image to obtain the downsampled image. For example, if the downsampling ratio is 2, the electronic device can perform average downsampling processing on the first image by 2 times, wherein the average downsampling can be linear sampling, etc., and the embodiments of the present disclosure are not limited to this. When the electronic device performs average downsampling processing on the first image, the electronic device can determine the average value of the pixel values ​​around the sampling point as the pixel value of the downsampled image. In this way, each pixel in the downsampled image can be integrated with the surrounding pixel information, so that more image information of the first image can be retained in the downsampled image, thereby improving the clarity of the image after downsampling.

[0053] S203 , performing average pooling processing on pixels in the downsampled image to obtain a first pooled image.

[0054] Wherein, each pixel in the first pooled image is obtained by merging multiple pixels in the downsampled image. For example, for any pixel in the first pooled image, the value of the pixel (RGB value) may be the average value of the values ​​of multiple pixels in the downsampled image.

[0055] Specifically, the electronic device may determine to obtain the first pooled image based on the following feasible implementation method: determine N image blocks in the downsampled image, determine the average pixel value corresponding to multiple pixels in each image block, and generate the first pooled image based on the average pixel value corresponding to each image block, where N is an integer greater than 1.

[0056] In this way, each pixel in the first pooled image can fuse pixels in multiple downsampled images, and because the pixels in the downsampled images can retain more pixel information of the first image, the first pooled image can also retain more pixel information in the first image.

[0057] Among them, the first pooled image includes N pixels, and the pixel values ​​of the N pixels correspond one-to-one to the average values ​​of the N image blocks. For example, if the electronic device determines 10 image blocks in the downsampled image, the first pooled image may include 10 pixels, and if the electronic device determines 100 image blocks in the downsampled image, the first pooled image may include 100 pixels. For example, if the image block includes pixel 1, pixel 2, and pixel 3, the pixel value of the pixel in the first pooled image corresponding to the image block is the average value of the pixel value of pixel 1, the pixel value of pixel 2, and the pixel value of pixel 3.

[0058] Optionally, the image block may include multiple pixels. For example, the image block may include 1 pixel, 4 pixels, 9 pixels, etc., which is not limited in the embodiments of the present disclosure. It should be noted that the shape of the image block in the embodiments of the present disclosure may be a rectangle, a square, or any other shape, which is not limited in the embodiments of the present disclosure.

[0059] Optionally, the electronic device determines N image blocks in the downsampled image, specifically by: determining a sliding window, and based on a preset sliding step length, controlling the sliding window to slide N times in the downsampled image to obtain N groups of pixels, and determining each group of pixels as 1 image block to obtain N image blocks. For example, if the preset sliding step length is M pixel units, the sliding window can slide to the right by M pixel units each time, and after each row of sliding is completed, the sliding window can slide downward by M pixel units until N image blocks are obtained.

[0060] It should be noted that the sliding window in the embodiment of the present disclosure may also slide based on any feasible implementation method (eg, the sliding window slides down by 1 pixel unit, the sliding window slides to the right by 1 pixel unit, etc.), and the embodiment of the present disclosure is not limited to this.

[0061] The size of the sliding window is the same as the size of the image block. For example, if the size of the image block is: 2 pixels long and 2 pixels wide, the size of the sliding window corresponding to the image block is: 2 pixels long and 2 pixels wide; if the size of the image block is: 4 pixels long and 4 pixels wide, the size of the sliding window corresponding to the image block is: 4 pixels long and 4 pixels wide.

[0062] Next, combine Figure 3 , the process of determining image blocks in a downsampled image is described.

[0063] Figure 3 A schematic diagram of a process for determining an image block provided by an embodiment of the present disclosure. Figure 3 , including a downsampled image and a sliding window. The downsampled image may include pixel A1, pixel A2, pixel A3, pixel A4, pixel A5, pixel A6, pixel A7, pixel A8 and pixel A9, and the size of the sliding window may be 2*2 (length and width are both 2 pixels). The downsampled image is slid four times based on the sliding window, wherein, starting from the upper left corner of the downsampled image, the sliding window may be moved horizontally by 1 pixel each time, and after the horizontal sliding is completed, the sliding window may be moved vertically by 1 pixel, and moves horizontally by 1 pixel each time.

[0064] See also Figure 3 , after the sliding window slides 4 times in the downsampled image, image block 1, image block 2, image block 3 and image block 4 can be obtained. Among them, image block 1 may include pixel A1, pixel A2, pixel A4 and pixel A5, image block 2 may include pixel A2, pixel A3, pixel A5 and pixel A6, image block 3 may include pixel A4, pixel A5, pixel A7 and pixel A8, and image block 4 may include pixel A5, pixel A6, pixel A8 and pixel A9. In this way, the electronic device can determine multiple image blocks in the downsampled image based on the sliding window, thereby improving the efficiency and accuracy of determining the image blocks.

[0065] Optionally, the electronic device may determine the size of the sliding window based on the downsampling ratio corresponding to the first image. For example, if the downsampling ratio corresponding to the first image is downsampling by 2 times, the size of the sliding window may be 2*2 (both the length and width are 2 pixels); if the downsampling ratio corresponding to the first image is downsampling by 3 times, the size of the sliding window may be 3*3 (both the length and width are 3 pixels).

[0066] It should be noted that the electronic device may also determine the size of the sliding window based on any other feasible implementation manner, and the embodiments of the present disclosure are not limited to this.

[0067] Next, combine Figure 4 , the process of determining the first pooled image corresponding to the down-sampled image is described.

[0068] Figure 4 A schematic diagram of a process for determining a first pooled image provided by an embodiment of the present disclosure. Figure 4 , including: a downsampled image. The downsampled image may include pixel A1, pixel A2, pixel A3, pixel A4, pixel A5, pixel A6, pixel A7, pixel A8 and pixel A9. If the size of the sliding window is 2*2 (both the length and the width are 2 pixels), the electronic device may slide 4 times in the downsampled image based on the sliding window to obtain 4 image blocks.

[0069] See also Figure 4 Image block 1 may include pixel A1, pixel A2, pixel A4 and pixel A5, image block 2 may include pixel A2, pixel A3, pixel A5 and pixel A6, image block 3 may include pixel A4, pixel A5, pixel A7 and pixel A8, and image block 4 may include pixel A5, pixel A6, pixel A8 and pixel A9.

[0070] See also Figure 4 The electronic device can determine that the average value of the 4 pixels in image block 1 is pixel B1, the average value of the 4 pixels in image block 2 is pixel B2, the average value of the 4 pixels in image block 3 is pixel B3, and the average value of the 4 pixels in image block 4 is pixel B4.

[0071] See also Figure 4 , the electronic device may determine a first pooled image based on pixel B1, pixel B2, pixel B3, and pixel B4. The pixel in the upper left corner of the first pooled image is pixel B1, the pixel in the upper right corner is pixel B2, the pixel in the lower left corner is pixel B3, and the pixel in the lower right corner is pixel B4.

[0072] It should be noted that the positions of multiple pixels in the first pooled image are associated with the positions of the image blocks in the downsampled image, which is not limited in the embodiments of the present disclosure.

[0073] It should be noted that the average value of the image block may be the mean value of the pixel values ​​of each pixel included in the image block. For example, the average value of the image block may be implemented based on the following formula:

[0074]

[0075] Where B is the average value of the image block, C is the number of pixels in the image block, A is the pixel value of the pixel, and i is the number of the pixel in the image block. Figure 4In the illustrated embodiment, if the pixel value of pixel A1 in image block 1 is RGB1, the pixel value of pixel A2 is RGB2, the pixel value of pixel A4 is RGB3, and the pixel value of pixel A5 is RGB4, then the pixel value of pixel B1 may be the average of RGB1, RGB2, RGB3, and RGB4, so that each pixel in the first pooled image may be the average of multiple pixels in the downsampled image, so that each pixel in the first pooled image may include information of multiple pixels, thereby improving the accuracy of the first pooled image.

[0076] Optionally, the electronic device can process the downsampled image based on the no-padding average pooling module to obtain a first pooled image corresponding to the downsampled image, wherein the no-padding average pooling module is used to traverse the mean of the pixels in each image block of the downsampled image. In this way, the electronic device can obtain the first pooled image based on the no-padding average pooling module.

[0077] It should be noted that the electronic device can process the first image through the non-filled average pooling module, and downsample the processed image to obtain a downsampled image. For example, the electronic device can first process the first image through the non-filled average pooling module, and each pixel in the obtained pooled image can be integrated with multiple pixels in the first image, and then the pooled image of the first image is sampled through the sampling module (the sampling module can downsample the image based on the sampling point) to obtain a downsampled image.

[0078] S204: Determine a plurality of pixel merging weights corresponding to pixels in the downsampled image.

[0079] The pixel merging weight is used to merge pixels in the downsampled image and the first pooled image. For example, when the electronic device merges the downsampled image and the first pooled image, the electronic device may merge multiple pixels in the downsampled image and multiple pixels in the first pooled image based on multiple pixel merging weights.

[0080] Optionally, the pixels in the downsampled image may correspond to at least one pixel merging weight. For example, the pixels in the downsampled image may correspond to 1 pixel merging weight, 2 pixel merging weights, 3 pixel merging weights, etc.

[0081] Among them, the electronic device can determine multiple pixel merging weights corresponding to pixels in the downsampled image based on the following feasible implementation methods: determine the variance associated with the pixels in the downsampled image, determine the covariance associated with the pixels in the downsampled image, and determine the multiple pixel merging weights corresponding to the pixels in the downsampled image based on the variance and covariance.

[0082] The variance associated with the pixels in the downsampled image may include the variance of multiple pixels in each image block in the downsampled image. For example, if the downsampled image may include 4 image blocks, and each image block may include 4 pixels, then the downsampled image may correspond to 4 variances, wherein the values ​​of the 4 pixels in each image block may obtain a variance. In this way, the pixel merging weights corresponding to the pixels of the downsampled image may be accurately determined based on the variance and covariance, thereby improving the accuracy of the pixel merging process and thereby improving the clarity of the second image.

[0083] Among them, the electronic device can determine the variance associated with pixels in the downsampled image based on the following feasible implementation method: square the pixel value of each pixel in the downsampled image to obtain the squared downsampled image, perform average pooling on the squared downsampled image to obtain a second pooled image, square the pixel value of each pixel in the first pooled image to obtain a squared first pooled image, and determine the variance associated with pixels in the downsampled image based on the squared first pooled image and the second pooled image.

[0084] The downsampled image after the square processing may be an image obtained by squaring the value of each pixel in the downsampled image. For example, if the downsampled image includes 2 pixels, one of which has a value of 2 and the other has a value of 4, the downsampled image after the square processing may also include 2 pixels, one of which has a value of 4 and the other has a value of 16.

[0085] Among them, any pixel in the second pooled image is obtained by merging multiple pixels in the downsampled image after the square processing. For example, the electronic device can process the downsampled image after the square processing based on the non-filled average pooling module to obtain the second pooled image. It should be noted that the method for determining the second pooled image is similar to the method for determining the first pooled image, and the embodiments of the present disclosure will not be repeated here.

[0086] Optionally, the electronic device may determine the difference between the pixel values ​​of the plurality of pixels in the second pooled image and the pixel values ​​of the plurality of pixels in the first pooled image after square processing as the variance associated with the pixels in the downsampled image.

[0087] Next, combine Figure 5 , a process for determining the variance associated with pixels in a downsampled image is described.

[0088] Figure 5 A schematic diagram of a process for determining variance provided by an embodiment of the present disclosure. Figure 5, including: a downsampled image. The downsampled image may include pixel A1, pixel A2, pixel A3, pixel A4, pixel A5, pixel A6, pixel A7, pixel A8 and pixel A9. If the size of the sliding window is 2*2 (length and width are both 2 pixels), the electronic device processes the downsampled image based on the unfilled average pooling module to obtain a first pooled image, wherein the first pooled image may include pixel B1, pixel B2, pixel B3 and pixel B4. The electronic device squares the values ​​of the pixels in the first pooled image to obtain a first pooled image after square processing. The first pooled image after square processing may include pixel C1, pixel C2, pixel C3 and pixel C4.

[0089] See also Figure 5 , square the pixel values ​​of the downsampled image to obtain the downsampled image after square processing. Among them, the downsampled image after square processing may include pixel D1, pixel D2, pixel D3, pixel D4, pixel D5, pixel D6, pixel D7, pixel D8 and pixel D9. For example, the value of pixel D1 is the square of the value of pixel A1, the value of pixel D2 is the square of the value of pixel A2, etc. The electronic device processes the downsampled image after square processing based on the non-filled average pooling module to obtain a second pooled image. Among them, the second pooled image may include pixel E1, pixel E2, pixel E3 and pixel E4.

[0090] See also Figure 5 , the electronic device can subtract the pixels in the second pooled image from the first pooled image after square processing, and then obtain a variance image. Among them, the variance image can include pixel F1, pixel F2, pixel F3 and pixel F4, and the value of pixel F1, the value of pixel F2, the value of pixel F3 and the value of pixel F4 can be 4 variances associated with the pixels in the downsampled image, which can improve the accuracy of determining the variance associated with the pixels of the downsampled image. It should be noted that after the unfilled average pooling module processes the image, fewer pixels will be lost at the edge of the image, but the loss of a small number of edge pixels has little effect on the effect of image display.

[0091] Optionally, the electronic device may determine the covariance associated with pixels in the downsampled image based on the following feasible implementation method: squaring the pixel value of each pixel in the first image, and performing average pooling on the squared first image to obtain a third image, downsampling the third image to obtain a downsampled image associated with the third image, performing average pooling on the downsampled image associated with the third image to obtain a third pooled image, and determining the covariance associated with pixels in the downsampled image based on the third pooled image and the squared first pooled image.

[0092] The third image may be an image obtained by squaring the first image and performing average pooling. For example, the electronic device may square the pixel value of each pixel in the first image, and process the squared first image based on the non-filled average pooling module to obtain a third image, and each pixel in the third image may be obtained by merging multiple pixels in the squared first image.

[0093] Optionally, the electronic device may determine the downsampled image associated with the third image based on any feasible implementation method, which is not limited in the embodiments of the present disclosure. It should be noted that since each pixel in the third image is obtained by merging multiple pixels in the first image, after the third image is downsampled, the pixel information loss is small, thereby improving the display effect of the downsampled image associated with the third image.

[0094] Optionally, the third pooled image may be an image obtained by performing average pooling processing on the downsampled image associated with the third image. For example, the electronic device may process the downsampled image associated with the third image based on the non-filled average pooling module, thereby obtaining the third pooled image, wherein any pixel in the third pooled image is obtained by merging multiple pixels in the downsampled image associated with the third image.

[0095] Optionally, the electronic device may determine the covariance associated with the pixels in the downsampled image based on the third pooled image and the first pooled image after the square process. For example, the electronic device may determine the difference between the value of the pixel of the third pooled image and the value of the pixel of the first pooled image after the square process as the covariance associated with the pixels in the downsampled image.

[0096] Next, combine Figure 6 , the process of determining the third pooled image is explained.

[0097] Figure 6 A schematic diagram of determining a third pooled image provided by an embodiment of the present disclosure. Figure 6 , including: a first image. The first image may include 25 pixels (only for illustrating the process of determining the third pooled image, not for limiting the pixels in the first image), and the pixel value of each pixel in the first image is squared, and the squared first image may include 25 pixels.

[0098] See also Figure 6, based on the unfilled average pooling module, the first image after square processing is processed to obtain a third image. Each pixel in the third image is obtained by merging the pixels in the first image after square processing. For example, pixel F1 in the third image is determined based on pixel E1, pixel E2, pixel E6 and pixel E7, and pixel F2 is determined based on pixel E2, pixel E3, pixel E7 and pixel E8.

[0099] See also Figure 6 , the sampling module processes the third image to obtain a downsampled image corresponding to the third image. For example, the sampling module can determine 9 sampling points in the third image, and the sampling module determines the pixel closest to the sampling point as the pixel in the downsampled image associated with the third image. For example, the first sampling point is closest to pixel F1, so the first pixel in the downsampled image associated with the third image is pixel F1; the second sampling point is closest to pixel F2, so the second pixel in the downsampled image associated with the third image is pixel F2.

[0100] See also Figure 6 , based on the unfilled average pooling module, the downsampled image associated with the third image is processed, wherein the length and width of the sliding window are both 2 pixels, thereby obtaining a third pooled image. The third pooled image may include pixel G1, pixel G2, pixel G3, and pixel G4. For example, pixel G1 is determined based on pixel F1, pixel F2, pixel F5, and pixel F6, and pixel G2 is determined based on pixel F2, pixel F3, pixel F6, and pixel F7.

[0101] In this way, the pixels in the third pooled image acquired by the electronic device may also include pixel information in the first image. For example, pixel G1 is obtained based on pixel F1, pixel F2, pixel F5 and pixel F6, and pixel F1, pixel F2, pixel F5 and pixel F6 are obtained based on pixel E1, pixel E2, pixel E3, pixel E6, pixel E7, pixel E8, pixel E11, pixel E11 and pixel E13. Therefore, the third pooled image can retain more pixel information in the first image, thereby improving the display effect of the image.

[0102] Next, combine Figure 7 , a process for determining the covariance associated with pixels in a downsampled image is described.

[0103] Figure 7 A schematic diagram of a process for determining covariance provided by an embodiment of the present disclosure. Figure 7, including: a third pooled image and a first pooled image after square processing. The third pooled image may include pixels G1, G2, G3, and G4, and the first pooled image after square processing may include pixels C1, C2, C3, and C4. It should be noted that the processing process of the third pooled image and the first pooled image after square processing can refer to the above embodiment, and the embodiment of the present disclosure will not be repeated here.

[0104] See also Figure 7 , the pixels of the third pooled image are subtracted from the pixels of the first pooled image after the square process, and a covariance image can be obtained. Among them, the pixel value of pixel H1, the pixel value of pixel H2, the pixel value of pixel H3 and the pixel value of pixel H4 included in the covariance image can be the covariance associated with the pixels of the downsampled image corresponding to the first image. For example, pixel G1 is subtracted from pixel C1 to obtain pixel H1, pixel G2 is subtracted from pixel C2 to obtain pixel H2, pixel G3 is subtracted from pixel C3 to obtain pixel H3, and pixel G4 is subtracted from pixel C4 to obtain pixel H4.

[0105] It should be noted that, based on the embodiments shown in the present disclosure, the variance and covariance in the embodiments of the present disclosure are determined based on the pixels in the image block, for example, Figure 5 In the illustrated embodiment, the variance indicated by pixel F1 is determined based on pixel C1 and pixel E1, and pixel C1 is determined based on an image block including pixel A1, pixel A2, pixel A4, and pixel A5, and pixel E1 is determined based on an image block including pixel D1, pixel D2, pixel D4, and pixel D5.

[0106] Optionally, after the electronic device determines the multiple variances and the multiple covariances corresponding to the pixels in the downsampled image, the ratio of the variance to the covariance may be determined as the pixel merging weight. Figure 5 and Figure 7 In the embodiment shown in FIG. 1 , the electronic device may determine the ratio of pixel F1 to pixel H1 as the pixel merging weight, determine the ratio of pixel F2 to pixel H2 as the pixel merging weight, and so on. Figure 5 and Figure 7 In the illustrated embodiment, four binning weights may be associated with the pixels of the downsampled image.

[0107] It should be noted that the electronic device may also determine the pixel merging weight based on any other feasible implementation method, for example, determining the ratio of covariance to variance as the pixel merging weight, etc., and the embodiments of the present disclosure are not limited to this.

[0108] S205 . Perform pixel merging processing on the downsampled image and the first pooled image based on multiple pixel merging weights to obtain a second image.

[0109] Optionally, the second image may be an image obtained by downsampling the first image. For example, if the resolution of the first image is 800*400, and the downsampling ratio is 2 times, the resolution of the second image may be 400*200. It should be noted that the resolution of the second image is similar to that of the downsampled image of the first image, but since the second image may include more pixel information in the first image, the clarity of the second image is greater than that of the downsampled image of the first image.

[0110] The electronic device can obtain the second image based on the following feasible implementation method: for any pixel in the downsampled image, obtain one or more target pixel merging weights determined based on the pixel from multiple pixel merging weights, determine multiple target pixel merging weights corresponding to multiple pixels in the downsampled image, and target pixels corresponding to multiple pixels in the downsampled image, to determine the second image.

[0111] Among them, pixel merging processing of the downsampled image and the first pooled image refers to merging each pixel in the downsampled image with the pixel in the first pooled image based on the pixel merging weight to obtain a new pixel, and the new pixel can be a pixel in the second image. For example, if the R value in RGB of pixel A in the downsampled image is 100, the pixel in the first pooled image corresponding to pixel A is pixel B (which can be determined based on position or based on any feasible implementation method, and the disclosed embodiment is not limited to this), the R value in RGB of pixel B is 60, and if the pixel merging weight corresponding to pixel A is 0.8, the R value of RGB of the new pixel in the second image is 92 (100*0.8+60*0.2). In this way, the pixels of the second image can include more pixel information of the first image, and the second image loses less pixel information, thereby improving the clarity of the second image and the display effect of the second image.

[0112] The disclosed embodiment provides an image processing method, in which an electronic device can obtain a first image, perform downsampling processing on the first image to obtain a downsampled image, determine N image blocks in the downsampled image, determine the average pixel value corresponding to multiple pixels in each image block, and generate the first pooled image based on the average pixel value corresponding to each image block. The electronic device can determine multiple pixel merging weights corresponding to the pixels in the downsampled image, and perform pixel merging processing on the downsampled image and the first pooled image based on the multiple pixel merging weights to obtain a second image. In this way, since more image information in the first image can be retained in the downsampled image, and more image information in the downsampled image can be retained in the first pooled image, after the electronic device performs pixel fusion on the first pooled image and the downsampled image, more image information in the first image can be retained in the second image, which can not only reduce the resolution of the second image, but also improve the clarity of the second image.

[0113] exist Figure 2 Based on the embodiment shown below, combined with Figure 8 , in the above-mentioned image processing method, a method for obtaining a second image by performing pixel merging processing on the downsampled image and the first pooled image based on multiple pixel merging weights is described.

[0114] Figure 8 A schematic diagram of a method for determining a second image provided by an embodiment of the present disclosure. Figure 8 The method flow may include:

[0115] S801. For any pixel in a downsampled image, obtain one or more target pixel merging weights determined based on the pixel from a plurality of pixel merging weights.

[0116] For any pixel in the downsampled image, the target pixel merging weight may be a pixel merging weight determined based on the pixel. For example, the electronic device may determine that the downsampled image includes 100 pixel merging weights, and for a pixel in the downsampled image, if 3 of the pixel merging weights are determined based on the pixel, the electronic device may determine the 3 pixel merging weights as the target pixel merging weights associated with the pixel.

[0117] Optionally, the electronic device may determine at least one image block associated with a pixel in the downsampled image, and further determine a pixel merging weight associated with at least one image block associated with the pixel as a target pixel merging weight corresponding to the pixel.

[0118] Next, combine Fig. 9 , the process of determining the target pixel merging weight corresponding to the pixel is explained.

[0119] Fig. 9 A schematic diagram of a process for determining a target pixel merging weight provided by an embodiment of the present disclosure. Fig. 9 In the embodiment shown, the pixel A2 in the sampled image is taken as an example to explain in detail the process of obtaining the target pixel merging weight corresponding to the pixel A2. Fig. 9 , including: the downsampled image may include pixel A1, pixel A2, pixel A3, pixel A4, pixel A5, pixel A6, pixel A7, pixel A8 and pixel A9. If the size of the sliding window is 2*2 (the length and width are both 2 pixels), the electronic device can determine that the downsampled image includes 4 image blocks.

[0120] See also Fig. 9 , for pixel A2 in the downsampled image, since pixel A2 is in image block 1 and image block 2, the electronic device can determine the pixel merging weight determined based on image block 1 and the pixel merging weight determined based on image block 2 as the target pixel merging weight corresponding to pixel A. Fig. 9 In the illustrated embodiment, image block 1, image block 2, image block 3 and image block 4 can determine 4 pixels, and the electronic device can determine 4 pixel merging weights based on the 4 pixels (please refer to the above embodiment, the embodiment of the present disclosure will not be repeated here). Therefore, the electronic device can determine the 2 pixel merging weights corresponding to image block 1 and image block 2 as the target pixel merging weight of pixel A2.

[0121] Below, Figure 5-Figure 6 Take the example to specifically explain the process of determining the target pixel merging weight.

[0122] For Figure 5 Pixel A1 in the illustrated embodiment, pixel A1, pixel A2, pixel A4 and pixel A5 can determine pixel B1 in the first pooled image, pixel B1 is squared to obtain pixel C1, and pixel C1 can determine the variance corresponding to pixel F1.

[0123] Figure 6 In the illustrated embodiment, the third pooled image includes the covariance corresponding to pixel G1, the covariance corresponding to pixel G2, the covariance corresponding to pixel G3, and the covariance corresponding to pixel G4. A pixel merging weight can be determined based on pixel F1 and pixel G1, and therefore, the pixel merging weight can be the target pixel merging weight corresponding to pixel A1.

[0124] It should be noted that in Figure 6 In the embodiment shown, if the sliding window is 3*3 pixels, the sliding window slides 1 pixel unit each time. Figure 6 In the illustrated embodiment, the pixel D7 corresponds to four image blocks, and therefore, the pixel D7 can correspond to four target pixel merging weights at most.

[0125] S802: Determine a target pixel corresponding to each pixel in the downsampled image in the first pooled image.

[0126] Optionally, the target pixel may be a pixel in the first pooled image that is associated with a pixel in the downsampled image. For example, for any pixel in the downsampled image, the target pixel corresponding to the pixel may be a pixel associated with the pixel position in the first pooled image. For example, the target pixel corresponding to the pixel in the 1st row and 1st column of the downsampled image may be the pixel in the 1st row and 1st column of the first pooled image; the target pixel corresponding to the pixel in the 10th row and 5th column of the downsampled image may be the pixel in the 10th row and 5th column of the first pooled image.

[0127] For example, in Figure 4 In the illustrated embodiment, the target pixel corresponding to pixel A1 in the downsampled image may be pixel B1, the target pixel corresponding to pixel A2 in the downsampled image may be pixel B2, the target pixel corresponding to pixel A3 in the downsampled image may be pixel B3, and the target pixel corresponding to pixel A4 in the downsampled image may be pixel B4.

[0128] It should be noted that after the downsampled image is processed based on the no-fill pooling average pooling module, the first pooled image obtained will lose some edge pixels, for example, Figure 4 In the embodiment shown, the downsampled image loses 5 pixels compared to the first pooled image. In this way, some pixels in the downsampled image do not have target pixels. However, in actual applications, the number of pixels in the image is large, so after the non-filled average pooling, the display effect of the image is less affected.

[0129] S803: Determine a second image based on the downsampled image, a plurality of target pixel merging weights corresponding to a plurality of pixels in the downsampled image, and a plurality of target pixels corresponding to a plurality of pixels in the downsampled image.

[0130] Optionally, for any pixel in the downsampled image, the electronic device can determine one pixel in the second image based on the pixel, the target pixel merging weight associated with the pixel and the target pixel, so that the second image can be obtained by traversing each pixel in the downsampled image.

[0131] Optionally, the electronic device may determine the second image based on the following formula:

[0132]

[0133] Where N is the number of image blocks corresponding to the pixel, It can be the mean value of the pixels in the kth image block, It can be the variance associated with the pixels in the kth image block, It can be the covariance associated with the pixels in the kth image block, and L is the value of the target pixel.

[0134] In this way, based on the above formula, the electronic device can determine the pixel of the second image corresponding to each pixel in the downsampled image, and thus obtain the second image.

[0135] Next, combine Fig.10 , the processing process of filling the average pooling module involved in the embodiment of the present disclosure is explained.

[0136] Fig.10 A schematic diagram of a padded average pooling process provided by an embodiment of the present disclosure. Fig.10 , Fig.10 The image in may include pixel A1, pixel A2, pixel A3, pixel A4, pixel A5, pixel A6, pixel A7, pixel A8 and pixel A9. If the length and width of the sliding window are 3 pixels * 3 pixels, and the sliding window moves 1 pixel each time, the image block is 3 * 3. Therefore, in order to keep the number of pixels unchanged after the image is average pooled, a layer of pixels can be filled around the periphery of the image.

[0137] See also Fig.10 , after a circle of pixels is filled around the periphery of the image, the number of pixels in the image changes from 3*3 to 5*5, and after average pooling processing is performed on the image, a new image can be obtained, wherein the new image may include pixel B1, pixel B2, pixel B3, pixel B4, pixel B5, pixel B6, pixel B7, pixel B8, and pixel B9. For example, pixel B1 is obtained based on 0, pixel A1, pixel A2, pixel A1, pixel A1, pixel A2, pixel A4, pixel A4, and pixel A5 (the pixel value of pixel B1 is obtained based on the average value of each pixel value).

[0138] In this way, after the image is processed based on the filled average pooling module, the number of pixels of the image will not change (after the image is processed by the unfilled average pooling module, the number of pixels of the image will decrease), thereby improving the display effect of the image.

[0139] It should be noted that when filling pixels around an image, the electronic device may fill pixels with all 0s, or may fill based on the pixel values ​​of edge pixels, which is not limited in the embodiments of the present disclosure. Fig.10 In the illustrated embodiment, when filling pixels on the upper side of pixel A1, pixel A2, and pixel A3, the filled pixels may be pixel A1, pixel A2, and pixel A3. If the second circle of pixels is filled, pixel A4, pixel A5, and pixel A6 may be filled. In this way, the display effect of the image may be improved after average pooling processing.

[0140] Next, combine Fig.11 , the process of determining the second image is described.

[0141] Fig.11 A schematic diagram of a process for determining a second image provided by an embodiment of the present disclosure. Fig.11 , including: a first image H. The first image H is processed based on a non-filling average pooling module, and the first image H processed by the non-filling average pooling module is sampled by a sampling module to obtain an image L1, the image L1 is processed based on the non-filling average pooling module to obtain an image M, and the pixels in the image M are squared.

[0142] See also Fig.11 , square the pixels in the first image H, and process the squared first image H based on the no-fill average pooling module, and sample the output of the no-fill pooling module based on the sampling module to obtain image L2. Image L2 is processed based on the no-fill average pooling module. Based on the squared image M and the image L2 processed by the no-fill average pooling module, a covariance image (pixel values ​​in the image are related to covariance) can be determined.

[0143] See also Fig.11 , square the pixels of image L1 to obtain a square image of image L1, process the square image of image L1 based on the no-filling average pooling module, and determine the variance image (the pixel value in the image is related to the variance) based on the square image of image L1 processed by the no-filling average pooling module and the squared image M, and determine the pixel merging weight R (the pixel value in the image is related to the pixel merging weight) based on the variance image and the covariance image.

[0144] See also Fig.11 , image M is processed based on the filling average pooling module to obtain image m, image T is processed based on the filling average pooling module to obtain image t, wherein image T is determined based on image M and pixel merging weight R, pixel merging weight R is processed based on the filling average pooling module to obtain image r, all-1 arrays Q of the same size as M are processed based on the filling average pooling module to obtain matrix q, and a second image D is obtained based on q, image r, image t and image m.

[0145] In this way, after downsampling, the second image can still retain more image information in the first image, and the pixel merging weight corresponding to each pixel in the second image is related to the pixel, and the pixel merging weight has a high degree of flexibility, which can improve the flexibility of pixel merging and improve the clarity of the second image.

[0146] Optional, in Fig.11 Based on the embodiment shown in the figure, when determining the second image, a lightweight downsampling method may also be included. Fig.12 , the lightweight downsampling method is explained.

[0147] Fig.12 A schematic diagram of a lightweight downsampling method provided by an embodiment of the present disclosure. Fig.12 , including: a first image H. The first image H is bilinearly downsampled (linear downsampling) to obtain an image L1, the image L1 is processed based on a padded average pooling model to obtain an image M, and the image M is squared.

[0148] See also Fig.12 , square the pixels in the first image H, and process the squared first image H based on bilinear downsampling to obtain image L2, and process image L2 based on the padding average pooling module. Based on the padding average pooling module, the image L2 is processed and the squared image M is processed to obtain a covariance image.

[0149] See also Fig.12 , square the pixels of image L1 to obtain a square image of image L1, process the square image of image L1 based on the padding average pooling module to obtain a square image of image L1 processed by the padding average pooling module. Based on the square image of image L1 processed by the padding average pooling module and the squared image M, a variance image is obtained.

[0150] See also Fig.12 , the pixel merging weight R can be determined based on the covariance image and the variance image, and the second image D is obtained based on the image L1, the image M and the pixel merging weight R. In this way, the pixels in the second image are determined based on the image L1 and the image M, and the second image can retain more pixel information of the first image, thereby improving the clarity of the second image and improving the display effect of the second image.

[0151] It should be noted that in the lightweight solution, each pixel in the downsampled image corresponds to one target pixel merging weight. The electronic device can determine the target pixel merging weight based on any feasible implementation method. For example, it can arbitrarily determine a target pixel merging weight from multiple pixel merging weights, or determine the pixel merging weight determined by the image block with the pixel located in the center as the target pixel merging weight. The embodiments of the present disclosure are not limited to this.

[0152] The disclosed embodiment provides a method for determining a second image, for any pixel in a downsampled image, one or more target pixel merging weights determined based on the pixel are obtained from a plurality of pixel merging weights, the target pixel corresponding to each pixel in the downsampled image in the first pooled image is determined, and the second image is determined based on the downsampled image, the plurality of target pixel merging weights corresponding to the plurality of pixels in the downsampled image, and the plurality of target pixels corresponding to the plurality of pixels in the downsampled image. In this way, since each pixel in the second image is determined based on the downsampled image and the image after the downsampled image is filled with average pooling processing, more pixel information in the first image can be retained in the second image, and since the target pixel merging weight corresponding to each pixel is associated with the pixel, the plurality of target pixel merging weights corresponding to the downsampled image are more flexible, thereby improving the flexibility of image downsampling and improving the image clarity after image downsampling.

[0153] Fig.13 This is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure. Fig.13 The image processing device 130 includes an acquisition module 131, a sampling module 132, a pooling module 133, a determination module 134 and a processing module 135, wherein:

[0154] The acquisition module 131 is used to acquire a first image;

[0155] The sampling module 132 is used to perform down-sampling processing on the first image to obtain a down-sampled image;

[0156] The pooling module 133 is used to perform average pooling processing on the pixels in the downsampled image to obtain a first pooled image;

[0157] The determination module 134 is used to determine a plurality of pixel merging weights corresponding to pixels in the downsampled image;

[0158] The processing module 135 is used to perform pixel merging processing on the down-sampled image and the first pooled image based on the multiple pixel merging weights to obtain a second image.

[0159] According to one or more embodiments of the present disclosure, the pooling module 133 is specifically used for:

[0160] Determine N image blocks in the downsampled image, where N is an integer greater than 1;

[0161] Determine an average pixel value corresponding to a plurality of pixels in each image block;

[0162] Based on the average pixel value corresponding to each image block, the first pooled image is generated, wherein the first pooled image includes N pixels, and the pixel values ​​of the N pixels correspond one-to-one to the average pixel values ​​of the N image blocks.

[0163] According to one or more embodiments of the present disclosure, the pooling module 133 is specifically used for:

[0164] Determine a sliding window, wherein the size of the sliding window is the same as the size of the image block;

[0165] Based on a preset sliding step size, controlling the sliding window to slide N times in the downsampled image to obtain N groups of pixels;

[0166] Each group of pixels is determined as an image block, and N image blocks are obtained.

[0167] According to one or more embodiments of the present disclosure, the determination module 134 is specifically configured to:

[0168] determining a variance associated with pixels in the downsampled image;

[0169] determining a covariance associated with pixels in the downsampled image;

[0170] Based on the variance and the covariance, a plurality of pixel merging weights corresponding to pixels in the downsampled image are determined.

[0171] According to one or more embodiments of the present disclosure, the determination module 134 is specifically configured to:

[0172] Squaring the pixel value of each pixel in the downsampled image to obtain a downsampled image after the square processing;

[0173] Performing average pooling processing on the down-sampled image after the square processing to obtain a second pooled image;

[0174] Squaring the pixel value of each pixel in the first pooled image to obtain a squared first pooled image;

[0175] Based on the squared first pooled image and the second pooled image, a variance associated with pixels in the downsampled image is determined.

[0176] According to one or more embodiments of the present disclosure, the determination module 134 is specifically configured to:

[0177] Squaring the pixel value of each pixel in the first image, and performing average pooling on the squared first image to obtain a third image;

[0178] Performing down-sampling processing on the third image to obtain a down-sampled image associated with the third image;

[0179] Performing average pooling processing on the down-sampled image associated with the third image to obtain a third pooled image;

[0180] Based on the third pooled image and the squared first pooled image, a covariance associated with pixels in the downsampled image is determined.

[0181] According to one or more embodiments of the present disclosure, the processing module 135 is specifically used for:

[0182] For any pixel in the downsampled image, obtaining one or more target pixel merging weights determined based on the pixel from the plurality of pixel merging weights;

[0183] Determine a target pixel corresponding to each pixel in the downsampled image in the first pooled image;

[0184] The second image is determined based on the down-sampled image, a plurality of target pixel merging weights corresponding to a plurality of pixels in the down-sampled image, and a plurality of target pixels corresponding to a plurality of pixels in the down-sampled image.

[0185] The image processing device provided in the embodiment of the present disclosure may be used to execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0186] Fig.14 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Fig.14 , which shows a structural schematic diagram of an electronic device 1400 suitable for implementing an embodiment of the present disclosure. The electronic device 1400 can be any device with end-to-end computing capabilities. Fig.14 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0187] like Fig.14As shown, the electronic device 1400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1402 or a program loaded from a storage device 1408 to a random access memory (RAM) 1403. Various programs and data required for the operation of the electronic device 1400 are also stored in the RAM 1403. The processing device 1401, the ROM 1402, and the RAM 1403 are connected to each other via a bus 1404. An input / output (I / O) interface 1405 is also connected to the bus 1404.

[0188] Typically, the following devices may be connected to the I / O interface 1405: input devices 1406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 1407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1408 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1409. The communication device 1409 may allow the electronic device 1400 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.14 The electronic device 1400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0189] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1409, or installed from the storage device 1408, or installed from the ROM 1402. When the computer program is executed by the processing device 1401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0190] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0191] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0192] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0193] An embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the image processing method as described in the first aspect and various possible aspects of the first aspect are implemented.

[0194] An embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and various image processing methods that may be involved in the first aspect.

[0195] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0196] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0197] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0198] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0199] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0200] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0201] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0202] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0203] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message. As an optional but non-limiting implementation method, in response to receiving an active request from the user, the method of sending a prompt message to the user can be, for example, a pop-up window, in which the prompt message can be presented in text. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device. It can be understood that the above notification and the process of obtaining user authorization are only illustrative and do not constitute a limitation on the implementation method of the present disclosure. Other methods that meet relevant laws and regulations can also be applied to the implementation method of the present disclosure.

[0204] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of the relevant laws and regulations. The data may include information, parameters and messages, such as flow switching indication information.

[0205] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0206] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0207] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An image processing method, characterized in that: include: acquiring a first image; Performing downsampling processing on the first image to obtain a downsampled image; Performing average pooling processing on pixels in the downsampled image to obtain a first pooled image; Determining a plurality of pixel merging weights corresponding to pixels in the downsampled image; Based on the multiple pixel merging weights, pixel merging processing is performed on the downsampled image and the first pooled image to obtain a second image.

2. The method according to claim 1, characterized in that The performing average pooling processing on the pixels in the downsampled image to obtain a first pooled image includes: Determine N image blocks in the downsampled image, where N is an integer greater than 1; Determine an average pixel value corresponding to a plurality of pixels in each image block; Based on the average pixel value corresponding to each image block, the first pooled image is generated, wherein the first pooled image includes N pixels, and the pixel values ​​of the N pixels correspond one-to-one to the average pixel values ​​of the N image blocks.

3. The method according to claim 2, characterized in that The determining N image blocks in the downsampled image comprises: Determine a sliding window, wherein the size of the sliding window is the same as the size of the image block; Based on a preset sliding step size, controlling the sliding window to slide N times in the downsampled image to obtain N groups of pixels; Each group of pixels is determined as an image block, and N image blocks are obtained.

4. The method according to any one of claims 1 to 3, characterized in that: The determining a plurality of pixel merging weights corresponding to pixels in the downsampled image comprises: determining a variance associated with pixels in the downsampled image; determining a covariance associated with pixels in the downsampled image; Based on the variance and the covariance, a plurality of pixel merging weights corresponding to pixels in the downsampled image are determined.

5. The method according to claim 4, characterized in that Determining the variance associated with pixels in the downsampled image comprises: Squaring the pixel value of each pixel in the downsampled image to obtain a downsampled image after the square processing; Performing average pooling processing on the down-sampled image after the square processing to obtain a second pooled image; Squaring the pixel value of each pixel in the first pooled image to obtain a squared first pooled image; Based on the squared first pooled image and the second pooled image, a variance associated with pixels in the downsampled image is determined.

6. The method according to claim 5, characterized in that Determining the covariance associated with pixels in the downsampled image comprises: Squaring the pixel value of each pixel in the first image, and performing average pooling on the squared first image to obtain a third image; Performing down-sampling processing on the third image to obtain a down-sampled image associated with the third image; Performing average pooling processing on the down-sampled image associated with the third image to obtain a third pooled image; Based on the third pooled image and the squared first pooled image, a covariance associated with pixels in the downsampled image is determined.

7. The method according to any one of claims 1 to 3, characterized in that: The step of performing pixel merging processing on the downsampled image and the first pooled image based on the multiple pixel merging weights to obtain a second image includes: For any pixel in the downsampled image, obtaining one or more target pixel merging weights determined based on the pixel from the plurality of pixel merging weights; Determine a target pixel corresponding to each pixel in the downsampled image in the first pooled image; The second image is determined based on the down-sampled image, a plurality of target pixel merging weights corresponding to a plurality of pixels in the down-sampled image, and a plurality of target pixels corresponding to a plurality of pixels in the down-sampled image.

8. An image processing device, characterized in that: It includes an acquisition module, a sampling module, a pooling module, a determination module and a processing module, among which: The acquisition module is used to acquire a first image; The sampling module is used to perform down-sampling processing on the first image to obtain a down-sampled image; The pooling module is used to perform average pooling processing on the pixels in the downsampled image to obtain a first pooled image; The determination module is used to determine a plurality of pixel merging weights corresponding to pixels in the downsampled image; The processing module is used to perform pixel merging processing on the down-sampled image and the first pooled image based on the multiple pixel merging weights to obtain a second image.

9. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the image processing method according to any one of claims 1 to 7 is implemented.

Images