Image processing method, device, image sensor and electronic equipment

Abstract

The present application provides an image processing method, device, image sensor and electronic device, which obtains a first image; merges pixels in the first image to obtain a first pixel-merged image after the pixel merging, wherein the first image is a current image corresponding to a preset continuous frame, and the pixel merging methods of each image corresponding to the preset continuous frame are different, which can reduce the amount of data transmitted by the sensor, increase the data transmission rate, and improve the image quality after fusion.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to an image processing method, device, image sensor and electronic equipment. Background Art

[0002] Today's terminal devices require image sensors to support multiple resolution modes to meet the requirements of digital zoom, preview, video, photo, high-speed photography and other usage scenarios. For example, the current common high-speed photography mode requires the V2H2 pixel merging (Binning) function on the sensor to reduce the Bayer pattern resolution to one-fourth of the original output, so that a higher frame rate can be achieved. For example, the sensor's native Bayer pattern mode is 12 mega pixels, and a Bayer pattern output with a resolution of 3 mega pixels can be achieved through analog or digital Binning. Although a lower resolution can achieve a higher frame rate, the Binning operation will result in reduced image clarity, and the uneven spatial distribution of pixels in the image after Binning will cause pseudo-color and jagged edges on the image, affecting image quality. Summary of the invention

[0003] In response to the above-mentioned problems, the embodiments of the present application provide an image processing method, device, image sensor and electronic device, which can reduce the data transmission volume of the sensor, increase the image data transmission rate, and improve the image quality through fusion.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to an image sensor, comprising:

[0005] acquiring a first image; and

[0006] Pixels in the first image are merged to obtain a first pixel-merged image after the pixel merger, wherein the first image is a current image corresponding to a preset continuous frame, and pixel merging methods of each image corresponding to the preset continuous frame are different.

[0007] In some embodiments, the method further comprises:

[0008] All pixel-merged images corresponding to the preset continuous frames are fused based on the first pixel-merged image.

[0009] In some embodiments, the step of performing pixel merging on pixels in the first image to obtain a first pixel-merged image after pixel merging includes:

[0010] Determine pixel merging modes respectively corresponding to respective second images before the first image, wherein the second images are images before the first image in preset continuous frames;

[0011] Determine a pixel merging mode corresponding to the first image based on the pixel merging modes respectively corresponding to the second images;

[0012] Pixel merging is performed on the first image based on a pixel merging method corresponding to the first image to obtain a first pixel merging image after pixel merging.

[0013] In some embodiments, the position of the data window selected when performing pixel merging on the pixels of the first image based on the pixel merging method corresponding to the first image is a first position, and the position of the data window selected when performing pixel merging on the corresponding images based on the pixel merging method corresponding to each second image is a second position, the first position and the second position are different, and there is a fixed offset in the horizontal direction, vertical direction and / or diagonal direction.

[0014] In some embodiments, the sum of the pixel data amount of the data window corresponding to the first position and the pixel data amount of the data windows corresponding to each second position is equal to the pixel data amount corresponding to the first image.

[0015] In some embodiments, the preset continuous frames are 2 to 4 frames, and the pixel merging method corresponding to the preset continuous frames forms a cycle.

[0016] In some embodiments, the preset continuous frames are 4 frames.

[0017] In some embodiments, the pixel merging is to merge four adjacent same-color pixels into one pixel, and the merged same-color pixels in the first image and each of the second images are offset by one pixel unit in the horizontal direction, vertical direction and / or diagonal direction respectively.

[0018] In some embodiments, the pixel merging is performed by equal weighting or weighted method.

[0019] In some embodiments, the pixel merging in a weighted manner is weighted according to the distance between the position of the pixel after merging and the position of each pixel before merging.

[0020] In a second aspect, the present application provides an image processing method, applied to an image sensor, comprising:

[0021] acquiring a first image;

[0022] Perform pixel merging on pixels in the first image to obtain a first pixel-merged image after pixel merging;

[0023] The first pixel-merged image is sent to an electronic device, wherein, upon acquiring the first pixel-merged image, the electronic device fuses all pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image to obtain a fused image, wherein the pixel-merged images corresponding to the preset continuous frames are obtained by pixel-merging corresponding images based on different pixel-merging methods.

[0024] In a third aspect, an embodiment of the present application provides an image processing method, which is applied to an electronic device, including:

[0025] Acquire a first pixel-merged image after pixel merging sent by the image sensor, wherein the first pixel-merged image is obtained by the image sensor performing pixel merging on the acquired first image;

[0026] Based on the first pixel-merged image, all pixel-merged images corresponding to the preset continuous frames are fused, wherein the pixel-merged images corresponding to the preset continuous frames are obtained by pixel-merging corresponding images based on different pixel-merging methods.

[0027] In some embodiments, the method further comprises:

[0028] determining whether the preset continuous frame is reached based on the first pixel-merged image and a second pixel-merged image before the first pixel-merged image;

[0029] When the preset continuous frame is not reached, the first pixel-merged image and the second pixel-merged image are merged.

[0030] In a fourth aspect, an embodiment of the present application provides an image processing chip, which is applied to an image sensor, including:

[0031] A first acquisition module, used for acquiring a first image;

[0032] The first merging module is used to merge pixels in the first image to obtain a first pixel-merged image after the pixel merging; wherein the first image is a current image corresponding to a preset continuous frame, and the pixel merging methods of each image corresponding to the preset continuous frame are different.

[0033] In a fifth aspect, an embodiment of the present application provides an image processing device, which is applied to an image sensor, including:

[0034] A first acquisition module, used for acquiring a first image;

[0035] A second merging module, used for performing pixel merging on pixels in the first image to obtain a first pixel-merged image after pixel merging;

[0036] A sending module is used to send the first pixel-merged image to an electronic device, wherein the electronic device, when acquiring the first pixel-merged image, fuses all pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image to obtain a fused image, wherein the pixel-merged images corresponding to the preset continuous frames are obtained by pixel-merging corresponding images based on different pixel-merging methods.

[0037] In a sixth aspect, an embodiment of the present application provides an image processing device, which is applied to an electronic device, including:

[0038] A second acquisition module is used to acquire a first pixel-merged image after pixel merging sent by the image sensor, wherein the first pixel-merged image is obtained by the image sensor performing pixel merging on the acquired first image;

[0039] The first fusion module is used to fuse all pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image, wherein the pixel-merged images corresponding to the preset continuous frames are obtained by pixel-merging corresponding images based on different pixel-merging methods.

[0040] In a seventh aspect, an embodiment of the present application provides an image sensor, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the method provided in the first aspect is implemented when the processor executes the computer program.

[0041] In an eighth aspect, an embodiment of the present application provides an image sensor, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the method provided in the second aspect is implemented when the processor executes the computer program.

[0042] In a ninth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method provided in the third aspect when executing the computer program.

[0043] In a tenth aspect, an embodiment of the present application provides an image processing system, comprising: the image sensor provided in the eighth aspect and the electronic device provided in the ninth aspect.

[0044] In the eleventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method provided in the first aspect, the second aspect or the third aspect.

[0045] In the twelfth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it is at least used to implement a method as described in any one of the first aspect, the second aspect, or the third aspect.

[0046] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0047] The image processing method provided in the embodiment of the present application obtains a first image; and performs pixel merging on pixels in the first image to obtain a first pixel-merged image after the pixel merging, wherein the first image is a current image corresponding to a preset continuous frame, and the pixel merging methods of each image corresponding to the preset continuous frame are different, which can reduce the amount of data transmitted by the sensor, increase the data transmission rate of the sensor, and improve the image quality after fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 A schematic diagram of a Binning solution provided in the related art;

[0050] Figure 2 A schematic diagram of a flow chart of an image processing method provided by the present application;

[0051] Figure 3 A schematic diagram of a pixel merging process provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of pixel merging based on weighting of spatial distance provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application;

[0055] Figure 7 A specific example of an image processing method provided in an embodiment of the present application;

[0056] Figure 8A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application;

[0057] Fig. 9 A schematic diagram of the structure of an image processing chip provided in an embodiment of the present application;

[0058] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0060] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0061] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0062] As used in the specification of this application and the appended claims, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrases "if it is determined" or "if it is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce detected" or "in response to detecting" depending on the context.

[0063] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0064] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0065] Before introducing the embodiments of the present application, a brief introduction to the related technologies is given. Figure 1 A schematic diagram of a Binning solution provided in the related art is shown in FIG. Figure 1 As shown in the figure, after binning, the image resolution is one quarter of the Bayer pattern data before binning. The mathematical operation of the binning process is:

[0066] For channel B, Q0,a=(P0,a+P0,c+P2,a+P2,c) / 4;

[0067] For a Gb channel, Q0,b=(P0,b+P0,d+P2,b+P2,d) / 4;

[0068] For the Gr channel, Q1,a=(P1,a+P1,c+P3,a+P3,c) / 4;

[0069] For the R channel, Q1,b=(P1,b+P1,d+P3,b+P3,d) / 4;

[0070] Although the above Binning solution can achieve a higher frame rate, the Binning operation will reduce the image clarity, and the image edge will have pseudo color and jagged edges due to the uneven pixel spatial distribution of the image after Binning, which will affect the image quality. Therefore, it is of great significance to find a solution with small readout and transmission data volume and high image quality in applications such as digital zoom, preview, video, photo, and high-speed photography.

[0071] This application will also use a quarter Binning scheme as an example later, but those skilled in the art will understand that the Binning scheme of the present application can also be applied to Binning situations of other resolutions, such as reducing the resolution to 1 / 9 before Binning, that is, binning 9 adjacent pixels of the same color into 1 pixel. More specifically, the 9 adjacent pixels of the same color refer to a certain pixel as the center, plus 4 identical pixels above, below, left, and right, and 4 identical pixels in the diagonal direction. Of course, in another case, the Binning scheme of the present application is also applicable to reducing the resolution to 1 / 5 before Binning, that is, binning 5 adjacent pixels of the same color into 1 pixel. More specifically, the 5 adjacent pixels of the same color refer to a certain pixel as the center, plus 4 identical pixels above, below, left, and right. Therefore, the present application is not limited to a Binning scheme with a resolution reduced to a quarter.

[0072] Based on the technical problems of the related art, an embodiment of the present application provides an image processing method that can be applied to an image sensor.

[0073] The present application embodiment provides an image processing method. Figure 2 The present application provides a flowchart of an image processing method, such as Figure 2 As shown, the method includes:

[0074] Step S101, acquiring a first image.

[0075] In the embodiment of the present application, the image sensor can capture images in real time to obtain a first image, and the first image can be any image captured by the sensor.

[0076] Step S102, pixel merging is performed on pixels in the first image to obtain a first pixel-merged image after pixel merging, wherein the first image is a current image corresponding to a preset continuous frame, and pixel merging methods of each image corresponding to the preset continuous frame are different.

[0077] In the embodiment of the present application, pixel merging is an image processing technology that reduces the resolution of an image by merging multiple pixels into one pixel. During the merging process, the value of the new pixel can be calculated according to a preset algorithm, and the preset algorithm may include: an equal weight or a weighted algorithm, etc. The first pixel-merged image refers to an image obtained after pixel merging the first image, and the resolution of the first pixel-merged image is lower than that of the original image.

[0078] In the embodiment of the present application, each pixel of the first image can be traversed, and they can be merged into new pixels according to a preset pixel merging method. The pixel merging method of the first image can be determined according to the previous image frame.

[0079] The preset continuous frames can be configured, and the configuration of the preset continuous frames can be determined according to the number of frames of the image to be fused. The preset continuous frames can be at least 2 frames, and the number of preset continuous frames can be configured. Exemplarily, the preset continuous frames can be configured as 2 frames, 3 frames or 4 frames as a cycle period. It should be noted that the number of preset continuous frames cannot be set too much. When the number of frames is set too much, the image effect after fusion in a motion scene may be distorted due to the large interval between the current frame and the frontmost frame, and too few frames cannot cover more image data to solve the problem of edge pseudo-color or jagged edges. Therefore, the number of preset continuous frames of the present invention is optimally configured as 4 frames, that is, 4 frames as a cycle period.

[0080] In an embodiment of the present application, the preset continuous frames correspond to a corresponding number of images. For example, the preset continuous frames are 4 frames, which corresponds to 4 frames of images. If there are 4 frames of images, there are 4 pixel merging methods when performing pixel merging on the 4 frames of images.

[0081] The method provided in an embodiment of the present application obtains a first image; and performs pixel merging on pixels in the first image to obtain a first pixel-merged image after the pixel merging, wherein the first image is a current image corresponding to a preset continuous frame, and the pixel merging methods of each image corresponding to the preset continuous frame are different, which can reduce the amount of data transmitted by the sensor to increase the data transmission rate of the sensor.

[0082] In some embodiments, after step S102, the method further includes:

[0083] Step S103: fusing pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image.

[0084] In the embodiment of the present application, after the first pixel-merged image is generated, the image sensor can perform fusion processing on all pixel-merged images corresponding to the preset continuous frames. When performing the fusion processing, a preset fusion algorithm can be used, and the fusion algorithm can be: weighted average, multi-resolution analysis, etc. By fusing the preset continuous frame pixel-merged images, the image clarity, contrast, color saturation and other characteristics can be improved.

[0085] The method provided in the embodiment of the present application acquires a first image; performs pixel merging on pixels in the first image to obtain a first pixel-merged image after pixel merging; and fuses all pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image, thereby reducing the amount of data transmitted by the sensor, improving the data transmission rate of the sensor, and enabling the obtained image to have higher quality through fusion.

[0086] In some embodiments, step S102 may be implemented by the following steps:

[0087] Step S1021, determining pixel merging modes corresponding to respective second images before the first image, wherein the second image is an image before the first image in a preset continuous frame.

[0088] In an embodiment of the present application, the pixel merging methods corresponding to the images of preset consecutive frames can be pre-set, and the pixel merging methods corresponding to each frame of the image are different. Since the first image is the current image and the second image is the processed image, the pixel merging methods corresponding to the second images can be determined through historical information.

[0089] Step S1022: determining a pixel combining method corresponding to the first image based on pixel combining methods corresponding to each second image.

[0090] In the embodiment of the present application, the pixel merging modes corresponding to the preset continuous frames can form a cycle, and cycle based on the cycle, the pixel merging modes corresponding to the images of the preset continuous frames can adopt a cycle strategy to select the pixel merging mode, and the image sensor can maintain a list of pixel merging modes, which includes a cycle list of multiple pixel merging modes, and selects an unused merging mode in sequence each time a new image is processed. Exemplarily, taking 4 frames as an example, the pixel merging modes corresponding to the 4 frames form a cycle, and the cycle includes: pixel merging mode 1, pixel merging mode 2, pixel merging mode 3 and pixel merging mode 4, and so on. If the pixel merging modes corresponding to the first 3 frames are determined, the pixel merging mode of the fourth frame can be determined. For example, there are 3 frames of second images before the first image, and the pixel merging modes corresponding to the 3 frames of the second images are respectively: pixel merging mode 1, pixel merging mode 2, and pixel merging mode 3, then the pixel merging mode corresponding to the first image is: pixel merging mode 4, and if the pixel merging modes corresponding to the 3 frames of the second image are respectively: pixel merging mode 2, pixel merging mode 3, and pixel merging mode 4, then the pixel merging mode corresponding to the first image is: pixel merging mode 1, and so on.

[0091] For example, taking the preset continuous frames as 2 frames, the cycle includes: pixel merging mode 1, pixel merging mode 2. If the pixel merging mode corresponding to the second image is pixel merging mode 1, then the pixel merging mode corresponding to the first image is: pixel merging mode 2. If the pixel merging mode corresponding to the second image is pixel merging mode 2, then the pixel merging mode corresponding to the first image is: pixel merging mode 1.

[0092] In the embodiment of the present application, the pixel merging method may include: an algorithm used for pixel merging and position information of a data window on an image when performing pixel merging. The algorithm used for pixel merging may include: an equal weight algorithm or a weighted algorithm.

[0093] In the embodiment of the present application, the difference in pixel merging methods may be that the algorithm used is the same but the position information is different, or that the algorithm is different and the position is different. The embodiment of the present application does not limit this.

[0094] Step S1023 , performing pixel merging on the first image based on the pixel merging method corresponding to the first image, to obtain a first pixel merging image after pixel merging.

[0095] In the embodiment of the present application, after the pixel merging method of the first image is determined, the pixels of the first image can be merged in this way.

[0096] Continuing with the above example, if pixel merging method 4 is selected, pixel merging method 4 needs to be used to perform pixel merging on the first pixel image.

[0097] The method provided in the embodiment of the present application dynamically selects different pixel merging methods. Due to the different pixel merging methods, more sampling points can be made and the sampling points are more uniform, so that when an electronic device fuses the pixel-merged image, the quality of the fused image can be improved.

[0098] In some embodiments, the position of the data window selected when performing pixel merging on pixels of the first image based on the pixel merging method corresponding to the first image is a first position, and the position of the data window selected when performing pixel merging on the corresponding images based on the pixel merging methods corresponding to each second image is a second position. The first position and the second position are different, and there are fixed offsets in the horizontal direction, vertical direction and / or diagonal direction, respectively.

[0099] In an embodiment of the present application, a data window refers to a local area used to operate pixels during image processing. The data window is usually a rectangle containing a certain number of pixels. The first position is the specific position of the data window on the image when the first image is pixel-merged, and the position is determined according to the pixel-merging method corresponding to the first image. The second position refers to the specific position of the data window on the corresponding image when the pixels of each second image are merged. Since each second image corresponds to a different pixel-merging method, the data window position may also be different.

[0100] In the embodiment of the present application, the first position and the second position are different mainly in position, and there is a fixed offset in the horizontal direction, vertical direction and / or diagonal direction respectively. In some embodiments, the difference may also include size and shape.

[0101] In the embodiment of the present application, when processing the first image, the image sensor will search for the corresponding pixel merging method and determine the first position of the data window according to the method. Then, the image sensor will apply the pixel merging algorithm at this position to obtain the merged pixel value. For each second image, the image sensor will also perform similar steps, but the position of the data window (second position) will be dynamically adjusted according to the merging method of the image.

[0102] In some embodiments, pixel merging is merging four adjacent same-color pixels into one pixel, and the same-color pixels merged in the first image and each of the second images are offset by one pixel unit in the horizontal direction, vertical direction, and / or diagonal direction. Taking the Bayer array as an example, the one pixel unit includes four pixels in a 2x 2 format, and the four pixels can be a combination of three colors of red, green, and blue, or other color combinations, and the four pixels can also include split pixels.

[0103] In an embodiment of the present application, by shifting one unit to the right in the horizontal direction, the merged pixels can be staggered in the horizontal direction. Similarly, by shifting one pixel unit downward in the vertical direction, the merged pixels can be staggered in the vertical direction. By shifting to the lower right in the diagonal direction, the merged image can also be staggered in the diagonal direction.

[0104] In some embodiments, the sum of the pixel data amount of the data window corresponding to the first position and the pixel data amount of the data windows corresponding to each second position is equal to the pixel data amount corresponding to the first image. Exemplarily, the pixel data amount of the data window corresponding to the first position is A, and the number of pixels of the data windows corresponding to each second position is respectively recorded as B1, B2, ..., Bn (where n is the number of second images), then A (the pixel data amount of the data window at the first position) + B1 + B2 + ... + Bn (the sum of the pixel data amount of each second position data window) = C (the total pixel data amount of the first image).

[0105] Figure 3 A schematic diagram of a pixel merging process provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, since the first image is usually an image taken within a very short period of time, the first image may be basically unchanged from the previous second image. Therefore, the original image corresponding to the image after the four frames are pixel-merged is represented by one image. After the pixel merging, the four consecutive frames A, B, C, and D respectively use different Binning methods, mainly because the positions of the merged data windows are different. When the four frames are pixel-merged, the corresponding data windows are merged according to the spatial positions to obtain the same amount of data as before Binning.

[0106] The method provided in the embodiment of the present application can make spatial sampling uniform by making the sum of the pixel data amount of the data window corresponding to the first position and the pixel data amount of the data windows corresponding to each second position equal to the pixel data amount corresponding to the first image, so that the image fused by the electronic device does not have the problem of uneven spatial sampling, and the signal-to-noise ratio and clarity of the fused image are higher.

[0107] However, in other embodiments, the sum of the pixel data amount of the data window corresponding to the first position and the pixel number of the data window corresponding to each second position may be less than or greater than the total pixel data amount of the first image. For example, in the embodiment of the above-mentioned quarter Binning scheme, only the pixel-merged image of the A and D frames is fused. At this time, the sum of the pixel data amount of the data window corresponding to the first position and the pixel number of the data window corresponding to each second position is only half of the total pixel data amount of the first image. However, compared with each using a fixed single Binning method, the signal-to-noise ratio and clarity of the image obtained by fusing different Binning methods will also be improved. Similarly, the image quality of the image obtained by fusing only the pixel-merged images of A, B or A, C or A, B, C will also be improved to a certain extent. However, the present application gives priority to the fusion of the A and D frames or the A, B, C, D frames. This choice has better symmetry and can better improve the problem caused by the uneven resolution space, so the image effect obtained is better. In addition, in other embodiments, an offset may be performed on the left side of frame A to obtain more pixel-merged images for fusion, so that the sum of the pixel data amount of the data window corresponding to the first position and the pixel number of the data windows corresponding to each second position may be greater than the total pixel data amount of the first image. Despite the various embodiments described above, the present application adopts a quarter Binning scheme and uses 4 frames as a cycle, which can take into account both the transmission rate and the image performance, and can achieve the best effect.

[0108] In some embodiments, pixel merging is performed in an equal-weighted or weighted manner.

[0109] In an embodiment of the present application, when pixels are merged, the same weight is given to all pixels participating in the merging in an equal manner, that is, regardless of their original values, positional relationships or other characteristic differences, all pixel values ​​participating in the merging are averaged to obtain a new pixel value.

[0110] In some embodiments, the weighted approach includes weighting according to the distance between the position of the pixel after merging and the position of each pixel before merging. Figure 4 A schematic diagram of a pixel merging method for weighting according to the distance between the position of the pixel after merging and the position of each pixel before merging provided in an embodiment of the present application, such as Figure 4As shown, still taking the four frames A, B, C, and D as an example, taking A as an example, weighting is performed according to the distance between the pixel position after merging and the position of each pixel before merging, and the mathematical operation of pixel merging can be expressed as:

[0111] For channel B, A0,a=(9*P0,a+3*P0,c+3*P2,a+P2,c) / 16;

[0112] For a Gb channel, A0,b = (3*P0,b + 9*P0,d + P2,b + 3*P2,d) / 16;

[0113] For the Gr channel, A1,a=(3*P1,a+P1,c+9*P3,a+3*P3,c) / 16;

[0114] For the R channel, A1,b=(P1,b+3*P1,d+3*P3,b+9*P3,d) / 16;

[0115] Among them, 16 is the size of the selected data window, and the coefficient corresponding to each pixel is the relative weight of the distance between the position of the merged pixel and the position of each pixel before the merge. When weighting is performed according to the distance between the position of the merged pixel and the position of each pixel before the merge, the closer pixel will be given a larger weight, while the farther pixel will be given a smaller weight. The merged pixel value will reflect more information about the surrounding close-range pixels. When each image is processed using the corresponding pixel merging method, the position of the data window corresponding to each pixel merging method is different, that is, there is an offset between the position corresponding to A and the position corresponding to B, and there is also a fixed offset between the position corresponding to B and the position corresponding to C, and there is also an offset between the position corresponding to C and the position corresponding to D. Through the offset, when the electronic device uses 4 frames of data together for fusion, the fused image is equivalent to adding sampling points, so the image can have higher clarity.

[0116] Based on the above embodiments, the present application further provides an image processing method, which is applied to an image sensor. Figure 5 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, including:

[0117] Step S501, acquiring a first image;

[0118] Step S502, performing pixel merging on pixels in the first image to obtain a first pixel-merged image after pixel merging;

[0119] Step S503, sending the first pixel-merged image to an electronic device, wherein the electronic device, upon acquiring the first pixel-merged image, fuses all pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image to obtain a fused image, wherein the pixel-merged images corresponding to the preset continuous frames are obtained by pixel-merging corresponding images based on different pixel-merging methods.

[0120] In the embodiments of the present application, the electronic device may be any device capable of processing images, such as a smart phone, a tablet computer, a computer, a server, etc.

[0121] In an embodiment of the present application, after the first pixel-merged image is generated, the image sensor can send the first pixel-merged image to the electronic device for processing. The image sensor can communicate with the electronic device via a wired or wireless connection (such as USB, Wi-Fi, etc.) to send the first pixel-merged image to the electronic device. After acquiring the first pixel-merged image, the electronic device can fuse all pixel-merged images corresponding to the preset continuous frames.

[0122] In an embodiment of the present application, pixels in a first image are merged to obtain a first pixel-merged image after the pixel merging, including: determining pixel merging modes respectively corresponding to respective second images before the first image, wherein the second image is an image before the first image in a preset continuous frame; determining a pixel merging mode corresponding to the first image based on the pixel merging modes respectively corresponding to the respective second images; and merging the first image based on the pixel merging mode corresponding to the first image to obtain a first pixel-merged image after the pixel merging.

[0123] In some embodiments, the position of the data window selected when performing pixel merging on pixels of the first image based on the pixel merging method corresponding to the first image is a first position, and the position of the data window selected when performing pixel merging on the corresponding images based on the pixel merging methods corresponding to each second image is a second position. The first position and the second position are different, and there are fixed offsets in the horizontal direction, vertical direction and / or diagonal direction, respectively.

[0124] In some embodiments, the sum of the pixel data amount of the data window corresponding to the first position and the pixel data amount of the data windows corresponding to each second position is equal to the pixel data amount corresponding to the first image.

[0125] In some embodiments, the preset continuous frames are 2 to 4 frames, and the pixel merging methods corresponding to the preset continuous frames form a cycle.

[0126] In the embodiment of the present application, the preset continuous frames are 4 frames, and the pixel merging method corresponding to the 4 frames forms a cycle.

[0127] Based on the above-mentioned embodiments, the present application provides an image processing method that can be applied to electronic devices. The electronic devices may include: mobile phones, tablet computers, wearable devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), and the present application does not impose any restrictions on the specific types of electronic devices.

[0128] Figure 6 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, including:

[0129] Step S601, acquiring a first pixel-merged image after pixel merging sent by an image sensor, wherein the first pixel-merged image is obtained by performing pixel merging on a first image acquired by the image sensor;

[0130] Step S602, based on the first pixel merged image, all pixel merged images corresponding to the preset continuous frames are merged, and based on the first pixel merged image, all pixel merged images corresponding to the preset continuous frames are merged, wherein the pixel merged images corresponding to the preset continuous frames are obtained by pixel merging the corresponding images based on different pixel merging methods.

[0131] In an embodiment of the present application, if the number of frames corresponding to the first pixel-merged image and the pixel-merged image before the first pixel-merged image reaches the number of preset continuous frames, the first pixel-merged image and the pixel-merged image corresponding to the number of frames minus 1 of the preset continuous frame are fused. Exemplarily, taking the first pixel-merged image as the current frame and the preset continuous frames as 4 frames, the pixel-merged image corresponding to the current frame and the 3 pixel-merged images before the current frame are fused to obtain a fused image. In some embodiments, the pixel-merged image after the first pixel-merged image can also be fused with the first pixel-merged image. For example, also taking 4 frames as an example, if there are 3 pixel-merged images after the first pixel-merged image, the first pixel-merged image can be fused with the last 3 pixel-merged images.

[0132] In the embodiment of the present application, the fusion algorithm may include: a weighted average algorithm, a maximum value selection algorithm, a minimum value selection algorithm, etc.

[0133] In the embodiment of the present application, the pixel-merged image may be pre-processed before fusion, and a final image may be obtained by fusion. The pre-processing may include: denoising.

[0134] The method provided in the embodiment of the present application can generate a higher quality fused image by acquiring the first pixel-merged image after pixel merging sent by the image sensor, and fusing all pixel-merged images corresponding to preset continuous frames based on the image.

[0135] In some embodiments, before step S602, the method further includes:

[0136] Step S6021: Determine whether the preset number of continuous frames is reached based on the first pixel-merged image and a second pixel-merged image before the first pixel-merged image.

[0137] In the embodiment of the present application, the number of frames can be determined by the first pixel-merged image and the second pixel-merged image, and whether a preset number of continuous frames is reached can be determined by the number of frames.

[0138] Step S6022: When the preset continuous frame is not reached, the first pixel-merged image and the second pixel-merged image are merged.

[0139] In the embodiment of the present application, if the number of the preset continuous frames is not reached, fusion is also performed directly. This situation is usually at the beginning of the sensor acquisition stage.

[0140] Exemplarily, the preset continuous frames are still taken as 4 frames. If there is a second pixel-merged image before the first pixel-merged image, that is, the number of frames is 2 frames, and these 2 frames do not reach 4 frames, the first pixel-merged image and the second pixel-merged image are fused.

[0141] Based on the above embodiments, an embodiment of the present application provides an image processing method, which is applied to an image processing system. The image processing system includes: an image sensor and an electronic device. Figure 7 A flowchart of an image processing method provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, including:

[0142] Step S701, the image sensor acquires a first image;

[0143] Step S702, the image sensor performs pixel merging on pixels in the first image to obtain a first pixel-merged image after pixel merging;

[0144] Step S703: the image sensor sends a first pixel-merged image to an electronic device.

[0145] Step S704: the electronic device obtains a first pixel-merged image after pixel merging sent by the image sensor.

[0146] Step S705: The electronic device fuses all pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image.

[0147] For example, Figure 8 A specific example of an image processing method provided in an embodiment of the present application, the electronic device may be a platform, such as Figure 8 As shown, four consecutive frames of images A, B, C, and D use different Binning methods respectively. After the platform obtains the current frame and the previous three frames of data, it can complement and fuse the four frames of data. Image A, image B, image C, and image D can be fused to obtain a fused image E. After obtaining image A, images B, image C, image D and image A can be fused to output a fused image F.

[0148] The method provided in the embodiment of the present application acquires a first image through an image sensor; performs pixel merging on pixels in the first image to obtain a first pixel merged image after pixel merging; and sends the first pixel merged image to an electronic device. When the electronic device acquires the first pixel merged image, the electronic device fuses all pixel merged images corresponding to preset continuous frames based on the first pixel merged image. The pixel merged images corresponding to the preset continuous frames are obtained by performing pixel merging of corresponding images based on different pixel merging methods, which can reduce the amount of data transmitted by the sensor and can obtain higher image quality.

[0149] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0150] According to the aforementioned embodiments, the embodiments of the present application provide an image processing chip, and the modules included in the chip, as well as the units included in the modules, can be implemented by a processor in a computer device; of course, they can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU, Central Processing Unit), a microprocessor (MPU, Microprocessor Unit), a digital signal processor (DSP, Digital Signal Processing) or a field programmable gate array (FPGA, Field Programmable Gate Array), etc.

[0151] Fig. 9 A schematic diagram of the structure of an image processing chip provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the image processing chip 900 includes:

[0152] A first acquisition module 901, configured to acquire a first image;

[0153] The first merging module 902 is used to merge the pixels in the first image to obtain a first pixel-merged image after the pixel merging, wherein the first image is a current image corresponding to a preset continuous frame, and the pixel merging methods of each image corresponding to the preset continuous frame are different.

[0154] In some embodiments, the image processing chip 900 includes:

[0155] A fusion module is used to fuse all pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image.

[0156] In some embodiments, the first merging module includes:

[0157] A first determining unit, configured to determine pixel merging modes respectively corresponding to respective second images before the first image, wherein the second images are images before the first image in a preset continuous frame;

[0158] A second determining unit, configured to determine a pixel combining mode corresponding to the first image based on pixel combining modes respectively corresponding to each of the second images;

[0159] A merging unit is used to perform pixel merging on the first image based on the pixel merging method corresponding to the first image to obtain a first pixel merging image after pixel merging.

[0160] In some embodiments, the position of the data window selected when performing pixel merging on the pixels of the first image based on the pixel merging method corresponding to the first image is a first position, and the position of the data window selected when performing pixel merging on the corresponding images based on the pixel merging method corresponding to each second image is a second position, the first position and the second position are different, and there is a fixed offset in the horizontal direction, vertical direction and / or diagonal direction, respectively.

[0161] In some embodiments, the sum of the pixel data amount of the data window corresponding to the first position and the pixel data amount of the data windows corresponding to each second position is equal to the pixel data amount corresponding to the first image.

[0162] In some embodiments, the preset continuous frames are 2 to 4 frames, and the pixel merging methods corresponding to the preset continuous frames form a cycle.

[0163] In some embodiments, the preset consecutive frames are 4 frames.

[0164] In some embodiments, the pixel merging is to merge four adjacent same-color pixels into one pixel, and the merged same-color pixels in the first image and each of the second images are offset by one pixel unit in the horizontal direction, vertical direction and / or diagonal direction respectively.

[0165] In some embodiments, the weighted approach includes: weighting according to the distance between the position of the pixel after merging and the position of each pixel before merging.

[0166] Based on the above embodiments, the present application further provides an image processing device, which is applied to an image sensor, including:

[0167] A second acquisition module, used for acquiring a first image;

[0168] A second merging module, used for performing pixel merging on pixels in the first image to obtain a first pixel-merged image after pixel merging;

[0169] A sending module is used to send the first pixel-merged image to an electronic device, wherein the electronic device, when acquiring the first pixel-merged image, fuses all pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image to obtain a fused image, wherein the pixel-merged images corresponding to the preset continuous frames are obtained by pixel-merging corresponding images based on different pixel-merging methods.

[0170] Based on the above embodiments, the present application further provides an image processing device, which is applied to an electronic device, including:

[0171] A second acquisition module is used to acquire a first pixel-merged image after pixel merging sent by the image sensor, wherein the first pixel-merged image is obtained by the image sensor performing pixel merging on the acquired first image;

[0172] The first fusion module is used to fuse all pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image.

[0173] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0174] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0175] Fig.10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.10 As shown, the electronic device 3 of this embodiment may include: at least one processor 30 ( Fig.10 Only one processor 30 is shown in the figure), a memory 31, and a computer program 32 stored in the memory 31 and executable on at least one processor 30. When the processor 30 executes the computer program 32, the steps in any of the above-mentioned method embodiments are implemented; or, when the processor 30 executes the computer program 32, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0176] Exemplarily, the computer program 32 may be divided into one or more modules / units, one or more modules / units are stored in the memory 31, and are executed by the processor 30 to complete the present application. One or more modules / units may be a series of computer program 32 instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 32 in the electronic device 3.

[0177] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program 32. When the computer program 32 is executed by the processor 30, the steps in the above-mentioned method embodiments can be implemented.

[0178] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0179] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. According to this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program 32, and the computer program 32 can be stored in a computer-readable storage medium. When the computer program 32 is executed by the processor 30, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program 32 includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the terminal, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0180] Based on the foregoing embodiments, an embodiment of the present application provides an image sensor, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the foregoing methods when executing the computer program.

[0181] An embodiment of the present application provides a computer program product. When the computer program product runs on an image sensor, the image sensor can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0182] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0183] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0184] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0185] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0186] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

[0187] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0188] The user personal information processed by the applicant will vary depending on the specific product / service scenario, and shall be based on the specific scenario in which the user uses the product / service, which may involve the user's account information, device information, driving information, vehicle information or other related information. The applicant will treat the user's personal information and its processing with a high degree of diligence.

[0189] The Applicant attaches great importance to the security of user personal information and has adopted reasonable and feasible security protection measures that comply with industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.

Claims

1. A method for processing an image, characterized in that: Applications in image sensors, including: acquiring a first image; and Pixels in the first image are merged to obtain a first pixel-merged image after the pixel merger, wherein the first image is a current image corresponding to a preset continuous frame, and pixel merging methods of each image corresponding to the preset continuous frame are different.

2. The method according to claim 1, characterized in that Also includes: All pixel-merged images corresponding to the preset continuous frames are fused based on the first pixel-merged image.

3. The method according to claim 1, characterized in that The step of performing pixel merging on pixels in the first image to obtain a first pixel-merged image after pixel merging includes: Determine pixel merging modes corresponding to respective second images before the first image, wherein the second images are images before the first image in the preset continuous frames; Determine a pixel merging mode corresponding to the first image based on the pixel merging modes respectively corresponding to the second images; Pixel merging is performed on the first image based on a pixel merging method corresponding to the first image to obtain a first pixel merging image after pixel merging.

4. The method according to claim 3, characterized in that The position of the data window selected when performing pixel merging on the pixels of the first image based on the pixel merging method corresponding to the first image is the first position, and the position of the data window selected when performing pixel merging on the corresponding images based on the pixel merging method corresponding to each second image is the second position, the first position and the second position are different, and there is a fixed offset in the horizontal direction, vertical direction and / or diagonal direction respectively.

5. The method according to claim 4, characterized in that The sum of the pixel data amount of the data window corresponding to the first position and the pixel data amount of the data windows corresponding to each second position is equal to the pixel data amount corresponding to the first image.

6. The method according to claim 1, characterized in that The preset continuous frames are 2 to 4 frames, and the pixel merging mode corresponding to the preset continuous frames is one cycle.

7. The method according to claim 6, characterized in that The preset continuous frames are 4 frames.

8. The method according to claim 3, characterized in that The pixel merging is to merge four adjacent same-color pixels into one pixel, and the merged same-color pixels in the first image and each of the second images are offset by one pixel unit in the horizontal direction, vertical direction and / or diagonal direction respectively.

9. The method according to claim 1, characterized in that: The pixel merging is performed by using an equal weight or a weighted method.

10. The method according to claim 1, characterized in that The weighted method includes: weighting according to the distance between the position of the pixel after merging and the position of each pixel before merging.

11. A method for processing an image, characterized in that: Applications in image sensors, including: acquiring a first image; Perform pixel merging on pixels in the first image to obtain a first pixel-merged image after pixel merging; The first pixel-merged image is sent to an electronic device, wherein, upon acquiring the first pixel-merged image, the electronic device fuses all pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image to obtain a fused image, wherein all pixel-merged images corresponding to the preset continuous frames are obtained by pixel-merging corresponding images based on different pixel-merging methods.

12. A method for processing an image, characterized in that: Used in electronic equipment, including: Acquire a first pixel-merged image after pixel merging sent by the image sensor, wherein the first pixel-merged image is obtained by the image sensor performing pixel merging on the acquired first image; Based on the first pixel-merged image, all pixel-merged images corresponding to the preset continuous frames are fused, wherein all pixel-merged images corresponding to the preset continuous frames are obtained by pixel-merging corresponding images based on different pixel-merging methods.

13. The method according to claim 12, characterized in that The method further comprises: determining whether all pixel-merged images corresponding to the preset continuous frames are reached based on the first pixel-merged image and a second pixel-merged image before the first pixel-merged image; When all pixel-merged images corresponding to the preset continuous frames are not reached, the first pixel-merged image and the second pixel-merged image are merged.

14. An image processing chip, characterized in that: Applications in image sensors, including: A first acquisition module, used to acquire a first image; The first merging module is used to merge pixels in the first image to obtain a first pixel-merged image after the pixel merging; wherein the first image is a current image corresponding to a preset continuous frame, and the pixel merging methods of each image corresponding to the preset continuous frame are different.

15. An image processing device, characterized in that: Applications in image sensors, including: A second acquisition module, used for acquiring a first image; A second merging module, used for performing pixel merging on pixels in the first image to obtain a first pixel-merged image after pixel merging; A sending module is used to send the first pixel-merged image to an electronic device, wherein the electronic device, when acquiring the first pixel-merged image, fuses all pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image to obtain a fused image, wherein all pixel-merged images corresponding to the preset continuous frames are obtained by pixel-merging corresponding images based on different pixel-merging methods.

16. An image processing device, characterized in that: Used in electronic equipment, including: A second acquisition module is used to acquire a first pixel-merged image after pixel merging sent by the image sensor, wherein the first pixel-merged image is obtained by the image sensor performing pixel merging on the acquired first image; The first fusion module is used to fuse all pixel-merged images corresponding to preset continuous frames based on the first pixel-merged image, wherein all pixel-merged images corresponding to the preset continuous frames are obtained by pixel-merging corresponding images based on different pixel-merging methods.

17. An image sensor, characterized in that: include: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 10 or claim 11 when executing the computer program.

18. An electronic device, characterized in that: include: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 12 to 13 when executing the computer program.

19. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

Images