Image processing method, terminal and computer storage medium

Abstract

The present application relates to a kind of image processing method, terminal and computer storage medium, there is at least one block to be repaired in image, the block to be repaired includes multiple pixels, image processing method includes: in image, the block to be repaired is matched, obtains multiple matching blocks;According to the pixel mean of the block to be repaired calculated from matching block;According to the pixel mean of the block to be repaired and the texture feature of its surrounding normal pixel, the pixel value of each pixel in the block to be repaired is calculated.Through the above mode, the present application can repair the defective quality in image, improves imaging quality.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method, a terminal, and a computer storage medium. Background Technology

[0002] The PDAF (Pixel Detection Auto Focus) technology widely used in camera modules achieves phase detection focusing by embedding PD pixels in the sensor pixel array. Its principle is to reserve some masked pixels on the image sensor specifically for phase detection. The focus offset is determined by the distance between pixels and their changes, thus achieving accurate focusing. While PD pixels enable PDAF functionality, they do not provide normal image information. Since pixels used for PDAF are typically physically masked, they need to be corrected before the sensor outputs the image to restore them to normal pixels. CIS (CMOS Image Sensor) typically uses a multi-functional CFA pattern (Color Filter Array). Repairing PD pixels on this pattern with PDAF functionality differs significantly from repairing PD pixels on a standard bayer pattern, requiring the design of specific repair algorithms. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide an image processing method, a terminal, and a computer storage medium that can repair image defects and improve image quality.

[0004] To achieve the above objectives, a first aspect of the present invention provides an image processing method. In one embodiment, the image contains at least one block to be repaired, the block comprising multiple pixels. The method includes:

[0005] The block to be repaired is matched in the image to obtain multiple matching blocks;

[0006] The average pixel value of the block to be repaired is calculated based on the matching block;

[0007] The pixel value of each pixel in the block to be repaired is calculated based on the average pixel value of the block to be repaired.

[0008] In one implementation, the block to be repaired is matched in the image to obtain a plurality of matching blocks, including:

[0009] The block to be repaired is combined with its surrounding pixels to obtain a reference block;

[0010] The image region in the image that has similar brightness and / or texture features to the reference block is identified as the matching block.

[0011] One implementation method for obtaining the reference block includes:

[0012] The size of the block to be repaired is defined as m*n. A 3m*3n pixel area is selected with the block to be repaired as the center to obtain the reference block.

[0013] As one implementation method, the pixel mean value of the block to be repaired is calculated based on the matching block, including:

[0014] Calculate the weight value of each matching block relative to the reference block;

[0015] The average pixel value of the block to be repaired is calculated based on the weight value and the average pixel value of the matching block.

[0016] As one implementation method, the method for calculating the weight value of each matching block relative to the reference block is as follows:

[0017] Extract the brightness feature vectors of the matching block and the reference block. and ;

[0018] Extract the detail feature vectors of the matching block and the reference block. and ;

[0019] The brightness difference between the matching block and the reference block is calculated according to formula (1):

[0020] (1)

[0021] In the formula, i represents the sequence number of the matching block. This represents the brightness feature vector of the i-th block. This represents the brightness feature vector of the reference block. This represents the brightness difference between the matching block and the reference block;

[0022] The detail difference between the matching block and the reference block is calculated according to formula (2):

[0023] (2)

[0024] In the formula, This represents the detailed feature vector of the i-th block. This represents the detail feature vector of the reference block. The detailed difference between the matching block and the reference block;

[0025] The brightness difference and the detail difference are summed according to formula (3):

[0026] (3)

[0027] In the formula, w is negatively correlated with the gain, 0≤w≤1. This represents the sum of the brightness difference and the detail difference;

[0028] The weight value of the matching block relative to the block to be repaired is calculated according to formulas (4) and (5):

[0029] (4)

[0030] (5)

[0031] In the formula, The threshold is based on Obtain This represents the median weight of the i-th matching block. This represents the weight value of the i-th matching block.

[0032] As one implementation method, select The maximum value in is used as the threshold.

[0033] As one implementation method, when When the calculation result is negative, the intermediate value of the weight The value is 0.

[0034] As one implementation method, the brightness feature vectors of the matching block and the reference block are extracted. and ; and, extracting the detailed feature vectors of the matching block and the reference block. and ;include:

[0035] For each matching block, the mean value of each pixel unit in the pixel region corresponding to the surrounding pixels of the reference block is calculated to obtain the luminance feature vector. ;

[0036] For the reference block, the mean value of each pixel unit in the surrounding pixels is calculated to obtain the luminance feature vector. ;

[0037] For each matching block, the detail feature representation value of each pixel unit adjacent to the block to be repaired in the pixel region corresponding to the surrounding pixels of the reference block is calculated to obtain the detail feature vector. ;

[0038] For the reference block, the detail feature representation value of each pixel unit immediately adjacent to the block to be repaired in the surrounding pixels is calculated to obtain the detail feature vector. ;

[0039] For a 2x2 pixel unit, the calculation method for the detail feature representation value includes:

[0040] ;

[0041] ;

[0042] ;

[0043] In the formula, S1, S2, and S3 represent different components of the detail feature representation value, and G1, G2, G3, and G4 represent the pixel values ​​of the corresponding pixels in the pixel unit.

[0044] In one implementation, the average pixel value of the block to be repaired is calculated based on the weight value and the average pixel value of the matching block, using the following formula:

[0045] (6)

[0046] In the formula, The value represents the average pixel value of the block to be repaired, and i represents the sequence number of the matching block, ranging from 0 to n, where n is a positive integer. This represents the average pixel value of the pixels in the region corresponding to the block to be repaired in the i-th matching block.

[0047] In one implementation, obtaining the pixel value of each pixel within the block to be repaired based on the average pixel value of the block to be repaired and the texture features of the surrounding normal pixels includes:

[0048] The pixel value of each pixel in the block to be repaired is calculated based on the average pixel value of the block to be repaired and the local consistency of the texture features between the block to be repaired and the surrounding pixels.

[0049] As one implementation method, the pixel value of each pixel within the block to be repaired is calculated based on the local consistency of texture features between the block to be repaired and its surrounding pixels, including:

[0050] definition The G-channel detail descriptor representing the surrounding pixels of the block to be repaired in the vertical direction. The G-channel detail descriptor representing the surrounding pixels of the block to be repaired in the horizontal direction. The G-channel detail descriptor representing the surrounding pixels of the block to be repaired in the diagonal direction, wherein, based on , as well as Obtain the pixel values ​​of each pixel within the block to be repaired.

[0051] In one implementation, the relationship between the average pixel value of the block to be repaired and the pixel value of each pixel within the block to be repaired is as follows:

[0052] (7)

[0053] In the formula, B represents the pixel value of each pixel in the block to be repaired;

[0054] The G-channel detail descriptor expression for the pixels surrounding the block to be repaired is as follows:

[0055] (8)

[0056] In the formula, The G-channel detail descriptor representing the surrounding pixels of the block to be repaired in the vertical direction. The G-channel detail descriptor representing the surrounding pixels of the block to be repaired in the horizontal direction. The G-channel detail descriptor represents the surrounding pixels of the block to be repaired in the diagonal direction, M represents the pixel value of each surrounding pixel of the block to be repaired, af is the defined corresponding row, and 0-5 are the defined corresponding columns;

[0057] The detailed descriptor expression for the pixels of the block to be repaired is as follows:

[0058] (9)

[0059] In the formula, This represents the B-channel detail descriptor of the block to be repaired in the vertical direction. This represents the B-channel detail descriptor of the block to be repaired in the horizontal direction. This represents the B-channel detail descriptor of the block to be repaired in the diagonal direction;

[0060] The color ratio between the block to be repaired and its surrounding pixels is as follows:

[0061] (10)

[0062] In the formula, The color ratio parameter represents the ratio of the B channel of the block to be repaired to the G channel of the surrounding pixels of the block to be repaired; the pixel value of each pixel in the block to be repaired is calculated based on formulas (7) and (9).

[0063] As one implementation method, the color ratio parameter is obtained by:

[0064] For each matching block, the average pixel value of the pixel units adjacent to the block to be repaired in the pixel region corresponding to the surrounding pixels of the reference block is calculated and defined as the reference pixel average value;

[0065] The average pixel value of the surrounding channels of the block to be repaired is obtained based on the reference pixel mean and the weight of the matching block relative to the block to be repaired;

[0066] The color ratio parameter is obtained based on the ratio of the average pixel value of the block to be repaired to the average pixel value of the surrounding channels.

[0067] In one implementation, the pixel unit within the block to be repaired is 2*2, wherein at least one pixel is partially occluded; or, the pixel unit within the block to be repaired is 2*2 or 1*2, and they share a single microlens.

[0068] To achieve the above objectives, a second aspect of the present invention provides a terminal, the terminal including at least one processor and at least one memory, the memory being coupled to the processor and storing instructions for execution by the processor, the instructions, when executed by the processor, causing the terminal to perform the image processing method as described in the first aspect.

[0069] To achieve the above objectives, a third aspect of the present invention provides a computer storage medium storing computer program instructions; when the computer program instructions are executed by a processor, they implement the image processing method as described in the first aspect.

[0070] The present invention adopts the above technical solution, and compared with the prior art, it has the following beneficial effects:

[0071] The image processing method, terminal, and computer storage medium provided by this invention address an image containing at least one block to be repaired, which comprises multiple pixels. The image processing method includes: matching the block to be repaired in the image to obtain multiple matching blocks; calculating the average pixel value of the block to be repaired based on the matching blocks; and calculating the pixel value of each pixel within the block to be repaired based on the average pixel value of the block to be repaired and the texture features of its surrounding normal pixels. Through these methods, this invention can repair image defects and improve image quality. Attached Figure Description

[0072] Figure 1 This is a schematic diagram of a phase detection structure in an image sensor according to an embodiment of the present invention;

[0073] Figure 2 A flowchart of an image processing method provided in an embodiment of the present invention;

[0074] Figure 3 This is a schematic diagram of the overall algorithm of an image processing method provided in an embodiment of the present invention;

[0075] Figure 4 This is one of the schematic diagrams of a partial algorithm of an image processing method provided in an embodiment of the present invention;

[0076] Figure 5 This is a second schematic diagram of a partial algorithm of an image processing method provided in an embodiment of the present invention;

[0077] Figure 6 This is a third schematic diagram of a partial algorithm of an image processing method provided in an embodiment of the present invention;

[0078] Figure 7 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, embodiments of this invention, and are merely used to explain this invention and are not intended to limit it. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. In this patent specification, "an embodiment" or "an implementation" refers to a specific feature, structure, or characteristic described in connection with the examples being included in at least one embodiment of this invention. Specific features, structures, or characteristics may be included in integrated circuits, electronic circuits, combinational logic circuits, or other suitable components providing the described functionality. Furthermore, this specification only describes content related to the points of this invention; other content can be understood by those skilled in the art in conjunction with existing technology.

[0080] First, in order to better understand the present invention, before describing the present invention in detail, a general description of the inventive concept of the present invention will be given.

[0081] Please refer to Figure 1 , Figure 1 This is a schematic diagram of a phase detection structure in an image sensor according to an embodiment of the present invention. Figure 1 As shown, the pixels used for phase focusing within the block to be repaired are 2*2, where at least one pixel is partially occluded, and the unoccluded portion is used to acquire the required phase information. Alternatively, the pixel units within the block to be repaired are 2*2 or 1*2, and they share a single microlens.

[0082] Specifically, Figure 1 In (a), all four pixels in the block to be repaired are partially obscured. Figure 1 In (b), among the four pixels in the block to be repaired, the two left adjacent pixels are partially occluded. Figure 1 In (c), four pixels in the block to be repaired are covered by a circular on-chip lens.

[0083] Image sensors equipped with any one of the three phase detection structures described above will produce images with quality defects corresponding to the phase detection area, thus requiring image restoration.

[0084] Based on the above description, the present invention provides an image processing method. The specific technical solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings. Figure 2 This is a flowchart of an image processing method provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the overall algorithm of an image processing method provided in an embodiment of the present invention.

[0085] like Figure 2 and Figure 3 As shown, the image contains at least one block to be repaired, which includes multiple pixels, such as 4, 6, 8, etc. The method includes:

[0086] S201: Match the block to be repaired in the image to obtain multiple matching blocks;

[0087] S202: Calculate the average pixel value of the block to be repaired based on the matching block;

[0088] S203: Calculate the pixel value of each pixel in the block to be repaired based on the average pixel value of the block to be repaired.

[0089] It should be noted that, Figure 3 The following explanation uses a block to be repaired consisting of four pixels arranged in a 2x2 grid as an example. Figure 3 In (a), the area marked with a question mark represents the block to be repaired, indicating that the pixel values ​​of each pixel in the block are unknown. First, the average pixel value of the block to be repaired is calculated using block matching in Step 1. ,like Figure 3 As shown in (b). Then, through Step 2, the pixel values ​​of all pixels in the block to be repaired are calculated based on the local detail consistency between different color channels, i.e. Figure 3 The values ​​of B0, B1, B2, and B3 are shown in (c).

[0090] In one embodiment, the block to be repaired is matched in the image to obtain multiple matching blocks. Please refer to [reference needed]. Figure 4 , Figure 4 This is one of the schematic diagrams of a partial algorithm of an image processing method provided in an embodiment of the present invention. For example... Figure 4As shown, the block to be repaired is first combined with its surrounding pixels to obtain a reference block. The number of surrounding pixels can be set according to actual needs, such as the number of pixels in the block to be repaired. It should be noted that, in an optional example, the reference block is obtained by defining the size of the block to be repaired as m*n, and selecting a 3m*3n pixel region centered on the block to be repaired. In one example, all pixels in the pixel array have the same shape and size. For example, Figure 4 If the number of pixels in the block to be repaired is 2*2, then the range of surrounding pixels is two pixels outward from the block to be repaired, resulting in a reference block of 6*6 pixels.

[0091] Then, image regions in the image that generally share the same brightness and / or texture features as the reference block are identified as matching blocks. It should be noted that matching blocks can be selected from regions that share the same brightness and / or texture features as the reference block, or a range of brightness and / or texture feature values ​​can be determined based on the reference block, and regions within that range can be identified as matching blocks. For example, pixels with the same brightness and / or texture features as the surrounding pixels identified in the previous step can be searched; the region enclosed by these pixels constitutes a matching block. Multiple matching blocks that meet the criteria can be found in the image, for example, in... Figure 4 Select the matching block that matches the reference block. Eight matching blocks can be selected in the image area. Number these eight blocks sequentially from 0 to 7 according to their position in the image.

[0092] In one embodiment, calculating the pixel mean of the block to be repaired based on the matching blocks includes:

[0093] Calculate the weight value of each matching block relative to the reference block;

[0094] The average pixel value of the block to be repaired is calculated based on the weight value and the average pixel value of the matching block.

[0095] For details, please refer to Figure 5 , Figure 5 This is a second schematic diagram of a partial algorithm of an image processing method provided in an embodiment of the present invention. For example... Figure 5 As shown, when calculating the weight value of the matching block relative to the block to be repaired, the brightness feature vectors of the matching block and the reference block are first extracted. and ; and extract detailed feature vectors of the matching block and the reference block. and It should be noted that, Figure 5In this context, R, G, and B represent color channels. It is understood that in other embodiments, they may also be other corresponding color channels.

[0096] It should be noted that in extracting the brightness feature vectors of the matching block and the reference block... and At the same time, for each matching block, the mean value of each pixel unit in its corresponding surrounding pixels can be calculated to obtain the brightness feature vector. For the reference block, the mean value of each pixel region in the surrounding pixels can be calculated to obtain the brightness feature vector. The pixel unit can be a pixel area selected by those skilled in the art based on actual needs.

[0097] For example, for Figure 5 In the 2x2 pixel region of the block to be repaired, the reference block contains 8 2x2 pixel units. In this step, the average value is calculated for each 2x2 pixel unit to obtain the average values ​​of the 8 surrounding pixels of the corresponding block to be repaired, such as... Figure 5 The first line in the diagram is shown.

[0098] Extracting detailed feature vectors from matching and reference blocks and At that time, for each matching block, the detail feature representation value of each pixel unit immediately adjacent to the block to be repaired in the surrounding pixels can be calculated to obtain the detail feature vector. For the reference block, the detail feature representation values ​​of each pixel region immediately adjacent to the block to be repaired in the surrounding pixels can be calculated to obtain the detail feature vector. .

[0099] For example, for Figure 5 The pixel region to be repaired is arranged in a 2x2 grid. Its adjacent pixel regions are four 2x2 pixel regions distributed in the top, bottom, left, and right directions. Taking any pixel region adjacent to the block to be repaired as an example, its detailed feature representation value is calculated as follows:

[0100] ;

[0101] ;

[0102] ;

[0103] In the formula, S1, S2, and S3 represent different components of the detail feature representation value, and G1, G2, G3, and G4 represent the pixel value of the corresponding pixel in a pixel unit. Detail Feature Vector Channel detail features characterizing a single matching block or the entire reference block ( Figure 5This example uses the G channel as an example (of course, other color channels can also be used). As you can understand, in this example, there are 3 S values ​​for each adjacent pixel unit, so for a matching block, there are 12 corresponding S values.

[0104] Next, the brightness difference between the matching block and the reference block is calculated according to formula (1):

[0105] (1)

[0106] In the formula, i represents the sequence number of the matching block. This represents the brightness feature vector of the i-th block. This represents the brightness feature vector of the reference block. This represents the brightness difference between the matching block and the reference block. It should be noted that in this example, i = 0~7, meaning there are 8 matching blocks evenly distributed around the reference block, making the brightness value of the block to be repaired close to the brightness values ​​of the surrounding graphics. The brightness feature vector of each matching block is summed, and the average of the 8 values ​​in the feature vector is subtracted from the brightness feature vector of the reference block, and then summed again. At this point, each matching block corresponds to one value. value.

[0107] Calculate the detail difference between the matching block and the reference block according to formula (2):

[0108] (2)

[0109] In the formula, This represents the detailed feature vector of the i-th block. This represents the detailed feature vector of the reference block. The detail difference between the matching block and the reference block. For each matching block, sum the 12 values ​​in the detail feature vector, subtract the detail feature vector of the reference block, and then sum the results. At this point, each matching block corresponds to one detail feature vector. value.

[0110] Summing the brightness difference and detail difference according to formula (3):

[0111] (3)

[0112] In the formula, w is negatively correlated with the gain, 0≤w≤1. This represents the sum of the brightness difference and the detail difference;

[0113] The weight values ​​of the matching block relative to the block to be repaired are calculated according to formulas (4) and (5):

[0114] (4)

[0115] (5)

[0116] In the formula, For threshold, based on Obtain This represents the median weight of the i-th matching block. This represents the weight value of the i-th matching block. It should be noted that the weight can be selected... The maximum value in is used as the threshold. In other implementations, when When the calculation result is negative, the median weight. The value is 0.

[0117] In one implementation, the weight of each block match is calculated. Next, a weighted average is calculated for all matching blocks to obtain the mean B of the pixels in channel B of the block to be repaired. The average pixel value of the block to be repaired is calculated based on the weight value and the average pixel value of the matching block. The calculation formula is as follows:

[0118] (6)

[0119] In the formula, This represents the average pixel value of the block to be repaired, and i represents the sequence number of the matching block, ranging from 0 to n, where n is a positive integer. This represents the average pixel value of the pixels in the region corresponding to the block to be repaired in the i-th matching block.

[0120] In one embodiment, the pixel value of each pixel within the block to be repaired is obtained based on the average pixel value of the block to be repaired and the texture features of the surrounding normal pixels, including:

[0121] The pixel values ​​of each pixel in the block to be repaired are calculated based on the average pixel value of the block to be repaired and the local consistency of texture features between the block to be repaired and the surrounding pixels.

[0122] It should be noted that, firstly, based on the G-channel pixel values ​​of the pixels surrounding the block to be repaired, the G-channel detail descriptors in the horizontal, vertical, and diagonal directions are calculated. Then, the B-channel detail descriptor expressions for the block to be repaired in the horizontal, vertical, and diagonal directions are listed. Based on the statistical regularity of the local consistency between the detail of the B-channel of the block to be repaired and the detail of the G-channel of the pixels surrounding the block, and combined with the color ratio of the B-channel of the block to be repaired to the G-channel of the surrounding pixels, a relationship can be established between the B-channel detail descriptors of the block to be repaired and the G-channel detail descriptors of the pixels surrounding the block. Finally, solving the known relationship yields the B-channel pixel values ​​of each pixel within the block to be repaired. It should be noted that here, B represents the pixel channel of the block to be repaired, and G represents the pixel channel of the immediately adjacent pixel. In this example, the block to be repaired is represented by blue channel pixels, and the immediately adjacent pixels by green channel pixels; this method also applies when the corresponding channels change.

[0123] In one embodiment, the relationship between the average pixel value of the block to be repaired and the pixel values ​​of each pixel within the block to be repaired is as follows:

[0124] (7)

[0125] In the formula, B represents the pixel value of each pixel in the block to be repaired;

[0126] The G-channel detail descriptor expression for the pixels surrounding the block to be repaired is as follows:

[0127] (8)

[0128] In the formula, The G-channel detail descriptor represents the surrounding pixels of the block to be repaired in the vertical direction. The G-channel detail descriptor representing the surrounding pixels of the block to be repaired in the horizontal direction. The G-channel detail descriptor represents the surrounding pixels of the block to be repaired in the diagonal direction, M represents the pixel value of each surrounding pixel of the block to be repaired, af is the defined corresponding row, and 0-5 are the defined corresponding columns. These are defined here only for the convenience of description. Those skilled in the art can define them according to actual needs. Of course, the number of defined rows and columns can also be changed according to the actual situation.

[0129] The detail descriptor expression for the pixels of the block to be repaired is as follows:

[0130] (9)

[0131] In the formula, This represents the B-channel detail descriptor of the block to be repaired in the vertical direction. This represents the B-channel detail descriptor of the block to be repaired in the horizontal direction. This represents the B-channel detail descriptor of the block to be repaired in the diagonal direction;

[0132] The color ratio between the block to be repaired and its surrounding pixels is as follows:

[0133] (10)

[0134] In the formula, This parameter represents the color ratio between the B channel of the block to be repaired and the G channel of the surrounding pixels.

[0135] Solving the simultaneous equations (7)-(10) yields the result. , , and The value of .

[0136] In one embodiment, the method for obtaining the color ratio parameter includes:

[0137] For each matching block, the mean pixel value at the position corresponding to the nearest pixel of the block to be repaired is calculated and defined as the reference pixel mean; for example, with Figure 4 Taking the reference block shown in the dashed box as an example, calculate the average value of all pixels in the four 2*2 pixel units around it (the figure shows the pixel value of the G channel) to obtain the reference pixel average value C.

[0138] The average pixel values ​​of the surrounding channels of the block to be repaired are obtained based on the reference pixel mean and the weights of the matching block relative to the block to be repaired; for example, the weight values ​​here are the weight values ​​used in this embodiment. The calculation method is used to obtain the average pixel value of the surrounding channels;

[0139]

[0140] The color ratio parameter is obtained based on the ratio of the average pixel value of the block to be repaired to the average pixel value of the surrounding channels. For example, =meanB / meanC.

[0141] In summary, the image processing method provided by this invention addresses an image containing at least one block to be repaired, which comprises multiple pixels. The method includes: matching the block to be repaired within the image to obtain multiple matching blocks; calculating the average pixel value of the block to be repaired based on the matching blocks; and calculating the pixel value of each pixel within the block to be repaired based on the average pixel value of the block to be repaired and the texture features of its surrounding normal pixels. Through this method, this invention can repair image defects and improve image quality.

[0142] The present invention also provides a terminal including at least one processor and at least one memory, the at least one memory being coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions, when executed by the at least one processor, causing the terminal to perform the image processing method as described above.

[0143] Figure 7 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present invention. Figure 7 As shown, terminal 600 includes a processing unit 601, which can execute the methods of embodiments of the present disclosure according to a program stored in read-only memory (ROM) 602 or a program loaded from storage portion 608 into random access memory (RAM) 603. Processing unit 601 may include, for example, a general-purpose microprocessor (e.g., CPU), an instruction processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., application-specific integrated circuit (ASIC)), etc. Processing unit 601 may also include onboard memory for caching purposes. Processing unit 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of the present disclosure.

[0144] RAM 603 stores various programs and data required for the operation of terminal 600. Processing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Processing unit 601 executes various operations of the method flow according to embodiments of the present disclosure by executing programs stored in ROM 602 and / or RAM 603. It should be noted that the aforementioned programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processing unit 601 may also execute various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

[0145] According to embodiments of this disclosure, terminal 600 may further include an input / output (I / O) interface 605, which is also connected to bus 604. Terminal 600 may also include one or more of the following components connected to input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. Furthermore, drives, removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., may also be connected to input / output (I / O) interface 605 as needed, so that computer programs read from there may be installed into storage section 608 as needed.

[0146] The method flow according to embodiments of this disclosure can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product. This includes a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from a removable medium. When the computer program is executed by processing unit 601, it performs the functions defined in the system of embodiments of this disclosure. According to embodiments of this disclosure, the systems, clients, devices, modules, and units described above can be implemented using computer program modules.

[0147] This application also provides a computer storage medium storing computer program instructions; when the computer program instructions are executed by a processor, they implement the image processing method described above.

[0148] In practice, computer storage media are used in... Figure 7 In the terminal 600 shown.

[0149] Those skilled in the art will recognize that the example methods and steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0150] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the invention without departing from the scope of the invention shall still fall within the scope of the present invention.

Claims

1. An image processing method, characterized in that, The image contains at least one block to be repaired, the block comprising multiple pixels, and the method includes: The block to be repaired is matched in the image to obtain multiple matching blocks; The pixel mean of the block to be repaired is calculated based on the weight value of each matching block relative to the reference block and the pixel mean of the matching block. The reference block is obtained by combining the block to be repaired with its surrounding pixels. The pixel value of each pixel in the block to be repaired is calculated based on the average pixel value of the block to be repaired and the texture features of the surrounding pixels. The pixel values ​​of each pixel within the block to be repaired are calculated based on the average pixel value of the block to be repaired and the texture features of its surrounding pixels, including: The pixel values ​​of each pixel in the block to be repaired are calculated based on the average pixel value of the block to be repaired and the local consistency of the texture features between the block to be repaired and the surrounding pixels. The local consistency of the texture features includes the local detail consistency between different color channels.

2. The image processing method according to claim 1, characterized in that, Matching the block to be repaired in the image yields multiple matching blocks, including: The image region in the image that has similar brightness and / or texture features to the reference block is identified as the matching block.

3. The image processing method according to claim 2, characterized in that, The methods for obtaining the reference block include: The size of the block to be repaired is defined as m*n. A 3m*3n pixel area is selected with the block to be repaired as the center to obtain the reference block.

4. The image processing method according to claim 2, characterized in that, The method for calculating the weight value of each matching block relative to the reference block is as follows: Extract the brightness feature vectors of the matching block and the reference block. and ; Extract the detail feature vectors of the matching block and the reference block. and ; The brightness difference between the matching block and the reference block is calculated according to formula (1): (1) In the formula, i represents the sequence number of the matching block. This represents the brightness feature vector of the i-th block. This represents the brightness feature vector of the reference block. This represents the brightness difference between the matching block and the reference block; The detail difference between the matching block and the reference block is calculated according to formula (2): (2) In the formula, This represents the detailed feature vector of the i-th matching block. This represents the detail feature vector of the reference block. The detailed difference between the matching block and the reference block; The brightness difference and the detail difference are summed according to formula (3): (3) In the formula, w is negatively correlated with the gain, and 0 ≤ w ≤ 1. This represents the sum of the brightness difference and the detail difference of the i-th matching block; The weight value of the matching block relative to the block to be repaired is calculated according to formulas (4) and (5): (4) (5) In the formula, The threshold is based on Obtain This represents the median weight of the i-th matching block. This represents the weight value of the i-th matching block.

5. The image processing method according to claim 4, characterized in that, Select The maximum value in is used as the threshold.

6. The image processing method according to claim 4, characterized in that, when When the calculation result is negative, the intermediate value of the weight The value is 0.

7. The image processing method according to claim 4, characterized in that, Extract the brightness feature vectors of the matching block and the reference block. and ; and, extracting the detailed feature vectors of the matching block and the reference block. and ;include: For each matching block, the mean value of each pixel unit in the pixel region corresponding to the surrounding pixels of the reference block is calculated to obtain the luminance feature vector. ; For the reference block, the mean value of each pixel unit in the surrounding pixels is calculated to obtain the luminance feature vector. ; For each matching block, the detail feature representation value of each pixel unit adjacent to the block to be repaired in the pixel region corresponding to the surrounding pixels of the reference block is calculated to obtain the detail feature vector. ; For the reference block, the detail feature representation value of each pixel unit immediately adjacent to the block to be repaired in the surrounding pixels is calculated to obtain the detail feature vector. ; For a 2x2 pixel unit, the calculation method for the detail feature representation value includes: ； ； ； In the formula, S1, S2, and S3 represent different components of the detail feature representation value, and G1, G2, G3, and G4 represent the pixel values ​​of the corresponding pixels in the pixel unit.

8. The image processing method according to claim 4, characterized in that, The average pixel value of the block to be repaired is calculated based on the weight value of each matching block relative to the reference block and the average pixel value of the matching block. The calculation formula is as follows: (6) In the formula, The value represents the average pixel value of the block to be repaired, and i represents the sequence number of the matching block, ranging from 0 to n, where n is a positive integer. This represents the average pixel value of the pixels in the region corresponding to the block to be repaired in the i-th matching block. This represents the weight value of the i-th matching block.

9. The image processing method according to claim 1, characterized in that, The pixel values ​​of each pixel within the block to be repaired are calculated based on the local consistency of texture features between the block to be repaired and its surrounding pixels, including: definition The G-channel detail descriptor representing the surrounding pixels of the block to be repaired in the vertical direction. The G-channel detail descriptor representing the surrounding pixels of the block to be repaired in the horizontal direction. The G-channel detail descriptor representing the surrounding pixels of the block to be repaired in the diagonal direction, based on , as well as Obtain the pixel values ​​of each pixel within the block to be repaired.

10. The image processing method according to claim 1, characterized in that, The relationship between the average pixel value of the block to be repaired and the pixel values ​​of each pixel within the block to be repaired is as follows: (7) In the formula, B represents the pixel value of each pixel in the block to be repaired; The G-channel detail descriptor expression for the pixels surrounding the block to be repaired is as follows: (8) In the formula, The G-channel detail descriptor representing the surrounding pixels of the block to be repaired in the vertical direction. The G-channel detail descriptor representing the surrounding pixels of the block to be repaired in the horizontal direction. The G-channel detail descriptor represents the surrounding pixels of the block to be repaired in the diagonal direction, M represents the pixel value of each surrounding pixel of the block to be repaired, af is the defined corresponding row, and 0-5 are the defined corresponding columns; The detailed descriptor expression for the pixels of the block to be repaired is as follows: (9) In the formula, This represents the B-channel detail descriptor of the block to be repaired in the vertical direction. This represents the B-channel detail descriptor of the block to be repaired in the horizontal direction. This represents the B-channel detail descriptor of the block to be repaired in the diagonal direction; The color ratio between the block to be repaired and its surrounding pixels is as follows: (10) In the formula, The color ratio parameter represents the ratio of the B channel of the block to be repaired to the G channel of the surrounding pixels of the block to be repaired; The pixel values ​​of each pixel in the block to be repaired are calculated based on formulas (7) and (9).

11. The image processing method according to claim 10, characterized in that, The methods for obtaining the color ratio parameters include: For each matching block, the average pixel value of the pixel units adjacent to the block to be repaired in the pixel region corresponding to the surrounding pixels of the reference block is calculated and defined as the reference pixel average value; The average pixel value of the surrounding channels of the block to be repaired is obtained based on the reference pixel mean and the weight of the matching block relative to the block to be repaired; The color ratio parameter is obtained based on the ratio of the average pixel value of the block to be repaired to the average pixel value of the surrounding channels.

12. The image processing method according to any one of claims 1-11, characterized in that, The pixel unit within the block to be repaired is 2*2, wherein at least one pixel is partially occluded; or, the pixel unit within the block to be repaired is 2*2 or 1*2, and they share a single microlens.

13. A terminal, characterized in that, The terminal includes at least one processor and at least one memory, the memory being coupled to the processor and storing instructions for execution by the processor, the instructions, when executed by the processor, causing the terminal to perform the image processing method as described in any one of claims 1 to 12.

14. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, they implement the image processing method as described in any one of claims 1 to 12.

Images