Image processing methods, apparatus, electronic devices and storage media

Abstract

This application relates to an image processing method, apparatus, electronic device, and storage medium. The image processing method includes: acquiring an image effect block to be processed in an image; determining the similarity between the image effect block to be processed and an image block in a preset search box, and acquiring a target similar block, wherein the target similar block is the image block in the preset search box with the lowest similarity to the image effect block to be processed; if the similarity between the target similar block and the image effect block to be processed is greater than a preset similarity, maintaining the image information corresponding to the image effect block to be processed, wherein the image information includes: pixel information and position information of the image effect block to be processed. This application solves the problem of high image algorithm complexity in related technologies and reduces the complexity of image algorithms.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to image processing methods, apparatus, electronic devices, and storage media. Background Technology

[0002] Currently, DCT (Transform Coding) technology is one of the important methods for image / video compression in coding standards. In DCT-based compression coding systems, performing DCT on the entire image would result in a considerable computational burden because the transformed DCT coefficients are correlated with every pixel in the image. Therefore, DCT transformations are typically performed on pixel blocks. The image is first divided into 8×8 pixel blocks, and then DCT transformation is performed on each block to obtain 64 DCT coefficients, significantly reducing the computational load. However, since DCT transformation is performed on each block separately, the correlation between pixel blocks is ignored. When quantizing the DCT coefficients of each block, the DCT coefficient is divided by the quantization factor and then rounded down, discarding some high-frequency components that have little impact on the image, thus reducing the bit rate. However, if the quantization is coarse, a large amount of high-frequency information at the block edges will be lost, causing discontinuous transitions at the block boundaries in the reconstructed image—this is known as block artifacts.

[0003] In related technologies, low-rank matrix restoration is used as a means to solve image deblocking problems. It can make full use of the redundancy of the image and effectively eliminate the blocking effect. However, the use of low-rank matrix restoration will cause the image algorithm to have high complexity.

[0004] There is currently no effective solution to the problem of high complexity in image algorithms in related technologies. Summary of the Invention

[0005] This embodiment provides an image processing method, apparatus, electronic device, and storage medium to at least reduce the complexity of image algorithms in encoding techniques.

[0006] Firstly, this embodiment provides an image processing method, including:

[0007] Extract the image effect blocks to be processed from the image;

[0008] Determine the similarity between the image effect block to be processed and the image block in the preset search box, and obtain the target similar block, wherein the target similar block is the image block in the preset search box with the smallest similarity to the image effect block to be processed;

[0009] If the similarity between the target similar block and the image effect block to be processed is greater than a preset similarity, the image information corresponding to the image effect block to be processed is maintained, wherein the image information includes: pixel information and position information of the image effect block to be processed.

[0010] In some embodiments, the method further includes:

[0011] If the similarity between the target similar block and the image effect block to be processed is less than the preset similarity, a preset number of image blocks are selected, and a matrix is ​​constructed based on the similarity between the preset number of similar blocks and the image effect block to be processed.

[0012] The target matrix is ​​obtained by processing the matrix using the singular value thresholding algorithm.

[0013] Based on the target matrix, the effect block of the image to be processed is subjected to effect removal processing.

[0014] In some embodiments, the image includes a YUV format image; selecting the preset number of image blocks and constructing a matrix based on the similarity between the preset number of similar blocks and the image effect block to be processed includes:

[0015] Obtain preset YUV parameter information, wherein the preset YUV parameter information includes: parameter values ​​of Y parameter, parameter values ​​of U parameter, and parameter values ​​of V parameter; the parameter values ​​include pixel size and number of pixels, the parameter value of U parameter is 1 / 2 of the parameter value of Y parameter, and the parameter value of V parameter is 1 / 2 of the parameter value of Y parameter;

[0016] Based on the preset YUV parameter information, a preset number of image blocks are selected, and a matrix is ​​constructed based on the similarity between the preset number of similar blocks and the image effect block to be processed.

[0017] In some embodiments, after performing effect removal processing on the image effect block to be processed according to the target matrix, the method further includes:

[0018] Obtain the image feature values ​​of the image effect block to be processed after the effect removal process;

[0019] Adjust the image feature values ​​to a preset range.

[0020] In some embodiments, selecting the preset number of image blocks and constructing a matrix based on the similarity between the preset number of similar blocks and the image effect block to be processed includes:

[0021] Determine the distance between the image effect block to be processed and the image block in the preset search box;

[0022] Image blocks outside a preset distance range are matched using a downsampling method with a preset step size to obtain a first number of similar blocks, and image blocks within the preset distance range are matched one by one to obtain a second number of similar blocks;

[0023] If the sum of the first quantity and the second quantity is greater than the preset quantity, a preset number of similar blocks are selected from the first number of similar blocks and the second number of similar blocks, and a matrix is ​​constructed based on the similarity between the preset number of similar blocks and the image effect block to be processed.

[0024] In some embodiments, the method further includes:

[0025] If the sum of the first quantity and the second quantity is less than the preset quantity, the search box is enlarged according to a preset ratio and a similar block search is performed.

[0026] In some embodiments, before determining the similarity between the image effect block to be processed and the image block in the preset search box, and before obtaining the target similar block with the lowest similarity to the image effect block to be processed, the method further includes:

[0027] Detect whether there are image blocks with block effects in the preset search box;

[0028] If an image block with block effect is detected in the preset search box, the similarity calculation between the image block to be processed and the image block with block effect is skipped.

[0029] In some embodiments, determining the similarity between the image effect block to be processed and the image block in the preset search box, and obtaining the target similar block with the lowest similarity to the image effect block to be processed includes:

[0030] Cluster the image effect blocks to be processed in the search box to obtain clustering results;

[0031] Based on the clustering results, identify image blocks of the same type and image blocks of different types as the image effect block to be processed;

[0032] Determine the similarity between the image effect block to be processed and the image blocks of the same type, and select a preset number of similarity blocks in ascending order to construct the matrix.

[0033] Secondly, this embodiment provides an image processing apparatus, including:

[0034] The first acquisition module is used to acquire the image effect blocks to be processed in the image;

[0035] The determining module is used to determine the similarity between the image effect block to be processed and the image block in the preset search box, and to obtain the target similar block, wherein the target similar block is the image block in the preset search box with the smallest similarity to the image effect block to be processed;

[0036] The first processing module is configured to maintain the image information corresponding to the image effect block to be processed when the similarity between the target similar block and the image effect block to be processed is greater than a preset similarity. The image information includes the pixel information and position information of the image effect block to be processed.

[0037] Thirdly, this embodiment provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image processing method described in the first aspect above.

[0038] Fourthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the image processing method described in the first aspect above.

[0039] Compared with related technologies, the image processing method, apparatus, electronic device, and storage medium provided in this embodiment obtain an image effect block to be processed in an image; determine the similarity between the image effect block to be processed and an image block in a preset search box, and obtain a target similar block, wherein the target similar block is the image block in the preset search box with the smallest similarity to the image effect block to be processed; and when the similarity between the target similar block and the image effect block to be processed is greater than a preset similarity, maintain the image information corresponding to the image effect block to be processed, wherein the image information includes the pixel information and position information of the image effect block to be processed. This approach can avoid the problem of high image algorithm complexity caused by performing low-rank matrix recovery processing on the current image block that is completely unrelated to the surrounding image blocks, thereby reducing the image algorithm complexity.

[0040] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0041] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0042] Figure 1 This is a hardware structure block diagram of the terminal of the image processing method in this embodiment;

[0043] Figure 2 This is a flowchart of the image processing method in this embodiment;

[0044] Figure 3 This is a structural block diagram of the image processing apparatus in this embodiment. Detailed Implementation

[0045] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0046] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0047] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the image processing method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0048] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the image processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0049] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0050] First, it should be noted that the block effect is mainly caused by the quantization error after block quantization. However, within the image domain, the block effect can manifest in different forms depending on the image content. The two main forms are as follows:

[0051] Trapezoidal block effect: This occurs at strong edges in an image. At low bit rates, many high-frequency coefficients of the DCT are quantized to zero, resulting in high-frequency components related to strong edges not being fully represented in the transform domain. Furthermore, because image blocks are processed separately, the continuity of strong edges crossing block boundaries cannot be guaranteed, leading to jagged noise at image edges, known as the "trapezoidal block effect."

[0052] Grid block effect: This phenomenon often occurs in flat areas of an image. In the transform domain, the DC component (DC) coefficient reflects the average brightness of an image block, thus containing most of the block's energy. In flat areas, brightness variations are small. However, if there are increases or decreases in brightness, rounding during quantization may cause the DC coefficient to exceed the decision threshold of adjacent quantization levels. This results in abrupt brightness changes at block boundaries in the reconstructed image, visually manifesting as patchy outlines within flat areas. This noise is called the "grid block effect."

[0053] To address the block artifact problem, low-rank matrix restoration (LMR) is a primary technique employed in related technologies. LMR, as a method, effectively eliminates block artifacts by fully utilizing image redundancy. Its basic principle is as follows: An image without block artifacts typically has a small rank due to strong correlation within the same region. Therefore, in the image matrix, other vectors can represent the current vector, resulting in fewer non-zero rows and a lower rank after singular value decomposition (SVD). Introducing block artifacts disrupts this original correlation, making vectors within the image matrix more linearly independent. In this case, SVD results in more non-zero rows and a higher rank. LMR utilizes this characteristic by applying a thresholding algorithm to the singular values, reducing the image rank and minimizing block artifacts, thus bringing the image quality closer to the original. However, research on LMR has revealed that it requires performing singular value decomposition on each image block within the matrix, leading to excessive computational load and high complexity.

[0054] Therefore, in order to address the problem of high complexity in image algorithms in related technologies, this embodiment provides an image processing method. Figure 2 This is a flowchart of the image processing method in this embodiment, as shown below. Figure 2 As shown, the process includes the following steps:

[0055] Step S201: Obtain the image effect block to be processed in the image.

[0056] In this step, the image can be a real-time captured image or retrieved from a database storing images. The image effect block to be processed in the image refers to a blocky image generated by block artifacts. Block artifacts refer to the effect that, as the bit rate decreases, quantization becomes coarser, and discontinuities appear at the boundaries of blocks, resulting in obvious defects in the reconstructed image.

[0057] Step S202: Determine the similarity between the image effect block to be processed and the image block in the preset search box, and obtain the target similar block, wherein the target similar block is the image block in the preset search box with the smallest similarity to the image effect block to be processed.

[0058] In this step, the preset search box can be established with the image effect block to be processed as the center, or it can be set by the user. By determining the similarity between the image effect block to be processed and the image block in the preset search box, the influence of irrelevant image blocks that are far away from the image effect block to be processed can be eliminated.

[0059] It should be noted that the image blocks in the preset search box can all be similar blocks, that is, image blocks that are somewhat similar to the image effect block to be processed.

[0060] Step S203: If the similarity between the target similar block and the image effect block to be processed is greater than the preset similarity, the image information corresponding to the image effect block to be processed is maintained. The image information includes the pixel information and position information of the image effect block to be processed.

[0061] In this step, when the similarity between the target similarity block and the image effect block to be processed is greater than the preset similarity, the image effect block to be processed is directly used as the target image effect block, and the image information corresponding to the image effect block to be processed is preserved. There is no need to perform singular value algorithm processing of low-rank matrix recovery on the image effect block to be processed. This avoids performing low-rank matrix recovery processing on the current image block that is completely unrelated to the surrounding image blocks, thus reducing the complexity of the image algorithm.

[0062] Based on steps S201 to S203 above, by maintaining the image information corresponding to the image effect block to be processed when the similarity between the target similar block and the image effect block to be processed is greater than the preset similarity, the image information includes the pixel information and position information of the image effect block to be processed. This eliminates the need to perform singular value processing of low-rank matrix recovery on the image effect block to be processed, avoids performing low-rank matrix recovery on the current image block that is completely unrelated to the surrounding image blocks, and reduces the complexity of the image algorithm.

[0063] In some embodiments, if the similarity between the target similar block and the image effect block to be processed is less than a preset similarity, a preset number of image blocks can be selected, and a matrix can be constructed based on the similarity between the preset number of similar blocks and the image effect block to be processed; the matrix can be processed based on the singular value thresholding algorithm to obtain the target matrix; and the image effect block to be processed can be de-effected based on the target matrix.

[0064] In this embodiment, when the similarity between the target similar block and the image effect block to be processed is less than a preset similarity, the matrix is ​​processed by the singular value thresholding algorithm to obtain the target matrix; finally, the image effect block to be processed is processed according to the target matrix to remove the effect, thereby improving the image quality.

[0065] In some embodiments, the image includes a YUV format image; selecting a preset number of image blocks and constructing a matrix based on the similarity between the preset number of similar blocks and the image effect block to be processed includes: obtaining preset YUV parameter information, wherein the preset YUV parameter information includes: parameter values ​​of Y parameters, parameter values ​​of U parameters, and parameter values ​​of V parameters; the parameter values ​​include pixel size and number of pixels, the parameter value of U parameters is 1 / 2 of the parameter value of Y parameters, and the parameter value of V parameters is 1 / 2 of the parameter value of Y parameters; selecting a preset number of image blocks based on the preset YUV parameter information, and constructing a matrix based on the similarity between the preset number of similar blocks and the image effect block to be processed.

[0066] In this embodiment, when performing singular value thresholding for low-rank matrices, the YUV format image is processed using three preset YUV parameters as follows:

[0067] Method 1: Simultaneously apply the low-rank matrix restoration method to the YUV components. Since the total number of pixels in the UV components is half that of the Y components, the size of the image patch to be processed, the size of the image patch, the number of image patches, and the size of the search window are reduced to half that of the Y components. Furthermore, since the texture of the UV components is not as rich as that of the Y components, there is no need to use edge detection to protect the edges.

[0068] Method 2: Simultaneously apply the low-rank matrix recovery method to the YUV components. Since the Y component has richer texture, after processing the Y component, the position of the similar block searched by the Y component can be recorded. When processing the UV component, this position is directly used to guide the UV component to obtain the position of its corresponding similar block. Since the number of pixels in the UV component is 1 / 2 of that in the Y component, the recorded position must first be mapped to the corresponding position on the UV component, and then divided by the corresponding multiple to obtain the position of the similar block in the UV component.

[0069] Method 3: Simultaneously apply the low-rank matrix recovery method to the YUV components, and perform alternating iterations on the UV components with a number of iterations much smaller than that of the Y components (e.g., 1 time).

[0070] By using the above method, the three YUV components of the YUV format image are processed separately, and different parameters are set according to the characteristics of the YUV components. This helps to improve the image deblocking effect while reducing the algorithm complexity.

[0071] In some embodiments, after the image effect block to be processed is subjected to effect removal processing according to the target matrix, the image feature values ​​of the image effect block to be processed after effect removal processing can also be obtained; and the image feature values ​​are adjusted to a preset range.

[0072] During the low-rank matrix restoration process of the singular value thresholding algorithm on image effect blocks, some standard feature values ​​of the image may change, which may lead to poor image quality after the singular value thresholding algorithm. Therefore, in this embodiment, by obtaining the image feature values ​​of the image effect block to be processed after the effect removal process and adjusting the image feature values ​​to a preset range, the image edges can be protected and the overall image quality can be improved.

[0073] It should be noted that the image feature values ​​of the image effect block to be processed after effect removal are obtained; adjusting the image feature values ​​to a preset range can be done in the following ways:

[0074] Method 1 employs edge detection to protect edges, avoiding block removal at edges and preventing edge blurring. A common edge detection algorithm is Sobel edge detection, with the following formula:

[0075] G x =[P i+1,j-1 +2*P i+1,j +P i+1,j+1 ]-[P i-1,j-1 +2*P i-1,j +P i-1,j-1 ]

[0076] G y =[P i-1,j-1 +2*P i,j-1 +P i+1,j-1 ]-[P i-1,j+1 +2*P ij+1 +P i+1,j+1 ]

[0077]

[0078] |G|=|G x |+||G y ||

[0079] Among them, P i,j It is the pixel to be detected, G x and G y These are the horizontal and vertical gradients, respectively. |G| is the calculated gradient. If |G| > thr, then the pixel is marked as an edge point. thr is a user-defined constant, and (i,j) are the coordinates of pixels inside the image effect block to be processed.

[0080] Method 2 involves dilating the edges after low-rank matrix recovery to enhance edge information. The principle of the dilution algorithm is as follows:

[0081] If there are more than m edge points within an n x n range of the current point, then the current point is also marked as an edge point, where n = 3, 5, 7; m = 4, 10, 20.

[0082] In some embodiments, selecting a preset number of image blocks and constructing a matrix based on the similarity between the preset number of similar blocks and the image effect block to be processed includes: determining the distance between the image effect block to be processed and the image blocks in a preset search box; matching the image blocks outside the preset distance range using a downsampling method with a preset step size to obtain a first number of similar blocks; and matching the image blocks within the preset distance range one by one to obtain a second number of similar blocks; if the sum of the first number and the second number is greater than a preset number, selecting a preset number of similar blocks from the first number of similar blocks and the second number of similar blocks, and constructing a matrix based on the similarity between the preset number of similar blocks and the image effect block to be processed.

[0083] In this embodiment, matching is performed in a coarse-to-fine manner. For blocks that are far away from the current block in the spatial or temporal domain, a downsampling search is used to find matching blocks. The sampling step size can be 2, 3, or 4 pixels to obtain a first number of similar blocks. For blocks that are closer (e.g., within 3 to 5 pixels above, below, to the left, or to the right of the current image block), a pixel-by-pixel search is used to obtain a second number of similar blocks. Finally, if the sum of the first number of similar blocks and the second number of similar blocks is greater than a preset number, a preset number of similar blocks are selected from the first number of similar blocks and the second number of similar blocks. A matrix is ​​constructed based on the similarity between the preset number of similar blocks and the image effect block to be processed. This avoids the problem of noisy blocks in the matched similar blocks that are far away when pixel-by-pixel matching is used. This improves the noise reduction effect of image block effect.

[0084] In some embodiments, if the sum of the first quantity and the second quantity is less than a preset quantity, the search box can be expanded according to a preset ratio and a similar block search can be performed.

[0085] In this embodiment, by expanding the search box according to a preset ratio and performing similar block search when the sum of the first quantity and the second quantity is less than a preset quantity, the accuracy of similar block matching can be improved.

[0086] In some embodiments, before determining the similarity between the image effect block to be processed and the image block in the preset search box, and before obtaining the target similar block, it is also possible to detect whether there is an image block with block effect in the preset search box; if the presence of an image block with block effect in the preset search box is detected, the similarity calculation between the image effect block to be processed and the image block with block effect is skipped.

[0087] In this embodiment, by excluding the matching of similarity between effect blocks and effect blocks in the image to be processed, the influence of some irrelevant effect blocks is avoided, thereby improving the effect of deblocking noise.

[0088] In some embodiments, determining the similarity between the image effect block to be processed and the image block in the preset search box, and obtaining the target similar block, includes: clustering the image effect block to be processed in the search box to obtain the clustering result; determining the image blocks of the same type and non-similar type as the image effect block to be processed based on the clustering result; determining the similarity between the image effect block to be processed and the image blocks of the same type, and selecting a preset number of similarity blocks in ascending order to construct a matrix.

[0089] In this embodiment, an image segmentation method is used to skip calculating the similarity of similar blocks that are not located in the same content region. The image segmentation method is as follows:

[0090] Clustering is performed using k points in space as centers, grouping objects that are closest to them. The values ​​of each cluster center are updated iteratively until the best clustering result is obtained.

[0091] Basic steps:

[0092] Step 1: Select appropriate initial centers for c classes;

[0093] Step 2: In the k-th iteration, for any sample, calculate its distance to c centers, and assign the sample to the class containing the center with the shortest distance.

[0094] Step 3: Update the center value of the class using methods such as the mean;

[0095] Step 4: For all c cluster centers, if the values ​​remain unchanged after updating using the iterative methods in steps 2 and 3, the iteration ends; otherwise, the iteration continues.

[0096] By using the clustering method described above, matching of image blocks that are not of the same type is eliminated, the influence of image blocks of different types is avoided, and the image deblocking effect is improved.

[0097] In some embodiments, before determining the similarity between the image effect block to be processed and the image blocks in the preset search box, and before selecting a first preset number of image blocks in ascending order of similarity to construct the matrix, it can also be determined whether the number of image blocks in the preset search box is greater than a second preset number; if it is determined that the number of image blocks in the preset search box is less than the second preset number, the search box is expanded according to a preset ratio.

[0098] In this embodiment, by expanding the search box by a preset ratio when the number of image blocks in the preset search box is less than a second preset number, the accuracy of similar block matching can be improved.

[0099] This embodiment also provides an image processing apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0100] Figure 3 This is a structural block diagram of the image processing apparatus in this embodiment, as shown below. Figure 3 As shown, the device includes:

[0101] The first acquisition module 31 is used to acquire the image effect block to be processed in the image;

[0102] The determination module 32, coupled to the first acquisition module 31, is used to determine the similarity between the image effect block to be processed and the image block in the preset search box, and to acquire the target similar block, wherein the target similar block is the image block in the preset search box with the smallest similarity to the image effect block to be processed;

[0103] The first processing module 33, coupled to the determining module 32, is used to maintain the image information corresponding to the image effect block to be processed when the similarity between the target similar block and the image effect block to be processed is greater than a preset similarity. The image information includes the pixel information and position information of the image effect block to be processed.

[0104] In some embodiments, the device further includes: a construction module, configured to select a preset number of image blocks and construct a matrix based on the similarity between the target similar block and the image effect block to be processed when the similarity is less than a preset similarity; a second processing module, configured to process the matrix based on a singular value thresholding algorithm to obtain a target matrix; and a third module, configured to perform effect removal processing on the image effect block to be processed based on the target matrix.

[0105] In some embodiments, when the image includes a YUV format image, the construction module includes: an acquisition unit for acquiring preset YUV parameter information, wherein the preset YUV parameter information includes: parameter values ​​of Y parameters, parameter values ​​of U parameters, and parameter values ​​of V parameters; the parameter values ​​include pixel size and number of pixels, the parameter value of U parameters is 1 / 2 of the parameter value of Y parameters, and the parameter value of V parameters is 1 / 2 of the parameter value of Y parameters; and a construction unit for selecting a preset number of image blocks according to the preset YUV parameter information, and constructing a matrix based on the similarity between the preset number of similar blocks and the image effect blocks to be processed.

[0106] In some embodiments, the device further includes: a second acquisition module for acquiring image feature values ​​of the image effect block to be processed after the effect removal process; and an adjustment module for adjusting the image feature values ​​to a preset range.

[0107] In some embodiments, the determining module 32 includes: a determining unit, configured to determine the distance between the image effect block to be processed and the image block in the preset search box; a matching unit, configured to match the image blocks outside the preset distance range using a downsampling method with a preset step size to obtain a first number of similar blocks, and to match the image blocks within the preset distance range one by one to obtain a second number of similar blocks; and a first constructing unit, configured to select a preset number of similar blocks from the first number of similar blocks and the second number of similar blocks if the sum of the first number and the second number is greater than a preset number, and to construct a matrix based on the similarity between the preset number of similar blocks and the image effect block to be processed.

[0108] In some embodiments, the device further includes a search module for expanding the search box by a preset ratio and performing a similar block search when the sum of the first number and the second number is less than a preset number.

[0109] In some embodiments, the device further includes: a detection module for determining whether there is an image block with block effect in a preset search box; and a calculation module for skipping the similarity calculation between the image block to be processed and the image block with block effect if an image block with block effect is detected in the preset search box.

[0110] In some embodiments, the determining module 32 includes: a clustering unit for clustering the image effect blocks to be processed in the search box to obtain clustering results; a determining unit for determining, based on the clustering results, image blocks of the same type and image blocks of different types that are similar to the image effect blocks to be processed; and a second construction unit for determining the similarity between the image effect blocks to be processed and image blocks of the same type, and selecting a preset number of similarity blocks in ascending order to construct a matrix.

[0111] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0112] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0113] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0114] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0115] Step S1: Obtain the image effect block to be processed in the image.

[0116] Step S2: Determine the similarity between the image effect block to be processed and the image block in the preset search box, and obtain the target similar block, wherein the target similar block is the image block in the preset search box with the smallest similarity to the image effect block to be processed.

[0117] Step S3: If the similarity between the target similar block and the image effect block to be processed is greater than the preset similarity, the image information corresponding to the image effect block to be processed is maintained. The image information includes the pixel information and position information of the image effect block to be processed.

[0118] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0119] Furthermore, in conjunction with the image processing methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any one of the image processing methods in the above embodiments.

[0120] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0121] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0122] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0123] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. An image processing method, characterized in that, include: Extract the image effect blocks to be processed from the image; Detect whether there are image blocks with block artifacts in the preset search box; wherein, the block artifacts refer to the discontinuity at the boundaries of the blocks, which creates defects in the reconstructed image. If the presence of an image block with block effect is detected in the preset search box, the similarity calculation between the image block to be processed and the image block with block effect is skipped. Determine the similarity between the image effect block to be processed and the image block in the preset search box, and obtain the target similar block, wherein the target similar block is the image block in the preset search box with the smallest similarity to the image effect block to be processed; If the similarity between the target similar block and the image effect block to be processed is greater than a preset similarity, the image information corresponding to the image effect block to be processed is maintained, wherein the image information includes: pixel information and position information of the image effect block to be processed.

2. The image processing method according to claim 1, characterized in that, The method further includes: If the similarity between the target similar block and the image effect block to be processed is less than the preset similarity, a preset number of image blocks are selected, and a matrix is ​​constructed based on the similarity between the preset number of similar blocks and the image effect block to be processed. The target matrix is ​​obtained by processing the matrix using the singular value thresholding algorithm. Based on the target matrix, the effect block of the image to be processed is subjected to effect removal processing.

3. The image processing method according to claim 2, characterized in that, The image includes a YUV format image; selecting the preset number of image blocks, and constructing a matrix based on the similarity between the preset number of similar blocks and the image effect block to be processed includes: Obtain preset YUV parameter information, wherein the preset YUV parameter information includes: parameter values ​​of Y parameter, parameter values ​​of U parameter, and parameter values ​​of V parameter; the parameter values ​​include pixel size and number of pixels, the parameter value of U parameter is 1 / 2 of the parameter value of Y parameter, and the parameter value of V parameter is 1 / 2 of the parameter value of Y parameter; Based on the preset YUV parameter information, a preset number of image blocks are selected, and a matrix is ​​constructed based on the similarity between the preset number of similar blocks and the image effect block to be processed.

4. The image processing method according to claim 2, characterized in that, After performing effect removal processing on the image effect block to be processed according to the target matrix, the method further includes: Obtain the image feature values ​​of the image effect block to be processed after the effect removal process; Adjust the image feature values ​​to a preset range.

5. The image processing method according to claim 2, characterized in that, Selecting the preset number of image blocks and constructing a matrix based on the similarity between the preset number of similar blocks and the image effect block to be processed includes: Determine the distance between the image effect block to be processed and the image block in the preset search box; Image blocks outside a preset distance range are matched using a downsampling method with a preset step size to obtain a first number of similar blocks, and image blocks within the preset distance range are matched one by one to obtain a second number of similar blocks; If the sum of the first quantity and the second quantity is greater than the preset quantity, a preset number of similar blocks are selected from the first number of similar blocks and the second number of similar blocks, and a matrix is ​​constructed based on the similarity between the preset number of similar blocks and the image effect block to be processed.

6. The image processing method according to claim 5, characterized in that, The method further includes: If the sum of the first quantity and the second quantity is less than the preset quantity, the search box is enlarged according to a preset ratio and a similar block search is performed.

7. The image processing method according to claim 2, characterized in that, Determining the similarity between the image effect block to be processed and the image block in the preset search box, and obtaining the target similar block includes: Cluster the image effect blocks to be processed in the search box to obtain clustering results; Based on the clustering results, identify image blocks of the same type and image blocks of different types as the image effect block to be processed; Determine the similarity between the image effect block to be processed and the image blocks of the same type, and select a preset number of similarity blocks in ascending order to construct the matrix.

8. An image processing apparatus, characterized in that, include: The first acquisition module acquires the image effect blocks to be processed in the image; The determination module is used to detect whether there are image blocks with block effects in the preset search box; wherein, the block effect refers to the effect of discontinuity at the boundary of the block, which forms a defect in the reconstructed image; If the presence of an image block with block effect is detected in the preset search box, the similarity calculation between the image block to be processed and the image block with block effect is skipped. Determine the similarity between the image effect block to be processed and the image block in the preset search box, and obtain the target similar block with the smallest similarity to the image effect block to be processed; The processing module is configured to maintain the image information corresponding to the image effect block to be processed when the similarity between the target similar block and the image effect block to be processed is greater than a preset similarity. The image information includes the pixel information and position information of the image effect block to be processed.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the image processing method according to any one of claims 1 to 7.

Images