Image restoration method and image restoration device
By segmenting the original image and sorting the local grayscale entropy, the error problems that the Criminisi algorithm may occur in image repair are solved, which improves the accuracy and efficiency of repair and ensures the accuracy of image repair.
Patent Information
- Application Number
- CN202010936590.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-08
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-09-08
AI Technical Summary
During the image repair process, existing Criminisi algorithm may cause repair errors, because when determining sample blocks in areas other than the area to be repaired, some contents of other areas to be repaired may be included, and the repair order depends on the blocks to be repaired. There is irrationality in the number of pixels.
By segmenting the original image into multiple sub-images, each sub-image contains a marking ball, each sub-image is repaired using the Criminisi algorithm, the repair order of the blocks to be repaired is determined based on the local grayscale entropy, and the repair order is determined and repaired during the repair process. Block matching target sample blocks to avoid iteration errors and ladder effects.
It improves the accuracy and efficiency of image repair, ensures that each block to be repaired is the largest amount of information, so as to find the most accurate matching sample block, reduce iteration errors, and avoid repair errors.
Smart Images

Figure CN112184569B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of medical technology, and in particular to an image restoration method, an image restoration device, an electronic device, and a computer-readable storage medium. Background Art
[0002] For image restoration, there are currently mainly structure-based image restoration algorithms, texture-based image restoration algorithms, Criminisi algorithm, etc.
[0003] The current Criminisi algorithm can start from the edge of the area to be repaired in the image and repair from the outside to the inside. During the repair process, the block to be repaired can be determined on the edge of the area to be repaired, and then a sample block can be determined in the area outside the area to be repaired, and then the block to be repaired can be repaired by the sample block.
[0004] However, in some cases, there are multiple areas to be repaired in the image. For a certain area to be repaired, the sample blocks determined in the area outside the area to be repaired may include part of the content in other areas to be repaired, so it may happen that one area to be repaired is repaired by another area to be repaired, resulting in repair errors. Moreover, the repair order of the blocks to be repaired in the current Criminisi algorithm is mainly determined based on the number of pixels in the block to be repaired, which is somewhat unreasonable. Summary of the invention
[0005] The present disclosure provides an image restoration method, an image restoration device, an electronic device and a computer-readable storage medium to address deficiencies in the related art.
[0006] According to a first aspect of an embodiment of the present disclosure, an image restoration method is provided, comprising:
[0007] Segmenting the original image according to the position of the marker ball in the original image to obtain a plurality of sub-images, wherein each sub-image includes a marker ball;
[0008] The marked ball in the sub-image is used as the area to be repaired, and each sub-image is repaired separately by the Criminisi algorithm. In the repair process:
[0009] Determine a plurality of blocks to be repaired at the edge of the area to be repaired, calculate the local grayscale entropy of each block to be repaired, and determine the repair order of each block to be repaired according to the local grayscale entropy;
[0010] Repairing the block to be repaired according to the repair sequence until the repair of the area to be repaired is completed;
[0011] After the restoration is completed, the multiple restored sub-images are merged.
[0012] Optionally, the area to be repaired includes a plurality of blocks to be repaired, and the repair process for each block to be repaired includes:
[0013] The values of the pixels in the area to be repaired in the block to be repaired are set to 0, the values of the pixels in the area corresponding to the area to be repaired in the sample block are set to 0, the block to be repaired is converted into a matrix to be repaired, and the sample block is converted into a sample matrix;
[0014] According to the correlation coefficient between the sample matrix and the matrix to be repaired, a target sample block matching the block to be repaired is determined from a plurality of sample blocks.
[0015] Optionally, the target sample block is a plurality of target sample blocks, and the repair process for each block to be repaired further includes:
[0016] Determine the color difference, structural difference and distance between each target sample block and the block to be repaired;
[0017] determining a matching block among the plurality of target sample blocks according to the color difference, the structure difference and the distance;
[0018] The block to be repaired is repaired by using the matching block.
[0019] Optionally, the repair process for each block to be repaired includes:
[0020] When the local grayscale entropy of the block to be repaired is greater than a preset threshold, shrinking the block to be repaired and the sample block; and / or
[0021] When the local grayscale entropy of the block to be repaired is less than a preset threshold, the block to be repaired and the sample block are enlarged.
[0022] Optionally, before segmenting the original image, the method further includes:
[0023] Improving the contrast of the original image;
[0024] Converting the contrast-enhanced duplicate image of the original image into a binary image;
[0025] Hough circle detection is performed in the binary image to determine the position of the marker ball in the original image.
[0026] According to a second aspect of an embodiment of the present disclosure, an image restoration device is provided, comprising:
[0027] An image segmentation module, used for segmenting the original image according to the position of the marker ball in the original image to obtain a plurality of sub-images, wherein each sub-image contains a marker ball;
[0028] The image restoration module is used to use the marked ball in the sub-image as the area to be restored and to restore each sub-image separately using the Criminisi algorithm. During the restoration process:
[0029] Determine a plurality of blocks to be repaired at the edge of the area to be repaired, calculate the local grayscale entropy of each block to be repaired, and determine the repair order of each block to be repaired according to the local grayscale entropy;
[0030] Repairing the block to be repaired according to the repair sequence until the repair of the area to be repaired is completed;
[0031] The image merging module is used to merge the repaired sub-images after the repair is completed.
[0032] Optionally, the area to be repaired includes a plurality of blocks to be repaired, and the image repair module includes:
[0033] A matrix conversion submodule, used to set the values of pixels in the to-be-repaired area of the to-be-repaired block to 0, set the values of pixels in the area corresponding to the to-be-repaired area of the sample block to 0, convert the to-be-repaired block into a to-be-repaired matrix, and convert the sample block into a sample matrix;
[0034] The target determination submodule is used to determine a target sample block that matches the block to be repaired from among a plurality of sample blocks according to the correlation coefficient between the sample matrix and the matrix to be repaired.
[0035] Optionally, the target sample block is a plurality of target sample blocks, and the image restoration module further includes:
[0036] A difference determination submodule, used to determine the color difference, structural difference and distance between each target sample block and the block to be repaired;
[0037] a matching determination submodule, configured to determine a matching block among the plurality of target sample blocks according to the color difference, the structure difference and the distance;
[0038] The repair submodule is used to repair the block to be repaired by using the matching block.
[0039] Optionally, the image restoration module is used to reduce the block to be restored and the sample block when the local grayscale entropy of the block to be restored is greater than a preset threshold; and / or to expand the block to be restored and the sample block when the local grayscale entropy of the block to be restored is less than a preset threshold.
[0040] Optionally, the device further comprises:
[0041] A contrast adjustment module is used to improve the contrast of the original image;
[0042] A binary conversion module, used for converting the copy image of the original image with improved contrast into a binary image;
[0043] The circle detection module is used to perform Hough circle detection in the binary image to determine the position of the marker ball in the original image.
[0044] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including:
[0045] processor;
[0046] a memory for storing processor-executable instructions;
[0047] The processor is configured to execute instructions to implement the method described in any of the above embodiments.
[0048] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored, characterized in that when the instructions are executed by a processor, the steps in the method described in any of the above embodiments are implemented.
[0049] The present disclosure determines the repair order of each block to be repaired according to the local grayscale entropy. For example, the larger the local grayscale entropy is, the earlier the repair order is. Then, when the blocks to be repaired are repaired according to the repair order, the repair can be started from the blocks to be repaired containing the most information. Compared with the blocks to be repaired containing less information, it is easier to find the matching target sample blocks more accurately.
[0050] The original image is divided into multiple sub-images, each sub-image contains a marked ball. When repairing the marked ball in each sub-image, it is possible to avoid acquiring sample blocks in other marked balls from areas outside the marked ball to repair the marked ball, thereby reducing the iterative error in the Criminisi optimization algorithm and helping to avoid the step effect in repair, thereby ensuring the accuracy of the repair result.
[0051] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0053] Figure 1 It is a schematic flow chart of an image restoration method according to an embodiment of the present disclosure.
[0054] Figure 2 is a schematic flow chart of another image restoration method according to an embodiment of the present disclosure.
[0055] Figure 3 is a schematic flow chart of yet another image restoration method according to an embodiment of the present disclosure.
[0056] Figure 4 is a schematic flow chart of another image restoration method according to an embodiment of the present disclosure.
[0057] Figure 5 It is a schematic block diagram of an image restoration device according to an embodiment of the present disclosure.
[0058] Figure 6 It is a schematic block diagram of an image restoration module according to an embodiment of the present disclosure.
[0059] Figure 7 is a schematic block diagram of another image restoration module according to an embodiment of the present disclosure.
[0060] Figure 8 It is a schematic block diagram of another image restoration device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0061] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0062] Figure 1 FIG. 1 is a schematic flow chart of an image restoration method according to an embodiment of the present disclosure. Figure 1 As shown, the image repair method may include the following steps:
[0063] Step S101, segmenting the original image according to the position of the marker ball in the original image to obtain a plurality of sub-images, wherein each sub-image includes a marker ball;
[0064] Step S102: Use the marked balls in the sub-images as the areas to be repaired, and repair each sub-image separately using the Criminisi algorithm. In the repair process:
[0065] Step S103, determining a plurality of blocks to be repaired at the edge of the area to be repaired, calculating the local grayscale entropy of each block to be repaired, and determining the repair order of each block to be repaired according to the local grayscale entropy;
[0066] Step S104, repairing the block to be repaired according to the repair sequence until the repair of the area to be repaired is completed;
[0067] Step S105: after the restoration is completed, the restored sub-images are merged.
[0068] In one embodiment, the image restoration method can be applied to a parameter calibration system, which includes a robotic arm with six degrees of freedom or more and a registration plate, wherein the registration plate is fixed to the end of the robotic arm, and a plurality of metal balls (such as steel balls) are fixed on the registration plate, wherein the number of the plurality of metal balls is greater than or equal to 4, and at least one of the plurality of metal balls is not coplanar with the other metal balls, and the robotic arm can work in conjunction with an image acquisition device.
[0069] The image acquisition device may be a device capable of acquiring X-ray images, such as a C-arm machine. The end of the mechanical arm may be a flange, and the registration plate may be fixedly mounted on the flange.
[0070] The parameter calibration system can perform the following steps:
[0071] Step A, controlling the image acquisition device to capture in the i-th position to obtain the i-th X-ray image, and controlling the movement of the robotic arm so that the X-ray image captured by the image acquisition device contains at least a preset number of the multiple metal balls;
[0072] Step B, recording the i-th posture information of the robotic arm when the image acquisition device takes the X-ray image;
[0073] Step C, determining an i-th transformation matrix between the two-dimensional coordinates of the metal ball in the i-th X-ray image and the three-dimensional coordinates of the metal ball in the coordinate system of the end of the robotic arm;
[0074] Step D, determining the i-th geometric position relationship between the world coordinate system and the coordinate system where the i-th X-ray image is located according to the i-th transformation matrix and the i-th posture information;
[0075] Step E: Execute the above steps A to D n times to obtain n geometric position relationships, 1≤i≤n.
[0076] The original image may be an X-ray image containing a metal ball, the metal ball being the marker ball, and the area to be repaired being the area where the metal ball is located in the X-ray image. Since the X-ray image taken for the patient presents the patient's body structure, but the marker ball is used as a reference to determine the change matrix, and is not the inherent structure of the patient's body, the marker ball in the original image blocks the patient's body structure in the image, which is equivalent to the position where the marker ball is damaged. Therefore, the area to be repaired where the marker ball is located needs to be repaired so that the repaired image accurately reflects the patient's body structure.
[0077] In one embodiment, the position of the marker ball in the original image may be determined first, and then the original image may be segmented according to the position of the marker ball, thereby obtaining a plurality of sub-images. The segmentation criterion may be to segment different marker balls into different sub-images, and on this basis, to make the difference in length and width of the sub-images smaller. Of course, in some cases, different marker balls overlap in the original image, and then the overlapping marker balls may be segmented into one sub-image.
[0078] Then, the marked balls in the sub-images can be used as the areas to be repaired, and each sub-image can be repaired separately using the Criminisi algorithm.
[0079] In the repair process based on the Criminisi algorithm, pixels can be determined at the edge of the area to be repaired, and then the block to be repaired containing the pixel is ensured. For example, the pixel is the center of the block to be repaired. The block to be repaired contains both pixels in the area to be repaired and pixels in the area that does not need to be repaired. Multiple sample blocks can be determined outside the repair area through a window of the same size as the block to be repaired, and a target sample block matching the block to be repaired is determined from the multiple sample blocks, and the block to be repaired is repaired through the target sample block.
[0080] Similarly, the edge of the area to be repaired can be repaired, thereby repairing the outer part of the area to be repaired, making the area to be repaired smaller. For a new area to be repaired, the above method can be continued to be repaired from the edge, thereby macroscopically repairing the area to be repaired from the outside to the inside.
[0081] During the repair process, multiple blocks to be repaired can be determined at the edge of the area to be repaired, the local grayscale entropy of each block to be repaired (which can also be replaced by the average grayscale entropy) is calculated, and the repair order of each block to be repaired is determined according to the local grayscale entropy. Then, the blocks to be repaired can be repaired according to the repair order until the area to be repaired is completely repaired.
[0082] Since the local grayscale entropy can reflect the richness of information (such as texture) in the image, the larger the local grayscale entropy, the more information in the block to be repaired. In this embodiment, the repair order of each block to be repaired can be determined according to the local grayscale entropy. For example, the larger the local grayscale entropy, the earlier the repair order. Then, the blocks to be repaired can be repaired according to the repair order, and the repair can be started from the blocks to be repaired containing the most information, which is more accurate than the blocks to be repaired containing less information. Finding a matching target sample block is convenient.
[0083] There may be overlapping parts between adjacent blocks to be repaired. Therefore, after repairing a block to be repaired that contains more information (for example, called block A), the block to be repaired that contains less information adjacent to block A (for example, called block B) can include some of the information in the repaired block A, thereby increasing the amount of information in block B, which is conducive to improving the accuracy of determining the target sample block that matches block B, thereby improving the accuracy of repairing block B.
[0084] Moreover, this embodiment divides the original image into multiple sub-images, each of which contains a marker ball. When repairing the marker ball in each sub-image, it is possible to avoid acquiring sample blocks in other marker balls from areas outside the marker ball to repair the marker ball, thereby reducing the iterative error in the Criminisi optimization algorithm and helping to avoid the step effect in the repair, thereby ensuring the accuracy of the repair result.
[0085] Figure 2 FIG. 1 is a schematic flow chart of another image restoration method according to an embodiment of the present disclosure. Figure 2 As shown, the area to be repaired includes multiple blocks to be repaired, and the repair process for each block to be repaired includes:
[0086] Step S201, setting the values of pixels in the area to be repaired in the block to be repaired to 0, setting the values of pixels in the area corresponding to the area to be repaired in the sample block to 0, converting the block to be repaired into a matrix to be repaired, and converting the sample block into a sample matrix;
[0087] Step S202: determining a target sample block matching the block to be repaired from a plurality of sample blocks according to the correlation coefficient between the sample matrix and the matrix to be repaired.
[0088] In one embodiment, during the repair process, it is necessary to determine a target sample block that matches the block to be repaired from a plurality of sample blocks outside the area to be repaired.
[0089] Specifically, the values of pixels in the area to be repaired in the block to be repaired can be set to 0, while the values of other pixels in the block to be repaired remain unchanged. Then, the values of pixels in the area corresponding to the area to be repaired in the sample block are set to 0, while the values of other pixels in the sample block remain unchanged.
[0090] For example, the block to be repaired and the sample block are squares of the same size, and the pixels in the lower half of the block to be repaired are within the area to be repaired. Then, the area in the sample block corresponding to the area to be repaired is the lower half of the sample block, so that the values of the pixels in the lower half of the block to be repaired can be set to 1, and the values of the pixels in the lower half of the sample block can be set to 1.
[0091] Then, the block to be repaired is converted into a matrix to be repaired, and the sample block is converted into a sample matrix, and then the correlation coefficient between the sample matrix and the matrix to be repaired is determined. Accordingly, since the values of the pixels in the area corresponding to the area to be repaired in the matrix to be repaired and the sample matrix are set to 0, when calculating the correlation coefficient of the corresponding matrix, it is not necessary to consider the values of the pixels in the area to be repaired, which is conducive to improving the accuracy of determining the correlation coefficient. Then, according to the relevant information, a target sample block matching the block to be repaired is determined in multiple sample blocks, which is conducive to accurately determining the target sample block, so as to accurately repair the block to be repaired through the target sample block.
[0092] Figure 3 FIG. 1 is a schematic flow chart of another image restoration method according to an embodiment of the present disclosure. Figure 3 As shown, the target sample block is a plurality of target sample blocks, and the repair process for each block to be repaired further includes:
[0093] Step S203, determining the color difference, structural difference and distance between each target sample block and the block to be repaired;
[0094] Step S204, determining a matching block among the plurality of target sample blocks according to the color difference, the structure difference and the distance;
[0095] Step S205: repairing the block to be repaired by using the matching block.
[0096] In one embodiment, the above steps S201 to S202 can preliminarily determine a target sample block for repairing the block to be repaired from multiple sample blocks. However, in some cases, multiple target sample blocks can be determined. For example, the correlation coefficients between the sample matrices corresponding to the multiple sample blocks and the matrix to be repaired meet the requirements (for example, greater than a preset coefficient threshold). In this case, a matching block for repairing the block to be repaired can be further determined from the target sample blocks.
[0097] Specifically, the color difference, structural difference and distance between each target sample block and the block to be repaired can be determined. The color difference of pixels at the same position in the target sample block and the block to be repaired can be calculated, and then the average of the color differences of the pixel differences at all positions can be calculated as the color difference between the target sample block and the block to be repaired; the color gradient in the target sample block can be calculated, and the difference between the color gradient in the block to be repaired can be used as the structural difference.
[0098] Then, a matching block can be determined from the multiple target sample blocks according to the color difference, the structural difference and the distance, for example, weights can be determined respectively according to the color difference, the structural difference and the distance, and then the weights corresponding to the three can be summed up, and the target sample block with the largest sum can be determined as the matching block. The weight corresponding to the color difference is inversely correlated with the color difference, the weight corresponding to the structural difference is inversely correlated with the structural difference, and the weight corresponding to the distance is inversely correlated with the distance.
[0099] Since the correlation between blocks with greater color differences is lower, the correlation between blocks with greater structural differences is lower, and the correlation between blocks with greater distances is lower, by determining the matching blocks based on the above method, the matching blocks with the highest correlation with the block to be repaired can be determined among multiple target sample blocks, and the block to be repaired can be repaired according to the matching blocks, which can improve the accuracy of the repair result.
[0100] Optionally, the repair process for each block to be repaired includes:
[0101] Step S206, when the local grayscale entropy of the block to be repaired is greater than a preset threshold, shrinking the block to be repaired and the sample block; and / or
[0102] Step S207: when the local grayscale entropy of the block to be repaired is less than a preset threshold, the block to be repaired and the sample block are enlarged.
[0103] In one embodiment, when the local grayscale entropy of the block to be repaired is greater than a preset threshold, the block to be repaired contains more information, so the block to be repaired and the sample block can be reduced, thereby reducing the amount of information in the block to be repaired and the sample block, thereby increasing the speed of determining the correlation between the sample block and the block to be repaired, so as to determine the target sample block that matches the block to be repaired in the sample block as soon as possible to complete the repair.
[0104] In one embodiment, when the local grayscale entropy of the block to be repaired is less than a preset threshold, the information in the block to be repaired is less, so the block to be repaired and the sample block can be enlarged, thereby increasing the amount of information in the block to be repaired and the sample block, thereby improving the accuracy of determining the correlation between the sample block and the block to be repaired, so as to improve the accuracy of determining the target sample block that matches the block to be repaired in the sample block, and improve the accuracy of the repair result.
[0105] Figure 4 FIG. 1 is a schematic flow chart of another image restoration method according to an embodiment of the present disclosure. Figure 4 As shown, before segmenting the original image, the method further includes:
[0106] Step S106, improving the contrast of the original image;
[0107] Step S107, converting the copy image of the original image with improved contrast into a binary image;
[0108] Step S108, performing Hough circle detection in the binary image to determine the position of the marker ball in the original image.
[0109] In one embodiment, since the marker ball can be a metal ball, and the main content of the original image is the human body structure, there is a large difference between the main content of the marker ball and the original image, so the contrast of the original image can be improved so that the area to be repaired of the marker ball in the original image is more obvious than other areas in the original image, so as to facilitate subsequent distinction. Among them, the contrast of a part of the original image (such as the area where the marker ball is located) can be enhanced by the ACE enhancement algorithm.
[0110] The image with enhanced contrast can be copied, and the copied image can be segmented into two values using a predetermined threshold value, so as to convert the copied image into a binary image. Then, Hough circle detection can be performed in the binary image to determine the coordinates of a circular or approximately circular area in the binary image, and the coordinates are matched to the original image, so that the area where the marker ball is located can be determined in the original image, that is, the area to be repaired.
[0111] Corresponding to the above-mentioned embodiment of the image restoration method, the present disclosure also proposes an embodiment of an image restoration device.
[0112] Figure 5 FIG. 1 is a schematic block diagram of an image restoration device according to an embodiment of the present disclosure. Figure 5 As shown, the image restoration device comprises:
[0113] An image segmentation module 101 is used to segment the original image according to the position of the marker ball in the original image to obtain a plurality of sub-images, wherein each sub-image includes a marker ball;
[0114] The image restoration module 102 is used to use the marked balls in the sub-images as the areas to be restored and to restore each sub-image by using the Criminisi algorithm. In the restoration process:
[0115] Determine a plurality of blocks to be repaired at the edge of the area to be repaired, calculate the local grayscale entropy of each block to be repaired, and determine the repair order of each block to be repaired according to the local grayscale entropy;
[0116] Repairing the block to be repaired according to the repair sequence until the repair of the area to be repaired is completed;
[0117] The image merging module 103 is used to merge the restored sub-images after the restoration is completed.
[0118] Figure 6 FIG. 1 is a schematic block diagram of an image restoration module according to an embodiment of the present disclosure. Figure 6 As shown, the area to be repaired includes a plurality of blocks to be repaired, and the image repair module 102 includes:
[0119] The matrix conversion submodule 1021 is used to set the values of pixels in the area to be repaired in the block to be repaired to 0, set the values of pixels in the area corresponding to the area to be repaired in the sample block to 0, convert the block to be repaired into a matrix to be repaired, and convert the sample block into a sample matrix;
[0120] The target determination submodule 1022 is used to determine a target sample block that matches the block to be repaired from among multiple sample blocks according to the correlation coefficient between the sample matrix and the matrix to be repaired.
[0121] Figure 7 FIG. 1 is a schematic block diagram of another image restoration module according to an embodiment of the present disclosure. Figure 7 As shown, the target sample block is a plurality of target sample blocks, and the image restoration module 102 further includes:
[0122] A difference determination submodule 1023 is used to determine the color difference, structural difference and distance between each target sample block and the block to be repaired;
[0123] a matching determination submodule 1024, configured to determine a matching block among the plurality of target sample blocks according to the color difference, the structure difference and the distance;
[0124] The repair submodule 1025 is configured to repair the block to be repaired by using the matching block.
[0125] Optionally, the image restoration module is used to reduce the block to be restored and the sample block when the local grayscale entropy of the block to be restored is greater than a preset threshold; and / or to expand the block to be restored and the sample block when the local grayscale entropy of the block to be restored is less than a preset threshold.
[0126] Figure 8 FIG. 1 is a schematic block diagram of another image restoration device according to an embodiment of the present disclosure. Figure 8As shown, the device also includes:
[0127] A contrast adjustment module 104 is used to improve the contrast of the original image;
[0128] A binary conversion module 105, used for converting the copy image of the original image with improved contrast into a binary image;
[0129] The circle detection module 106 is used to perform Hough circle detection in the binary image to determine the position of the marker ball in the original image.
[0130] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0131] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.
[0132] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0133] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. An image restoration method, characterized in that: include: Segmenting the original image according to the position of the marker ball in the original image to obtain a plurality of sub-images, wherein each sub-image includes a marker ball; The marked ball in the sub-image is used as the area to be repaired, and each sub-image is repaired separately by the Criminisi algorithm. In the repair process: Determine a plurality of blocks to be repaired at the edge of the area to be repaired, calculate the local grayscale entropy of each block to be repaired, and determine the repair order of each block to be repaired according to the local grayscale entropy; Repairing the block to be repaired according to the repair sequence until the repair of the area to be repaired is completed; After the restoration is completed, the restored multiple sub-images are merged; The repair process for each block to be repaired includes: When the local grayscale entropy of the block to be repaired is greater than a preset threshold, shrinking the block to be repaired and the sample block; and / or When the local grayscale entropy of the block to be repaired is less than a preset threshold, the block to be repaired and the sample block are enlarged.
2. The method according to claim 1, characterized in that The area to be repaired includes a plurality of blocks to be repaired, and the repair process for each block to be repaired includes: The values of the pixels in the area to be repaired in the block to be repaired are set to 0, the values of the pixels in the area corresponding to the area to be repaired in the sample block are set to 0, the block to be repaired is converted into a matrix to be repaired, and the sample block is converted into a sample matrix; According to the correlation coefficient between the sample matrix and the matrix to be repaired, a target sample block matching the block to be repaired is determined from a plurality of sample blocks.
3. The method according to claim 2, characterized in that The target sample block is a plurality of target sample blocks, and the repair process for each block to be repaired further includes: Determine the color difference, structural difference and distance between each target sample block and the block to be repaired; determining a matching block among the plurality of target sample blocks according to the color difference, the structure difference and the distance; The block to be repaired is repaired by using the matching block.
4. The method according to any one of claims 1 to 3, characterized in that Before segmenting the original image, the method further includes: Improving the contrast of the original image; Converting the contrast-enhanced duplicate image of the original image into a binary image; Hough circle detection is performed in the binary image to determine the position of the marker ball in the original image.
5. An image restoration device, characterized in that: include: An image segmentation module, used for segmenting the original image according to the position of the marker ball in the original image to obtain a plurality of sub-images, wherein each sub-image contains a marker ball; The image restoration module is used to use the marked ball in the sub-image as the area to be restored and to restore each sub-image separately using the Criminisi algorithm. During the restoration process: Determine a plurality of blocks to be repaired at the edge of the area to be repaired, calculate the local grayscale entropy of each block to be repaired, and determine the repair order of each block to be repaired according to the local grayscale entropy; Repairing the block to be repaired according to the repair sequence until the repair of the area to be repaired is completed; An image merging module, used to merge the restored sub-images after restoration is completed; Among them, the image restoration module is used to reduce the block to be repaired and the sample block when the local grayscale entropy of the block to be repaired is greater than a preset threshold; and / or expand the block to be repaired and the sample block when the local grayscale entropy of the block to be repaired is less than a preset threshold.
6. The device according to claim 5, characterized in that The area to be repaired includes a plurality of blocks to be repaired, and the image repair module includes: A matrix conversion submodule, used to set the values of pixels in the to-be-repaired area of the to-be-repaired block to 0, set the values of pixels in the area corresponding to the to-be-repaired area of the sample block to 0, convert the to-be-repaired block into a to-be-repaired matrix, and convert the sample block into a sample matrix; The target determination submodule is used to determine a target sample block that matches the block to be repaired from among a plurality of sample blocks according to the correlation coefficient between the sample matrix and the matrix to be repaired.
7. The device according to claim 6, characterized in that The target sample block is a plurality of target sample blocks, and the image restoration module further includes: A difference determination submodule, used to determine the color difference, structural difference and distance between each target sample block and the block to be repaired; a matching determination submodule, configured to determine a matching block among the plurality of target sample blocks according to the color difference, the structure difference and the distance; The repair submodule is used to repair the block to be repaired by using the matching block.
8. The device according to any one of claims 5 to 7, characterized in that The device also includes: A contrast adjustment module is used to improve the contrast of the original image; A binary conversion module, used for converting the copy image of the original image with improved contrast into a binary image; The circle detection module is used to perform Hough circle detection in the binary image to determine the position of the marker ball in the original image.
9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute instructions to implement the method according to any one of claims 1 to 4.
10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps in the method according to any one of claims 1 to 4 are implemented.
