Image artifact recognition method, image artifact recognition device, terminal and computer storage medium

Abstract

The invention provides an image artifact recognition method, an image artifact recognition device, a terminal and a computer storage medium, and the method comprises the steps: carrying out the preprocessing of artifact enhancement of a to-be-processed image, so as to obtain an artifact-enhanced image; based on a preset artifact identification model, performing artifact preliminary identification on the artifact enhanced image to obtain each artifact area in the artifact enhanced image; for a single artifact region, extracting a corresponding maximum connected domain, and performing spatial feature extraction on the maximum connected domain to obtain a sub-artifact recognition result of the corresponding artifact region; and obtaining a total artifact identification result of the to-be-processed image based on the sub-artifact identification results of all the artifact areas, thereby accurately and efficiently identifying the artifact areas on the to-be-processed image so as to facilitate the proceeding of a subsequent artifact elimination step.

Description

Technical Field

[0001] This application belongs to the field of image processing, and relates to an image artifact removal technology, in particular to an image artifact recognition method, an image artifact recognition device, a terminal, and a computer storage medium. Background Art

[0002] Image artifacts refer to non-real signals or image distortions generated due to various reasons during the imaging process, which will lead to inaccurate analysis results during the image analysis process. Therefore, in order to improve the accuracy of image analysis results, it is necessary to accurately identify and eliminate the artifacts on the image.

[0003] In the prior art, the method for identifying artifacts is usually: magnify the image to be processed through a third-party image viewing software, and then manually check the magnified image to find the defect location, so as to realize the identification of artifacts. However, this method not only takes time and effort, but also the recognition result is affected by the subjective judgment of the operator, resulting in insufficient accuracy of artifact detection, and further affecting the artifact removal effect.

[0004] Therefore, how to accurately and conveniently identify the artifacts on the image is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide an image artifact recognition method, an image artifact recognition device, a terminal, and a computer storage medium, which are used to solve the problems that the existing artifact recognition method is not only time-consuming and laborious, but also has insufficient accuracy of artifact detection.

[0006] In the first aspect, this application provides an image artifact recognition method, including:

[0007] Perform preprocessing of artifact enhancement on the image to be processed to obtain an artifact-enhanced image;

[0008] Based on a preset artifact recognition model, perform preliminary artifact recognition on the artifact-enhanced image to obtain each artifact region in the artifact-enhanced image;

[0009] For a single artifact region, extract the corresponding largest connected component, perform spatial feature extraction on the largest connected component, and obtain a sub-artifact recognition result corresponding to the artifact region;

[0010] Based on the sub-artifact recognition results of all the artifact regions, obtain the total artifact recognition result of the image to be processed.

[0011] In this application, through a preset artifact recognition module, the recognition of artifact regions on the image to be processed is realized, avoiding manual inspection, reducing labor costs, having high recognition efficiency, saving recognition time, and having high recognition accuracy.

[0012] In one embodiment of the present application, the preset artifact recognition model is an object detection network integrating the ECA lightweight attention mechanism; the object detection network includes the YOLO type or the SEE type.

[0013] In one embodiment of the present application, for a single artifact region, extracting the corresponding largest connected component includes:

[0014] Performing local threshold segmentation on the artifact region to obtain a binary artifact region;

[0015] Performing color inversion processing on the binary artifact region to obtain an inverted color region;

[0016] Extracting each connected component in the inverted color region and obtaining the connected component with the largest area as the largest connected component.

[0017] In one embodiment of the present application, performing preprocessing on the image to be processed for artifact enhancement to obtain an artifact-enhanced image includes:

[0018] Performing frequency domain transformation on the image to be processed to obtain a frequency domain image;

[0019] Filtering the frequency domain image based on a preset filter to obtain a filtered frequency domain image;

[0020] Performing spatial domain transformation on the filtered frequency domain image to obtain a filtered image;

[0021] Performing image enhancement based on the filtered image to obtain the artifact-enhanced image.

[0022] In one embodiment of the present application, performing preprocessing on the image to be processed for artifact enhancement to obtain an artifact-enhanced image includes:

[0023] Decomposing the image to be processed to obtain a pyramid of the image to be processed;

[0024] Performing frequency domain transformation on each layer image in the pyramid of the image to be processed to obtain a pyramid of frequency domain images;

[0025] Filtering each layer image in the pyramid of frequency domain images based on a preset filter to obtain a pyramid of filtered frequency domain images;

[0026] Performing spatial domain transformation on each layer image in the pyramid of filtered frequency domain images to obtain a pyramid of filtered images;

[0027] Performing image reconstruction based on each layer image in the pyramid of filtered images to obtain a filtered image;

[0028] Perform image enhancement based on the filtered image to obtain the artifact-enhanced image.

[0029] In an embodiment of the present application, the performing image enhancement based on the filtered image to obtain the artifact-enhanced image includes: performing non-linear transformation processing on the filtered image to enhance the contrast of the filtered image, and using the processed image as the artifact-enhanced image;

[0030] Wherein, the non-linear transformation processing includes at least one of histogram truncation or gamma transformation.

[0031] In an embodiment of the present application, the performing spatial feature extraction on the largest connected component to obtain a sub-artifact recognition result corresponding to the artifact region includes:

[0032] Based on the largest connected component, obtain the position of the largest connected component;

[0033] Based on the position of the largest connected component, obtain a sub-artifact recognition result corresponding to the artifact region.

[0034] In an embodiment of the present application, the obtaining a sub-artifact recognition result corresponding to the artifact region based on the position of the largest connected component includes:

[0035] Obtain the sizes of the largest connected component in the first direction and the second direction respectively, and use the maximum value of the sizes as the artifact size corresponding to the artifact region;

[0036] Based on the corresponding artifact size, and in combination with the position of the largest connected component, obtain an artifact recognition result corresponding to the artifact region;

[0037] Wherein, the first direction is the direction in which the pixel points on the image to be processed are arranged horizontally, and the second direction is the direction in which the pixel points on the image to be processed are arranged vertically.

[0038] In a second aspect, the present application provides an image artifact recognition device, including an enhancement module, a preliminary recognition module, an accurate recognition module, and an artifact output module;

[0039] The enhancement module is configured to perform preprocessing of artifact enhancement on the image to be processed to obtain an artifact-enhanced image;

[0040] The preliminary recognition module is configured to perform preliminary artifact recognition on the artifact-enhanced image based on a preset artifact recognition model to obtain all artifact regions of the artifact-enhanced image;

[0041] The precise recognition module is used to extract the corresponding largest connected component for a single artifact region, perform spatial feature extraction on the largest connected component, and obtain a sub-artifact recognition result corresponding to the artifact region;

[0042] The artifact output module is used to obtain the total artifact recognition result of the image to be processed based on the sub-artifact recognition results of all the artifact regions.

[0043] In a third aspect, the present application provides a terminal, including: a processor and a memory, which are communicatively connected between the memory and the processor;

[0044] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the image artifact recognition method as described above.

[0045] In a fourth aspect, the present application provides a computer storage medium, which stores a computer program, and when the computer program is executed by a processor, the image artifact recognition method as described above is implemented.

[0046] As described above, the present application provides an image artifact recognition method, an image artifact recognition device, a terminal, and a computer storage medium. Through the artifact recognition model, the recognition of the artifact region on the image to be processed is realized. Compared with manual inspection, the recognition by the artifact recognition model not only has higher efficiency, but also avoids the recognition error caused by the operator's subjective experience, etc., which is beneficial to improving the accuracy of artifact recognition, facilitating the subsequent artifact elimination, and further improving the image quality of the obtained image. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It shows a schematic flowchart of an image artifact recognition method according to an embodiment of the present application.

[0048] Figure 2 It shows a schematic flowchart of a method for obtaining an artifact-enhanced image according to an embodiment of the present application.

[0049] Figure 3 It shows a schematic flowchart of another method for obtaining an artifact-enhanced image according to an embodiment of the present application.

[0050] Figure 4 It shows a schematic flowchart of a method for obtaining a sub-artifact recognition result according to an embodiment of the present application.

[0051] Figure 5 It shows a schematic flowchart of a method for obtaining an artifact size according to an embodiment of the present application.

[0052] Figure 6It shows a schematic flowchart of a method for obtaining the largest connected component according to an embodiment of the present application.

[0053] Figure 7 It shows a schematic structural diagram of an image artifact recognition device according to an embodiment of the present application.

[0054] Figure 8 It shows a schematic structural diagram of a terminal according to an embodiment of the present application.

[0055] Description of the reference numerals

[0056] 41 Enhancement module

[0057] 42 Preliminary recognition module

[0058] 43 Precise recognition module

[0059] 44 Artifact output module

[0060] 50 Terminal

[0061] 51 Processor

[0062] 52 Memory

[0063] 53 Network interface

[0064] 54 User interface

[0065] 55 Bus system

[0066] 521 Operating system

[0067] 522 Application program Detailed implementation manners

[0068] The following uses specific specific examples to illustrate the implementation manners of the present application. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0069] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present application in a schematic manner. Therefore, only the components related to the present application are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0070] In existing image artifact recognition methods, the image to be processed is often magnified by a third-party image viewing software so that the artifacts on the image can be observed by the human eye, and then the artifacts on the image are searched for and located through manual inspection by an operator. However, the manual inspection method requires a large amount of time and labor costs, and due to the influence of the operator's experience or vision differences, the accuracy of manual inspection is insufficient. As a result, the recognition of image artifacts is not only very inefficient, but also the recognition results are poor, which further affects the artifact elimination effect and leads to poor quality of the finally obtained image.

[0071] In view of the technical problems existing in the prior art, the following embodiments of the present application provide an image artifact recognition method, an image artifact recognition device, a terminal, and a computer storage medium. By using a preset artifact recognition model, the artifact area in the image to be processed is recognized, and then the largest connected domain inside is obtained based on the artifact area. Combining the position of the largest connected domain, the artifacts on the image to be processed are accurately and efficiently recognized, which not only effectively improves the efficiency of artifact recognition and saves costs, but also does not require manual inspection, thus reducing the labor cost of artifact recognition and avoiding the influence of the operator's subjective consciousness on the artifact recognition result, and improving the accuracy of artifact recognition.

[0072] The following embodiments of the present application provide an image artifact recognition method, an image artifact recognition device, a terminal, and a computer storage medium, including but not limited to when there are stains or black spots on the irradiation window of an X-ray detector, etc., resulting in artifacts on the acquired detection image, and performing artifact recognition on the detection image to facilitate subsequent artifact elimination. The following will describe taking the removal of artifacts on an X-ray detection image as an example.

[0073] It should be noted that the image artifact recognition method, image artifact recognition device, terminal, and computer storage medium provided in the following embodiments of the present application can also perform artifact recognition on other types of images, including but not limited to CT scan images, nuclear magnetic resonance images, and positron emission tomography images, and the reasons for the formation of artifacts on these images include but are not limited to defects of the imaging device itself and interference of the imaging environment. The present application does not make specific limitations here.

[0074] Next, the technical solutions in the embodiments of the present application will be described in detail with reference to the accompanying drawings in the embodiments of the present application.

[0075] As Figure 1 shown, this embodiment provides an image artifact recognition method, including:

[0076] S100, perform preprocessing of artifact enhancement on the image to be processed to obtain an artifact-enhanced image.

[0077] Among them, artifact enhancement refers to enhancing the recognizable degree of the artifact part on the image data. By performing preprocessing on the written artifacts, the subsequent artifact recognition results can be made more accurate and efficient.

[0078] Exemplarily, the preprocessing for artifact enhancement of the image to be processed includes but is not limited to: implementing an image processing method by at least one of increasing the contrast of the image to be processed, performing gray-scale transformation on the image to be processed, and performing frequency-domain filtering on the image to be processed.

[0079] For X-ray detection images, the artifact part usually has specific frequency characteristics in the frequency domain. Based on this, in order to better improve the visibility of the artifacts in the image to be processed for subsequent artifact recognition, in some optional embodiments of the present application, as Figure 2 shown, the preprocessing for artifact enhancement of the image to be processed to obtain an artifact-enhanced image includes:

[0080] S110, perform a frequency-domain transformation on the image to be processed to obtain a frequency-domain image.

[0081] Among them, the frequency-domain transformation refers to converting the image data in the spatial domain to the frequency domain.

[0082] It should be noted that the artifact part of the image to be processed usually has fixed frequency characteristics in the frequency-domain image data, that is, within a fixed frequency range. Based on this, through the frequency-domain transformation, the image to be processed is transformed from the spatial domain to the frequency domain, and then the signal of the normal part other than the artifact part on the image to be processed can be suppressed by frequency-domain filtering, thereby realizing artifact enhancement.

[0083] Exemplarily, the frequency-domain transformation can be any one of the Fourier transform or the wavelet transform. Of course, it can also be other forms of frequency-domain transformation methods, as long as the image to be processed can be transformed from the spatial domain to the frequency domain to facilitate subsequent frequency-domain filtering. This embodiment does not make specific limitations here.

[0084] S120, filter the frequency-domain image based on a preset filter to obtain a filtered frequency-domain image.

[0085] The preset filter is a filter constructed based on the frequency characteristics of the artifact part in the image to be processed. Specifically, the filter is any one of a high-pass filter, a Gabor filter, and a band-pass filter.

[0086] Through frequency-domain filtering, only the signal of the artifact part passes through the filter, while the signal of the normal part other than the artifact part on the image to be processed is suppressed. As a result, the difference in signal strength between the artifact part and the normal part in the frequency-domain image data after filtering by the filter becomes more obvious, effectively enhancing the recognizable degree of the artifact part on the obtained filtered image, thereby realizing artifact enhancement.

[0087] S130. Perform a spatial domain transformation on the filtered frequency domain image to obtain a filtered image.

[0088] Among them, the spatial domain transformation refers to converting the image data in the frequency domain to the spatial domain. Specifically, corresponding to step S110, the frequency domain image data after frequency domain filtering is reconverted into spatial domain image data to facilitate the subsequent steps.

[0089] It should be noted that the spatial domain transformation can be any one of the Fourier transform or the wavelet transform. When the frequency domain transformation is the Fourier transform, the spatial domain transformation is the inverse Fourier transform; when the frequency domain transformation is the wavelet transform, the spatial domain transformation is the inverse wavelet transform. Of course, it can also be other forms of spatial domain transformation methods, as long as it can correspond to step S110 to convert the frequency domain image data after frequency domain filtering into spatial domain image data. This embodiment does not make specific restrictions here.

[0090] Furthermore, in order to further improve the artifact visibility of the image to be processed, this embodiment also obtains an artifact enhanced image based on the filtered image:

[0091] S140. Perform image enhancement based on the filtered image to obtain an artifact enhanced image.

[0092] Among them, image enhancement refers to improving the recognizable degree of the artifact part in the image data through algorithms or other means. Exemplarily, the image enhancement is a non-linear transformation, and the filtered image is processed by non-linear transformation to enhance the contrast of the filtered image, and the processed image is used as the artifact enhanced image.

[0093] Among them, the non-linear transformation processing includes at least one of histogram truncation or gamma transformation.

[0094] Based on this, this embodiment obtains an artifact enhanced image through frequency domain filtering and image enhancement, effectively improving the artifact visibility of the image to be processed, which is beneficial to improving the efficiency and accuracy of artifact recognition in subsequent steps.

[0095] In some other alternative embodiments of the present application, as Figure 3 shown, the method for obtaining the artifact enhanced image can also be:

[0096] S110'. Decompose the image to be processed to obtain a pyramid of the image to be processed.

[0097] Among them, the pyramid of the image to be processed is an image data structure containing multiple levels of images obtained based on the image to be processed, and each level contains an image, and the resolutions of the images at each level are different.

[0098] Specifically, the image pyramid to be processed can be a Gaussian pyramid, or the image pyramid to be processed can also be a Laplacian pyramid. It should be noted that those skilled in the art should know the method of decomposing the image to be processed to obtain a Gaussian pyramid or a Laplacian pyramid as the image pyramid to be processed, and this application will not elaborate on it here.

[0099] By obtaining the image pyramid to be processed and filtering each level image of the image pyramid to be processed based on a preset filter, the image to be processed is filtered at multiple resolutions, effectively improving the effect of image filtering and further enhancing the artifact visibility of the processed image.

[0100] S120’, perform a frequency domain transformation on each layer image in the image pyramid to be processed to obtain a frequency domain image pyramid.

[0101] S130’, filter each layer image of the frequency domain image pyramid respectively based on a preset filter to obtain a filtered frequency domain image pyramid.

[0102] S140’, perform a spatial domain transformation on each layer image of the filtered frequency domain image pyramid to obtain a filtered image pyramid.

[0103] It should be noted that by performing a frequency domain transformation, filtering, and spatial domain transformation on each level image in the image pyramid respectively, the image to be processed is filtered at multiple resolutions, thereby improving the effect of image filtering. Among them, the frequency domain transformation is any one of Fourier transform and wavelet transform, and the spatial domain transformation is any one of inverse Fourier transform and inverse wavelet transform. Among them, when the frequency domain transformation is Fourier transform, the spatial domain transformation is inverse Fourier transform; when the frequency domain transformation is wavelet transform, the spatial domain transformation is inverse wavelet transform. The preset filter is any one of high-pass filter, low-pass filter, and band-pass filter.

[0104] Specifically, for the execution manner and principle of performing a frequency domain transformation, filtering, and spatial domain transformation on each level image in the image pyramid in the above steps S120’ - S140’, please refer to steps S110 - S130, and this embodiment will not elaborate on it here.

[0105] S150’, perform image reconstruction based on each layer image of the filtered image pyramid to obtain a filtered image.

[0106] Corresponding to step S110’, through image reconstruction, add adjacent layer images of the filtered image pyramid to obtain a filtered image as the image data result after filtering the image to be processed.

[0107] Specifically, add the gray values of the corresponding pixel points of the adjacent layers of the filtered image pyramid to obtain a new image pyramid with the number of layers reduced by 1 as the new filtered image pyramid, and repeat the above step of adding the adjacent layers of images until a single-layer image is finally obtained as the filtered image, so as to obtain the image data result after filtering the image to be processed.

[0108] S160’, perform image enhancement on the filtered image to obtain an artifact-enhanced image.

[0109] Through image enhancement, further improve the visibility of artifacts in the image to be processed. Exemplarily, the image enhancement is a non-linear transformation to enhance the contrast of the filtered image through non-linear transformation processing of the filtered image, where the non-linear transformation processing includes at least one of histogram truncation or gamma transformation.

[0110] Specifically, for the implementation manner and principle of performing image enhancement on the filtered image to obtain an artifact-enhanced image, please refer to step S140, which will not be elaborated in this embodiment.

[0111] S200, based on a preset artifact recognition model, perform preliminary artifact recognition on the artifact-enhanced image to obtain all artifact regions of the artifact-enhanced image.

[0112] Among them, the artifact region is the coverage region of the artifact part on the artifact-enhanced image, which is used to represent the preliminary recognition result of the artifact. That is, the boundary of the artifact region encloses the boundary of the corresponding artifact part, and the area of the artifact region is larger than the area of the corresponding artifact part. Exemplarily, it is a square region.

[0113] The artifact recognition model is a big data model constructed based on deep learning and used to identify the artifact parts on the image. Based on the recognition result of the artifact recognition model, obtain

[0114] Specifically, based on a large amount of image data with artifact labels, train the target detection network to obtain the artifact recognition model. Among them, these image data can be artifact-enhanced images obtained based on X-ray detection images, and these image data are manually marked to obtain artifact labels.

[0115] Furthermore, in order to improve the performance of the model, this embodiment also adds an attention mechanism to the artifact recognition model. Exemplarily, this attention mechanism is an ECA lightweight attention mechanism to reduce the computing resources required by the model, thereby improving the processing efficiency of the model.

[0116] Specifically, the preset artifact recognition model is an object detection network integrating the ECA lightweight attention mechanism. By adding the ECA lightweight attention mechanism to each convolutional layer of the detection network, the performance of the obtained artifact recognition model is improved, thereby enhancing the effect of preliminary artifact recognition. Further, the object detection network includes the YOLO type or the SEE type.

[0117] It should be noted that since the artifact recognition model is trained with a large amount of image data, it has strong data processing and analysis capabilities, can accurately and efficiently obtain the artifact regions of the artifact-enhanced images, does not require manual inspection, reduces the time and labor costs, and effectively improves the accuracy and efficiency of preliminary artifact recognition, which is conducive to obtaining better artifact recognition effects.

[0118] S300. Perform spatial feature extraction on the largest connected component to obtain a sub-artifact recognition result corresponding to the artifact region.

[0119] Among them, the largest connected component is the connected component with the largest area within the corresponding artifact region. Generally speaking, due to the existence of image noise, there are more than one connected components within the artifact region. Based on this, since the artifact recognition model can identify all the artifacts on the artifact-enhanced image and obtain the corresponding artifact regions, that is, there is only one artifact in a single artifact region, by extracting the largest connected component in the artifact region, the corresponding artifact part within the artifact region can be obtained, thereby realizing the accurate recognition of artifacts.

[0120] Further, the sub-artifact recognition result is characterized by the spatial features of the largest connected component to describe the spatial information of the corresponding artifact part on the image, facilitating the subsequent artifact elimination.

[0121] Exemplarily, the sub-artifact recognition result includes the position of the artifact part on the image. As Figure 4 shown, the acquisition method of the sub-artifact recognition result includes:

[0122] S310. Based on the largest connected component, obtain the position of the largest connected component.

[0123] Specifically, construct the coordinate system of the image, and obtain the coordinates of each pixel point within the artifact region in this coordinate system as the position of the largest connected component, that is, the position of the corresponding artifact part.

[0124] S320. Based on the position of the largest connected component, obtain the sub-artifact recognition result corresponding to the artifact region.

[0125] Further, the sub-artifact recognition result further includes the artifact size of the artifact part. Specifically, the artifact size corresponding to the artifact part is characterized by the size of the largest connected component. Exemplarily, the artifact size is the area, width, or length of the corresponding largest connected component.

[0126] For ease of understanding, as Figure 5 shown below, an exemplary method for obtaining the artifact size is given, including:

[0127] S321, respectively obtain the sizes of the largest connected component in the first direction and the second direction, and use the maximum value of the sizes as the artifact size of the corresponding artifact region.

[0128] Specifically, the size of the largest connected component in the first direction is the number of pixels of the largest connected component in the first direction, and the size of the largest connected component in the second direction is the number of pixels of the largest connected component in the second direction.

[0129] Wherein, the first direction is the direction in which the pixel points on the image to be processed are arranged horizontally, and the second direction is the direction in which the pixel points on the image to be processed are arranged vertically.

[0130] S322, based on the corresponding artifact size and combined with the position of the largest connected component, to obtain the artifact recognition result of the corresponding artifact region.

[0131] That is, the artifact recognition result of a single artifact region includes the position and artifact size corresponding to the artifact region.

[0132] Of course, it is also possible to obtain the area of the largest connected component as the artifact size of the corresponding artifact region. Specifically, those skilled in the art can obtain the required artifact size according to actual needs, and the present application does not make specific limitations here.

[0133] S400, based on the sub-artifact recognition results of all artifact regions, obtain the total artifact recognition result of the image to be processed.

[0134] Based on the sub-artifact recognition results of all artifact regions on the image, the recognition results of all artifact parts on the image can be obtained, that is, the total artifact recognition result of the image to be processed is obtained. Specifically, by synthesizing the sub-artifact recognition results of all artifact regions, the total artifact recognition result is obtained, that is, the total artifact recognition result is the set of the sub-artifact recognition results of all artifact regions.

[0135] It should be noted that in the process of extracting the largest connected component within a single artifact region in step S300 for characterizing the artifact part corresponding to the artifact region, since there are strong and weak parts in the artifact part on the image, and the gray values of the weaker artifact parts are close to those of the normal parts on the image, in the process of extracting the connected components, the weaker artifact parts may not be extracted or the extraction accuracy is poor. Based on this, in order to further improve the accuracy of artifact recognition, in some alternative embodiments, such as Figure 6 shown, for a single artifact region, extracting the corresponding largest connected component includes:

[0136] S331, performing local threshold segmentation on the artifact region to obtain a binary artifact region.

[0137] Based on the gray distribution state of the artifacts within the artifact region, obtain the corresponding local threshold. Based on this local threshold, perform binarization on the artifact region. Specifically, for the pixel points within the artifact region whose gray values are greater than this local threshold, set the gray value of the corresponding pixel points in the binary artifact region to 1, and for the pixel points within the artifact region whose gray values are less than or equal to this local threshold, set the gray value of the corresponding pixel points in the binary artifact region to 0, so as to increase the gray difference between the artifact part and other normal parts within the artifact region, thereby achieving the accurate extraction of the weaker artifact parts and further achieving a better artifact recognition effect.

[0138] Exemplarily, the ISOdata segmentation algorithm is used to implement the local threshold segmentation of the artifact region.

[0139] S332, perform color inversion processing based on the binary artifact region to obtain an inverted region.

[0140] It should be noted that since the existing connected component extraction algorithms usually extract the pixel points with a gray value of 1 and connected to each other within the image as a connected component, however, since the artifact part usually appears as a black region on the image, that is, the gray value of the artifact part is relatively low, after local threshold segmentation, the gray value of the artifact part is usually 0. Based on this, in order to identify the artifact part by extracting the connected component, before extracting the connected component, it is usually necessary to perform color inversion processing on the binary artifact region.

[0141] Specifically, for each pixel point with a gray value of 1 within the artifact region, set the gray value of the corresponding pixel point in the inverted region to 0, and for each pixel point with a gray value of 0 within the artifact region, set the gray value of the corresponding pixel point in the inverted region to 1.

[0142] S333, extract all the connected components in the inverted region and obtain the connected component with the largest area as the largest connected component.

[0143] Extract all the pixels with a gray value of 1 in the inverse color region, and take the set formed by the mutually connected pixels as a connected component. Obtain the connected component with the largest area among all the connected components, that is, the connected component containing the most pixels, as the largest connected component, so as to realize the artifact recognition of this artifact region.

[0144] Based on this, the image artifact recognition method provided in this embodiment obtains the artifact region through the artifact recognition model, thereby effectively improving the efficiency and accuracy of artifact recognition. And by extracting the largest connected component in the artifact region to obtain the result of artifact recognition, the operation is simple and the accuracy is high, which is beneficial to achieving a better artifact recognition effect, so as to facilitate the progress of artifact elimination.

[0145] As Figure 7 shown, an image artifact recognition device provided in this embodiment includes an enhancement module 41, a preliminary recognition module 42, an accurate recognition module 43, and an artifact output module.

[0146] Among them, the enhancement module 41 is used to perform preprocessing of artifact enhancement on the image to be processed to obtain an artifact-enhanced image.

[0147] The preliminary recognition module 42 is used to perform preliminary artifact recognition on the artifact-enhanced image based on a preset artifact recognition model to obtain all the artifact regions of the artifact-enhanced image.

[0148] The accurate recognition module 43 is used for a single said artifact region to extract the corresponding largest connected component, perform spatial feature extraction on the largest connected component, and obtain a sub-artifact recognition result of the corresponding artifact region;

[0149] The artifact output module 44 is used to obtain the total artifact recognition result of the image to be processed based on the sub-artifact recognition results of all the artifact regions.

[0150] Based on the same technical concept, the image artifact recognition method provided in the embodiment of the present invention can be implemented on the terminal side or the server side.

[0151] As Figure 8 shown, a schematic diagram of an optional hardware structure of a terminal provided in an embodiment of the present invention. The terminal 50 may be a mobile phone, a computer device, a tablet device, a personal digital processing device, a factory background processing device, etc. The terminal 50 includes: at least one processor 51, a memory 52, at least one network interface 53, and a user interface 54. Each component in the device is coupled together through a bus system 55. It can be understood that the bus system 55 is used to realize the connection and communication between these components. The bus system 55 includes not only a data bus, but also a power bus, a control bus, and a status signal bus.

[0152] Among them, the user interface 54 may include a display, a keyboard, a mouse, a trackball, a pointing gun, a key, a button, a touchpad, or a touch screen, etc.

[0153] It can be understood that the memory 52 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM, Read Only Memory), a programmable read-only memory (PROM, Programmable Read-Only Memory), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM, Static Random Access Memory), synchronous static random access memory (SSRAM, Synchronous Static Random Access Memory). The memory characterized in the embodiments of the present invention is intended to include but not limited to these and any other suitable categories of memory.

[0154] The memory 52 in the embodiments of the present invention is used to store various categories of data to support the operation of the terminal. Examples of these data include: any executable program for operating on the terminal 50, such as the operating system 521 and the application program 522; the operating system 521 contains various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application program 522 can include various application programs, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. Implementing the image artifact recognition method provided by the embodiments of the present invention can be included in the application program 522.

[0155] The method disclosed in the embodiments of the present invention above can be applied to the processor 51 or implemented by the processor 51. The processor 51 may be an integrated circuit chip with the ability to process signals. During implementation, the steps of the above method can be completed by the integrated logic circuit in hardware or instructions in software form in the processor 51. The above processor may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 51 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The processor 51 may be a microprocessor or any conventional processor, etc. Combining with the steps of the accessory optimization method provided by the embodiments of the present invention, it can be directly embodied as being completed by the hardware decoding processor, or completed by the combination of hardware and software modules in the decoding processor. The software module may be located in the storage medium, and this storage medium is located in the memory. The processor reads the information in the memory and combines its hardware to complete the steps of the foregoing method.

[0156] In an exemplary embodiment, the terminal 50 may be an application-specific integrated circuit (ASIC), a DSP, a programmable logic device (PLD), or a complex programmable logic device (CPLD) for executing the foregoing method.

[0157] The embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, and when the program is called by a processor, it implements the image artifact recognition method provided by the present invention.

[0158] Among them, the computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. The computer-readable storage medium may be, for example (but not limited to), an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding device.

[0159] The computer-readable programs represented herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network adapter or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0160] In summary, through the preset artifact recognition model, the present application accurately and efficiently obtains the artifact regions in the to-be-processed push, and obtains the artifact recognition result by obtaining the maximum connected domain of all artifact regions, without the need for manual inspection of the artifacts on the image, effectively reducing the labor and time costs, being simple and easy to operate, having accurate recognition results and high recognition efficiency, and having high industrial application value.

[0161] The descriptions of the processes or structures corresponding to the above-mentioned respective drawings each have their own focuses. For parts not detailed in a certain process or structure, reference can be made to the relevant descriptions of other processes or structures.

[0162] The above embodiments are only illustrative of the principles and effects of the present application, and are not used to limit the present application. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed by the present application should still be covered by the claims of the present application.

Claims

1. A method for identifying image artifacts, comprising: Performing artifact enhancement preprocessing on the image to be processed to obtain an artifact enhanced image; Based on a preset artifact recognition model, preliminary artifact recognition is performed on the artifact-enhanced image to obtain each artifact region in the artifact-enhanced image; For a single artifact region, extract the corresponding maximum connected domain, perform spatial feature extraction on the maximum connected domain, and obtain a sub-artifact recognition result corresponding to the artifact region; Based on the sub-artifact recognition results of all the artifact regions, a total artifact recognition result of the image to be processed is obtained.

2. The image artifact recognition method according to claim 1, characterized in that: The preset artifact recognition model is a target detection network integrating an ECA lightweight attention mechanism; the target detection network includes a YOLO type or a SEE type.

3. The image artifact recognition method according to claim 1, characterized in that: For a single artifact region, extracting the corresponding maximum connected domain includes: Performing local threshold segmentation on the artifact area to obtain a binary artifact area; Performing color inversion processing based on the binary artifact area to obtain an inverted color area; The connected domains in the inverted color region are extracted, and the connected domain with the largest area is obtained as the maximum connected domain.

4. The image artifact recognition method according to claim 1, characterized in that: The preprocessing of the image to be processed with artifact enhancement to obtain an artifact enhanced image includes: Performing frequency domain transformation on the image to be processed to obtain a frequency domain image; Filtering the frequency domain image based on a preset filter to obtain a filtered frequency domain image; Performing a spatial domain transform on the filtered frequency domain image to obtain a filtered image; Image enhancement is performed based on the filtered image to obtain the artifact enhanced image.

5. The image artifact recognition method according to claim 1, characterized in that: The preprocessing of the image to be processed with artifact enhancement to obtain an artifact enhanced image includes: Decomposing the image to be processed to obtain a pyramid of the image to be processed; Performing frequency domain transformation on each layer of the image pyramid to be processed to obtain a frequency domain image pyramid; Based on a preset filter, each layer image of the frequency domain image pyramid is filtered to obtain a filtered frequency domain image pyramid; Performing spatial domain transformation on each layer image of the filtered frequency domain image pyramid to obtain a filtered image pyramid; Reconstructing the image based on the images of each layer of the filter image pyramid to obtain a filter image; Image enhancement is performed based on the filtered image to obtain the artifact enhanced image.

6. The image artifact recognition method according to any one of claims 4 to 5, characterized in that: The performing image enhancement based on the filtered image to obtain the artifact enhanced image includes: performing nonlinear transformation processing on the filtered image to enhance the contrast of the filtered image, and using the processed image as the artifact enhanced image; The nonlinear transformation processing includes at least one of histogram truncation and gamma transformation.

7. The image artifact recognition method according to claim 1, characterized in that: The performing of spatial feature extraction on the maximum connected domain to obtain a sub-artifact recognition result corresponding to the artifact area includes: Based on the maximum connected domain, obtaining a position of the maximum connected domain; Based on the position of the maximum connected domain, a sub-artifact recognition result corresponding to the artifact area is obtained.

8. The method for identifying image artifacts according to claim 7, characterized in that: The obtaining, based on the position of the maximum connected domain, a sub-artifact recognition result corresponding to the artifact area includes: Obtaining the sizes of the maximum connected domain in the first direction and the second direction respectively, and taking the maximum value of the sizes as the artifact size of the corresponding artifact area; Based on the size of the artifact and in combination with the position of the maximum connected domain, obtaining an artifact recognition result corresponding to the artifact area; The first direction is the direction in which each pixel point on the image to be processed is arranged horizontally, and the second direction is the direction in which each pixel point on the image to be processed is arranged vertically.

9. An image artifact recognition device, characterized in that: It includes an enhancement module, a preliminary recognition module, an accurate recognition module and an artifact output module; The enhancement module is used to perform artifact enhancement preprocessing on the image to be processed to obtain an artifact enhanced image; The preliminary identification module is used to perform preliminary artifact identification on the artifact-enhanced image based on a preset artifact identification model to obtain all artifact regions of the artifact-enhanced image; The precise identification module is used to extract the corresponding maximum connected domain for a single artifact region, perform spatial feature extraction on the maximum connected domain, and obtain a sub-artifact identification result corresponding to the artifact region; The artifact output module is used to obtain the total artifact recognition result of the image to be processed based on the sub-artifact recognition results of all the artifact areas.

10. A terminal, characterized in that: include: A processor and a memory, wherein the memory is communicatively connected to the processor; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the image artifact recognition method according to any one of claims 1 to 8.

11. A computer storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image artifact recognition method according to any one of claims 1 to 8 is implemented.

Images