Image processing method and device and computer equipment

Abstract

The invention discloses an image processing method and device and computer equipment. The method comprises the steps that truncation processing is carried out on gray levels with the frequency number larger than a target truncation threshold value in a gray level histogram of a target gray level image, a sparse gray level histogram of the target gray level image is generated, the target gray level image comprises a saliency area of a to-be-detected object, and the target gray level image meets the preset gray level number; carrying out random sampling processing on the gray level of the target gray level image based on the sparse gray level histogram, and generating a sparse image corresponding to the sampling gray level; and filtering the sparse image to generate a saliency image of the to-be-detected object. According to the method, the dense region can be suppressed, interference of the dense region is avoided, low-contrast change of the local region can be detected more easily, the accuracy of image saliency detection is improved, and reliable data support is provided for subsequent analysis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual saliency detection, and in particular to an image processing method and device and computer equipment. BACKGROUND

[0002] Saliency detection is an important part of computer vision, aiming to detect and highlight the part attracting attention from the image. In the image preprocessing stage, key information can be extracted, data redundancy can be reduced, and system efficiency can be improved. In complex scenes and multi-source data fusion, the performance of the visual system can be improved.

[0003] In the existing saliency detection method, it is considered that the pixels with higher change frequency in the image represent the saliency target. However, this method has low distinguishability for object edges and textures and is not sensitive to subtle changes in local areas, so that the saliency detection cannot meet the expected requirements. SUMMARY

[0004] Therefore, the present application provides an image processing method, device and computer equipment, which can amplify the high gradient gray level related to the object edge and texture, so that the subtle changes are easier to detect.

[0005] According to one aspect of the present application, an image processing method is provided, comprising:

[0006] Truncating the gray levels with a frequency greater than a target truncation threshold in a gray histogram of a target gray image to generate a sparse gray histogram of the target gray image, wherein the target gray image includes a saliency region of a to-be-detected object, and the target gray image satisfies a preset number of gray levels;

[0007] Randomly sampling the gray levels of the target gray image based on the sparse gray histogram to generate a sparse image corresponding to the sampled gray levels;

[0008] Filtering the sparse image to generate a saliency image of the to-be-detected object.

[0009] Optionally, the image processing method further comprises:

[0010] If the to-be-processed gray image does not satisfy the preset number of gray levels, determining a gray level control unit according to a gray histogram of the gray image;

[0011] Downwardly rounding the quotient of the gray value of the gray image and the gray level control unit to obtain a mapped gray value;

[0012] Multiplying the mapped gray value and the gray level control unit to generate the target gray image.

[0013] Optionally, the determining the gray scale control unit according to the gray histogram of the gray image comprises:

[0014] determining the number of gray values with a frequency of zero in the gray image according to the gray histogram;

[0015] determining the gray scale control unit based on the quotient of the number and the preset number of gray scales.

[0016] Optionally, the random sampling processing of the gray scale of the target gray image based on the sparse gray histogram comprises:

[0017] selecting at least part of the gray scales in the sparse gray histogram as the sampling gray scales based on a random algorithm;

[0018] determining the sampling proportion of each sampling gray scale based on the number of pixels of each sampling gray scale in the sparse gray histogram;

[0019] randomly selecting the pixels corresponding to each sampling gray scale in the target gray image as the sampling pixels corresponding to the sampling gray scale based on the sampling proportion;

[0020] synthesizing the sparse image based on the sampling pixels.

[0021] Optionally, the image processing method further comprises:

[0022] performing a truncation processing on the gray histogram of the sample gray image based on different candidate truncation thresholds to generate a sample histogram;

[0023] calculating the entropy change value of the sample histogram compared with the gray histogram of the sample gray image, and the variance value of the gray scale in the sample histogram;

[0024] inputting the entropy change value and the variance value corresponding to different candidate truncation thresholds into a preset evaluation function respectively, and solving the preset evaluation function with the goal of maximizing the output to determine the target truncation threshold from the candidate truncation thresholds.

[0025] Optionally, the image processing method further comprises:

[0026] determining the edge pixel proportion of the object to be measured in the target gray image based on an edge detection algorithm;

[0027] if the edge pixel proportion is less than a proportion threshold, increasing the target truncation threshold;

[0028] if the edge pixel proportion is greater than the proportion threshold, decreasing the target truncation threshold.

[0029] Optionally, the image processing method further comprises:

[0030] displaying a gray scale configuration module and a preview area of the target gray scale image;

[0031] in response to a sliding operation on the gray scale configuration module, processing the target gray scale image based on a truncation threshold indicated by the sliding operation to generate a preview image, and displaying the preview image in the preview area, wherein a first pixel and a second pixel in the preview image are displayed differently, a gray scale value of the first pixel is within a gray scale interval corresponding to the truncation threshold indicated by the sliding operation, and a gray scale value of the second pixel is outside the gray scale interval corresponding to the truncation threshold indicated by the sliding operation;

[0032] in response to a configuration instruction, taking a truncation threshold corresponding to a current preview image as the target truncation threshold.

[0033] Optionally, the image processing method further comprises:

[0034] determining a gray scale threshold based on a mean value of a histogram array of the target gray scale image;

[0035] performing denoising processing on the sparse image based on the gray scale threshold.

[0036] According to another aspect of the present application, an image processing device is provided, comprising:

[0037] a truncation module configured to perform truncation processing on a gray scale level in a gray scale histogram of a target gray scale image whose frequency is greater than a target truncation threshold, to generate a sparse gray scale histogram of the target gray scale image, wherein the target gray scale image comprises a saliency region of a to-be-detected object, and the target gray scale image satisfies a preset number of gray scale levels;

[0038] a sampling module configured to perform random sampling processing on a gray scale level of the target gray scale image based on the sparse gray scale histogram, to generate a sparse image corresponding to the sampled gray scale level;

[0039] a filtering module configured to perform filtering processing on the sparse image, to generate a saliency image of the to-be-detected object.

[0040] Optionally, the image processing device further comprises:

[0041] a mapping module configured to, if a to-be-processed gray scale image does not satisfy the preset number of gray scale levels, determine a gray scale level control unit according to a gray scale histogram of the gray scale image, perform down-round processing on a quotient of a gray scale value of the gray scale image and the gray scale level control unit to obtain a mapped gray scale value, and multiply the mapped gray scale value and the gray scale level control unit to generate the target gray scale image.

[0042] Optionally, the mapping module is specifically configured to determine the number of gray scale values with non-zero frequency in the gray scale image according to the gray scale histogram; and determine the gray scale control unit based on the quotient of the number and the preset number of gray scales.

[0043] Optionally, the sampling module is specifically configured to select at least part of the gray scales from the sparse gray scale histogram as sampling gray scales based on a random algorithm; determine a sampling ratio of each sampling gray scale based on the number of pixels of each sampling gray scale in the sparse gray scale histogram; randomly select the pixels corresponding to each sampling gray scale in the target gray scale image as the sampling pixels corresponding to the sampling gray scale based on the sampling ratio; and synthesize the sparse image based on the sampling pixels.

[0044] Optionally, the image processing apparatus further comprises:

[0045] The first configuration module is configured to perform truncation processing on a gray scale histogram of a sample gray scale image based on different candidate truncation thresholds to generate a sample histogram; calculate an entropy change value of the sample histogram compared with a gray scale histogram of the sample gray scale image and a variance value of the gray scales in the sample histogram; input the entropy change value and the variance value corresponding to different candidate truncation thresholds into a preset evaluation function respectively, and solve the preset evaluation function with the goal of maximizing the output to determine the target truncation threshold from the candidate truncation thresholds.

[0046] Optionally, the image processing apparatus further comprises:

[0047] The edge detection module is configured to determine the edge pixel proportion of the object to be detected in the target gray scale image based on an edge detection algorithm.

[0048] The adjustment module is configured to increase the target truncation threshold if the edge pixel proportion is less than a proportion threshold, and decrease the target truncation threshold if the edge pixel proportion is greater than the proportion threshold.

[0049] Optionally, the image processing apparatus further comprises:

[0050] The display module is configured to display the gray scale configuration module and a preview area of the target gray scale image.

[0051] The preview module is configured to generate a preview image by processing the target gray scale image based on a gray scale interval indicated by a sliding operation on the gray scale configuration module in response to the sliding operation.

[0052] The display module is further configured to display the preview image in the preview area, wherein a first pixel and a second pixel in the preview image are displayed differently, a gray value of the first pixel is located in a gray interval corresponding to the cut-off threshold indicated by the sliding operation, and a gray value of the second pixel is located outside the gray interval corresponding to the cut-off threshold indicated by the sliding operation.

[0053] The third configuration module is configured to, in response to a configuration instruction, take the cut-off threshold corresponding to the current preview image as the target cut-off threshold.

[0054] Optionally, the image processing device further comprises:

[0055] The denoising module is configured to determine a gray threshold based on a mean value of the histogram array of the target gray image, and perform denoising processing on the sparse image based on the gray threshold.

[0056] According to still another aspect of the present application, a readable storage medium is provided, which has a program or instruction stored thereon, and the program or instruction is executed by a processor to implement the steps of the image processing method.

[0057] According to yet another aspect of the present application, a computer device is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the image processing method when executing the program.

[0058] By means of the above technical solution, the frequency of a gray level with a frequency greater than the target cut-off threshold in the target gray image is adjusted to the target cut-off threshold through cut-off processing, so as to retain high gradient gray levels related to object edges and textures, and highlight the saliency region of the object to be detected. Then, the pixels of the same gray level are extracted from the target gray image through sparse sampling, so that the gray difference between the sampled pixels and the remaining pixels is relatively more significant in the sparse image, the non-uniform distribution characteristics of the edges of the object to be detected are maintained, and the sparse image can reflect the approximate outline of the object to be detected. Finally, the pixels of the sparse non-zero points are smoothed by using Gaussian filtering, the local continuity is strengthened, the subtle changes are more coherent, and the complete outline of the object to be detected is formed. Thus, the dense region can be suppressed, the interference of the dense region is avoided, the low contrast changes in the local region are easier to be detected, the accuracy of the saliency detection of the image is improved, and reliable data support is provided for improving subsequent analysis.

[0059] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the specific embodiments of the present application can be implemented in accordance with the content of the description, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0060] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0061] Figure 1 A flow diagram of an image processing method provided by an embodiment of the application is shown;

[0062] Figure 2 A contrast diagram of a saliency image and an original image provided by an embodiment of the application is shown;

[0063] Figure 3 Another contrast diagram of a saliency image and an original image provided by an embodiment of the application is shown;

[0064] Figure 4 A structure block diagram of an image processing apparatus provided by an embodiment of the application is shown;

[0065] Figure 5 A structure diagram of a computer device provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0066] The application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0067] The embodiments of the application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, in which the same or similar notations represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are only used to explain the application, and cannot be interpreted as a limitation on the application.

[0068] Those skilled in the art can understand that the singular forms "a", "an" and "the" used herein include plural forms unless specifically stated otherwise. It should be further understood that the use of the term "include" in the specification of the application means that the features, integers, steps, operations, elements and / or components described in the specification exist, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we say that an element is "connected" or "joined" to another element, it can be directly connected or joined to the other element, or there can be intermediate elements. In addition, "connected" or "joined" used herein can include wireless connection or wireless connection. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0069] Exemplary embodiments according to this application will now be described in more detail with reference to the accompanying drawings. These exemplary embodiments may, however, be implemented in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the inventive concept of these exemplary embodiments to those skilled in the art.

[0070] The image processing method provided in the embodiments of the present application can be applied to a terminal, can be applied to a server, and can also be software running in the terminal or the server. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.; the server can be configured as a stand-alone physical server, can be configured as a server cluster or a distributed system composed of multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform; and the software can be an application program implementing the image processing method, but is not limited to the above forms.

[0071] In the embodiments of the present application, an image processing method is provided, as shown in Figure 1 The method comprises the following steps.

[0072] In step 101, the gray levels with a frequency greater than a target cutoff threshold in the gray histogram of a target gray image are subjected to a cutoff processing, so as to generate a sparse gray histogram of the target gray image.

[0073] The target gray image is obtained by subjecting an original image to a gray processing, and satisfies a preset number of gray levels. The original image can be an infrared image or a color photo, which is not limited in the embodiments of the present application. The target gray image includes a saliency region of a to-be-detected object, and is used to highlight the difference between the to-be-detected object and the background, so that the saliency region is more easily recognized by an algorithm or the human eye. The gray histogram is a distribution graph of the frequency of each gray level pixel in an image, the horizontal axis of which is a gray level marked with a gray value, and the vertical axis of which is the frequency (frequency number). The gray histogram is used to reflect the brightness distribution, contrast, and noise condition of the image.

[0074] In this embodiment, the gray levels with a high frequency in the histogram are subjected to a cutoff processing, so that the gray levels are retained only for the pixels of the target cutoff threshold. Thus, the histogram is controlled to suppress the dense region, the interference of the dense region such as a large-area background or a non-key region is avoided, the gray feature of the to-be-detected object is more highlighted, and the stability of the detection result is enhanced. Moreover, due to the reduction of the frequency of the gray level, the weight of the gray level in the subsequent processing is reduced, the calculation amount of the saliency detection is reduced, and the detection accuracy and efficiency are improved.

[0075] In an embodiment, the following scheme can be adopted for configuring the target gray interval:

[0076] In a first mode, the gray histogram of the sample gray image is truncated based on different candidate truncation thresholds to generate a sample histogram; an entropy variation value of the sample histogram compared with the gray histogram of the sample gray image and a variance value of the gray levels in the sample histogram are calculated; the entropy variation value and the variance value corresponding to different candidate truncation thresholds are respectively input into a preset evaluation function, and the preset evaluation function is solved with the goal of maximizing the output to determine the target truncation threshold from the candidate truncation thresholds.

[0077] The sample gray image can be the target gray image or another image homologous to the target gray image.

[0078] In this embodiment, the evaluation function is solved by the entropy variation value and the variance value, so that the truncation threshold is dynamically adjusted according to the histogram characteristics of the sample image, and the quantifiable information retention degree and the dispersion degree of the gray level frequency are taken into account. Further, it is ensured that the key gray features such as the frequency distribution corresponding to the edges and textures of the salient region are not excessively truncated, and the sparse histogram has sufficient distribution distinguishability.

[0079] Specifically, the entropy variation value is used to quantify the change rate of the global structural information of the image before and after truncation, and can be calculated according to the total entropy (S original ) of the gray histogram of the sample gray image and the entropy (S sparse ) of the sample histogram after truncation. The calculation formula is entropy variation ΔS = |S original -S sparse | / S original . The variance value can be calculated according to the probability distribution of the retained gray levels of the sample histogram after truncation. The evaluation function expression is as follows:

[0080] F = α × ΔS + β × σ 2 ;

[0081] In the formula, ΔS is the entropy variation value, σ 2 is the variance, α and β are weight coefficients, α + β = 1, and α and β can be calibrated by experiments.

[0082] Further, the image processing method further includes: determining the edge pixel proportion of the object to be measured in the target gray image based on an edge detection algorithm; if the edge pixel proportion is less than a proportion threshold, increasing the target truncation threshold; and if the edge pixel proportion is greater than the proportion threshold, decreasing the target truncation threshold.

[0083] The edge pixel proportion is used to reflect the outline clarity of the object to be measured and the background.

[0084] In this embodiment, edge detection is performed on the significant object to be detected in the image. When the proportion of edge pixels of the object to be detected is lower than a threshold value, it indicates that the edge of the object to be detected in the target gray image is blurred, and then the target cutoff threshold value is increased to further sparsify, so as to avoid too much background gray being included and reduce the background interference. When the proportion of edge pixels is higher than the threshold value, it indicates that the edge of the object to be detected in the image is sharp and easy to distinguish, and then the target cutoff threshold value can be appropriately increased to retain more gray levels to maintain the edge structure and avoid contour breakage. Thus, through dynamic adjustment, the target cutoff threshold value is always matched with the actual contour feature of the object, the false detection and missed detection caused by too large or too small target cutoff threshold value are reduced, the distinguishability of the object edge and texture is improved, and the significant detection can still stably capture the target in a complex background.

[0085] In a second mode, a gray configuration module and a preview area of a target gray image are displayed; in response to a sliding operation on the gray configuration module, the target gray image is processed based on a gray interval indicated by the sliding operation to generate a preview image, and the preview image is displayed in the preview area; and in response to a configuration instruction, a cutoff threshold value corresponding to the current preview image is taken as a target cutoff threshold value.

[0086] In the preview image, the first pixel and the second pixel are differentially displayed. The gray value of the first pixel is located in the gray interval corresponding to the cutoff threshold value indicated by the sliding operation, and the gray value of the second pixel is located outside the gray interval corresponding to the cutoff threshold value indicated by the sliding operation. For example, the preview image is a binary image, in which case the first pixel is assigned white color and the second pixel is assigned black color; or the preview image presents a stepped effect in the significant area, in which case the first pixel is assigned white color and the second pixel (a pixel corresponding to the gray interval) is automatically matched in a color interval obviously different from white color according to the actual gray value of the second pixel.

[0087] Specifically, the gray configuration module can be a sliding bar, an interval selector or the like.

[0088] It can be understood that the gray configuration module can be directly used to adjust the target gray interval. The gray configuration module can also act on the cutoff threshold value and indirectly adjust the target gray interval through the cutoff threshold value.

[0089] In this embodiment, the abstract gray interval parameter is converted into a visual interactive control, and a user can directly adjust the gray range through a sliding operation until the best cutoff effect is achieved without manually inputting a value. The processing effect is displayed in real time in the preview area, so that the user can intuitively feel the influence of the gray range change on the cutoff effect of the target gray image. Even a non-professional user can quickly determine the target gray interval that accurately covers the target feature, greatly improving the flexibility of interval determination to better adapt to different detection requirements.

[0090] In an embodiment, the image processing method before step 101 further comprises: determining whether the to-be-processed gray-scale image satisfies a preset number of gray scales; if the to-be-processed gray-scale image satisfies the preset number of gray scales, taking the to-be-processed gray-scale image as a target gray-scale image; if the to-be-processed gray-scale image does not satisfy the preset number of gray scales, determining a gray-scale control unit according to a gray-scale histogram of the gray-scale image; performing down-rounding processing on a quotient of a gray-scale value of the gray-scale image and the gray-scale control unit to obtain a mapped gray-scale value; and multiplying the mapped gray-scale value and the gray-scale control unit to generate the target gray-scale image.

[0091] In this embodiment, when the to-be-processed gray-scale image does not satisfy the preset number of gray scales, the total number of gray scales of the image is controlled according to histogram analysis of the gray-scale image. Thus, low-information gray scales are compressed, so as to concentrate the gray-scale distribution of the target region and make the histogram peak value of the saliency region more prominent. In subsequent sparsification, high-frequency gray scales (corresponding to edges) can still retain sufficient pixels through random sampling due to the dispersed distribution, and the low-contrast difference of a local region is relatively more prominent in the sparse image, avoiding edge blurring caused by uniform quantization. Moreover, the gray-scale images of different sources are unified to the same gray-scale range, ensuring that the images collected by different cameras or sensors have consistent gray-scale representation, improving the scene adaptability of saliency detection, reducing the local debugging cost, and being helpful for batch processing of images.

[0092] Specifically, the gray-scale control unit is determined according to the gray-scale histogram of the gray-scale image, specifically comprising: determining the number of gray-scale values with a frequency of zero in the gray-scale image according to the gray-scale histogram; and determining the gray-scale control unit based on a quotient of the number and the preset number of gray scales.

[0093] In this embodiment, the mapping unit is dynamically calculated based on the effective number of gray scales and the preset number of gray scales. Not only can more similar gray scales be combined to avoid increasing the amount of calculation due to redundant information, but also the loss of details caused by excessive combination can be reduced, and more original gray-scale differences can be retained. This ensures that subsequent processing links can be based on the expected size of the gray scale, avoiding the influence of unstable input dimensions on the performance of the algorithm.

[0094] For example, the number of gray scales of the gray-scale image is divided by the preset number of gray scales N1 (for example, N1 = 32 or 75), and the result is rounded down to obtain a mapping unit L of the gray scale.

[0095]

[0096] The gray-scale value of the gray-scale image is then divided by the control unit L of the gray scale, the image gray-scale value is compressed to the preset number of gray scales N1, and the value is mapped back to the value range of the original image to form a target gray-scale image with the preset number of gray scales.

[0097]

[0098] In an embodiment, after the target gray image is generated, a de-noising process can be performed on the target gray image. The de-noising threshold can be calculated in proportion to the mean value of the histogram array. Not only can the de-noising process remove possible noise points, but also can avoid misjudgment of low-contrast targets caused by a fixed threshold.

[0099] The mean value of the histogram array is used to represent the trend of the image gray distribution, that is, a weighted average value of all elements (frequencies of each gray level) in the array is calculated for each gray level. For example, the histogram array of a certain image is simplified as hist = [120, 80, 50, 30], the gray level range is [0, 3], that is, the number of gray levels is 4, and the corresponding frequencies of the gray levels are 120, 80, 50, and 30, respectively; the total number of pixels = 120 + 80 + 50 + 30 = 280, and the weighted value = 0 x 120 + 1 x 80 + 2 x 50 + 3 x 30 = 270; the mean value = 270 / 280 = 0.97.

[0100] Specifically, the preset weight of the de-noising threshold can be reasonably set according to the detection accuracy, which is not limited in the embodiments of the present application. For example, the preset weight is in the range of 0.001-0.05, and the de-noising threshold can be set as 0.03μ, 0.0098μ, 0.01μ, etc., where μ is the mean value of the histogram array. The histogram gray level data with a gray value greater than or equal to the de-noising threshold is retained, and the histogram data corresponding to the gray value less than the de-noising threshold is set to zero, so as to reduce the influence of noise points on subsequent processing.

[0101] In step 102, the gray levels of the target gray image are randomly sampled based on the sparse gray histogram to generate a sparse image corresponding to the sampled gray levels.

[0102] In this embodiment, random sampling is used to further compress the image while ensuring information integrity, so that the sparse image ignores low-probability gray levels and retains high-probability gray levels, weakens background interference, and facilitates subsequent feature extraction.

[0103] In an embodiment, step 102, that is, the random sampling process of the gray levels of the target gray image based on the sparse gray histogram to generate a sparse image corresponding to the sampled gray levels, specifically includes the following steps:

[0104] In step 102-1, at least part of the gray levels in the sparse gray histogram are selected as the sampled gray levels based on a random algorithm.

[0105] In step 102-2, the sampling proportion of each sampled gray level is determined based on the number of pixels of each sampled gray level in the sparse gray histogram.

[0106] Step 102-3, randomly select the pixel corresponding to each sampling gray level in the target gray image as the sampling pixel corresponding to the sampling gray level based on the sampling ratio.

[0107] Step 102-4, synthesize the sparse image based on the sampling pixels.

[0108] In this embodiment, the sampling ratio is adaptively allocated by the gray level frequency, so that the number of sampling points is proportional to the number of gray levels retained in the sparse gray histogram. Thus, high-frequency gray levels (usually corresponding to the saliency region of the object to be detected) are given more sampling opportunities, and low-frequency gray levels are appropriately reduced. This ensures that the statistical significance of high-frequency features (such as edges) is not diluted, and the discriminative features of the object to be detected and the background are not lost while reducing the dimension. Moreover, the random selection of sampling pixels introduces controllable randomness, which can simulate the gray fluctuations caused by light and noise, reduce the dependence of the algorithm on local abnormal pixels, improve the robustness of the model in complex scenes, and avoid overfitting.

[0109] For example, the ratio of the truncated sparse gray histogram to the original gray histogram, i.e., the proportion of the gray levels retained in the sparse gray histogram in the total number of pixels in the original image (sampling ratio), is calculated, which represents the proportion of pixels that each gray level needs to retain. For each gray level, locate the pixel position corresponding to the gray level and count the number of pixels, and according to the sampling ratio, randomly select a corresponding number of pixels from these pixels; initialize a sparse image with all zeros, assign the randomly selected pixels to the corresponding positions in the sparse image with a value of 1, and keep other positions as 0, to generate a sparse image.

[0110] Step 103, filtering the sparse image to generate a saliency image of the object to be detected.

[0111] Specifically, a Gaussian filter or a mean filter can be applied to the sparse image for smoothing filtering to remove noise and highlight the salient region.

[0112] The image processing method provided in the embodiments of the present application first adjusts the frequency of the gray level whose frequency is greater than the target cut-off threshold in the target gray image to the target cut-off threshold through truncation processing, so as to retain the high gradient gray level related to the object edge and texture and highlight the saliency region of the object to be detected. Then, the pixels of the same gray level are extracted from the target gray image through sparse sampling, so that the gray difference between the sampled pixel corresponding to the gray level and the remaining pixels is relatively more significant in the sparse image, the non-uniform distribution characteristics of the edge of the object to be detected are retained, and the sparse image can reflect the approximate contour of the object to be detected. Finally, the pixels of the sparse non-zero points are smoothed by using the Gaussian filter, the local continuity is strengthened, the subtle changes are more coherent, and the complete contour of the object to be detected is formed. Thus, the dense region can be suppressed, the interference of the dense region is avoided, the low contrast change in the local region is more easily detected, the accuracy of the saliency detection of the image is improved, and reliable data support is provided for improving subsequent analysis.

[0113] It can be understood that, due to the local heating caused by poor contact, aging or overload of high-voltage equipment (such as transformers, cable joints, insulators and the like) of the power supply system in long-term operation. The original image can be the temperature distribution of the equipment captured by the infrared thermal imaging technology. The embodiments quickly extract the saliency region of the equipment heating through sparse gray histogram processing, and improve the fault detection efficiency. Alternatively, in the scene of quality inspection of the equipment structure, the cracks and rust of the metal parts are detected through the image to ensure the safety of the equipment. The embodiments retain the defect feature gray level through sparse gray histogram processing, reduce the data amount of the high-resolution industrial camera, and improve the quality inspection efficiency.

[0114] As shown in Figure 2 and Figure 3 , the saliency detection results of the experimental images of the embodiments are shown, wherein Figure 2 (a) and Figure 3 (a) is the original image, Figure 2 (b) and Figure 3 (b) is the saliency detection result image. As can be seen from the effect image after the experiment, the present application provides an image saliency detection method based on histogram analysis. The dense region is suppressed through truncation control, the interference of the dense region is avoided, and the accuracy of the target detection is improved.

[0115] It should be noted that the size of the serial number of each step in the above embodiments does not mean the execution order, the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0116] Further, as Figure 4As shown, as a specific implementation of the image processing method, the embodiment of the present application provides an image processing device 200, which comprises a truncation module 201, a sampling module 202 and a filtering module 203.

[0117] The truncation module 201 is configured to perform truncation processing on a gray level with a frequency greater than a target truncation threshold in a gray histogram of a target gray image to generate a sparse gray histogram of the target gray image, wherein the target gray image comprises a saliency region of a to-be-detected object, and the target gray image satisfies a preset number of gray levels.

[0118] The sampling module 202 is configured to perform random sampling processing on the gray levels of the target gray image based on the sparse gray histogram to generate a sparse image corresponding to the sampled gray levels.

[0119] The filtering module 203 is configured to perform filtering processing on the sparse image to generate a saliency image of the to-be-detected object.

[0120] In this embodiment, the frequency of the gray level with the frequency greater than the target truncation threshold in the target gray image is adjusted to the target truncation threshold through the truncation processing, so as to retain the high gradient gray levels related to the object edge and texture and highlight the saliency region of the to-be-detected object. Then, the pixels of the same gray level are extracted from the target gray image through the sparse sampling, so that the gray difference between the pixels corresponding to the sampled gray levels and the remaining pixels is relatively more significant in the sparse image, the non-uniform distribution characteristics of the edge of the to-be-detected object are maintained, and the sparse image can reflect the approximate outline of the to-be-detected object. Finally, the pixels of the sparse non-zero points are smoothed through the Gaussian filtering, the local continuity is strengthened, the subtle changes are more coherent, and the complete outline of the to-be-detected object is formed. Thus, the dense region can be suppressed, the interference of the dense region is avoided, the low contrast changes in the local region are more easily detected, the accuracy of the image saliency detection is improved, and reliable data support is provided for improving the subsequent analysis.

[0121] Further, the image processing device 200 further comprises:

[0122] a mapping module (not shown in the figure) configured to: if the to-be-processed gray image does not satisfy the preset number of gray levels, determine a gray level control unit according to the gray histogram of the gray image; perform down-round processing on a quotient of the gray value of the gray image and the gray level control unit to obtain a mapped gray value; and multiply the mapped gray value and the gray level control unit to generate the target gray image.

[0123] Further, the mapping module is specifically configured to: determine the number of gray values with a non-zero frequency in the gray image according to the gray histogram; and determine the gray level control unit based on a quotient of the number and the preset number of gray levels.

[0124] Further, the sampling module 202 is specifically configured to: select at least part of the gray scales from the sparse gray scale histogram as sampling gray scales based on a random algorithm; determine a sampling ratio of each sampling gray scale based on a number of pixels of each sampling gray scale in the sparse gray scale histogram; randomly select a pixel corresponding to each sampling gray scale in the target gray scale image as a sampling pixel corresponding to the sampling gray scale based on the sampling ratio; and synthesize the sparse image based on the sampling pixels.

[0125] Further, the image processing apparatus 200 further comprises:

[0126] The first configuration module (not shown in the figure) is configured to: perform truncation processing on a gray scale histogram of the sample gray scale image based on different candidate truncation thresholds to generate a sample histogram; calculate an entropy change value of the sample histogram compared with the gray scale histogram of the sample gray scale image and a variance value of the gray scales in the sample histogram; input the entropy change value and the variance value corresponding to the different candidate truncation thresholds into a preset evaluation function respectively, and solve the preset evaluation function with the goal of maximizing the output, so as to determine the target truncation threshold from the candidate truncation thresholds.

[0127] Further, the image processing apparatus 200 further comprises:

[0128] The edge detection module (not shown in the figure) is configured to determine an edge pixel proportion of the object to be detected in the target gray scale image based on an edge detection algorithm.

[0129] The adjustment module (not shown in the figure) is configured to: increase the target truncation threshold if the edge pixel proportion is less than a proportion threshold; and decrease the target truncation threshold if the edge pixel proportion is greater than the proportion threshold.

[0130] Further, the image processing apparatus 200 further comprises:

[0131] The display module (not shown in the figure) is configured to display the gray scale configuration module and the preview area of the target gray scale image.

[0132] The preview module (not shown in the figure) is configured to: in response to a sliding operation on the gray scale configuration module, process the target gray scale image based on a gray scale interval indicated by the sliding operation to generate a preview image.

[0133] The display module is further configured to display the preview image in the preview area, wherein the first pixel and the second pixel in the preview image are displayed differently, the gray scale value of the first pixel is located in a gray scale interval corresponding to the truncation threshold indicated by the sliding operation, and the gray scale value of the second pixel is located outside the gray scale interval corresponding to the truncation threshold indicated by the sliding operation.

[0134] The third configuration module (not shown in the figure) is configured to: in response to a configuration instruction, take the truncation threshold corresponding to the current preview image as the target truncation threshold.

[0135] Further, the image processing apparatus 200 further comprises:

[0136] a denoising module (not shown in the figure) configured to determine a gray threshold value based on a mean value of a histogram array of the target gray image, and perform denoising processing on the sparse image based on the gray threshold value.

[0137] The specific limitations of the image processing apparatus can refer to the limitations of the image processing method described above, which will not be repeated here. Each module in the above image processing apparatus can be realized by software, hardware and their combination in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0138] Based on the above method as shown in Figure 1 , correspondingly, the embodiment of the present application also provides a readable storage medium, which stores a computer program, and the program is executed by a processor to realize the image processing method as shown in Figure 1 .

[0139] Based on such understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in each implementation scenario of the present application.

[0140] Based on the above method as shown in Figure 1 , and Figure 4 the virtual device embodiment, in order to achieve the above purpose, as shown in Figure 5 , the embodiment of the present application also provides a computer device, which comprises a processor 401 and a memory 402, and the memory 402 stores a program or instruction which can be run on the processor 401, and the program or instruction is executed by the processor 401 to realize the image processing method as shown in Figure 1 .

[0141] The memory 402 can be used to store software programs and various data. The memory 402 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 402 can include a volatile memory or a non-volatile memory, or the memory 402 can include both volatile and non-volatile memories. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 402 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0142] The processor 401 can include one or more processing units; optionally, the processor 401 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 401.

[0143] The computer device can specifically be a personal computer, a server, a network device, and the like.

[0144] Optionally, the computer device can further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, and the like. The user interface can include a display screen, an input unit such as a keyboard, and the like. Optionally, the user interface can further include a USB interface, a card reader interface, and the like. The network interface can optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), and the like.

[0145] Those skilled in the art can understand that the computer device structure provided by the embodiment does not constitute a limitation on the computer device, and can include more or fewer components, or combine certain components, or different component arrangements.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and a necessary general hardware platform, or by hardware to implement the truncation threshold-based truncation processing on the gray level histogram of the target gray image that meets the preset number of gray levels, to generate a sparse gray level histogram of the target gray image, wherein the target gray image includes a saliency region of a to-be-detected object; the gray levels of the target gray image are randomly sampled based on the sparse gray level histogram to generate a sparse image corresponding to the sampled gray levels; and the sparse image is filtered to generate a saliency image of the to-be-detected object. In the embodiment, the target gray image is first truncated in the non-target gray interval to retain the high gradient gray levels related to the object edge and texture, so as to highlight the saliency region of the to-be-detected object. Then, the same gray level pixels are extracted from the target gray image through sparse sampling, so that the gray level difference between the pixels corresponding to the sampled gray levels and the remaining pixels is relatively more significant in the sparse image, the non-uniform distribution characteristics of the edge of the to-be-detected object are maintained, and the sparse image can reflect the approximate outline of the to-be-detected object. Finally, the pixels of the sparse non-zero points are smoothed by using the Gaussian filter, the local continuity is strengthened, the subtle changes are more coherent, and the complete outline of the to-be-detected object is formed. Thus, the dense region can be suppressed, the interference of the dense region is avoided, the low-contrast changes in the local region are more easily detected, the accuracy of the image saliency detection is improved, and reliable data support is provided for subsequent analysis.

[0147] Those skilled in the art can understand that the modules or flows in the drawings are not necessarily required for implementing the present application. Those skilled in the art can understand that the modules in the devices in the implementation scenarios can be distributed in the devices in the implementation scenarios according to the description of the implementation scenarios, or can be changed to be located in one or more devices different from the implementation scenarios. The modules in the above implementation scenarios can be combined into one module, or can be further split into multiple sub-modules.

[0148] The above application numbers are only for description, and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only some specific implementation scenarios of the present application, but the present application is not limited thereto, and any variations that can be thought of by those skilled in the art should fall within the protection scope of the present application.

Claims

1. An image processing method, characterized by, The method comprises: performing truncation processing on a gray level with a frequency greater than a target truncation threshold in a gray histogram of a target gray image to generate a sparse gray histogram of the target gray image, wherein the target gray image comprises a saliency region of a to-be-detected object, and the target gray image satisfies a preset number of gray levels; performing random sampling processing on the gray levels of the target gray image based on the sparse gray histogram to generate a sparse image corresponding to a sampling gray level; performing filtering processing on the sparse image to generate a saliency image of the to-be-detected object.

2. The image processing method of claim 1, wherein, The method further comprises: if the to-be-processed gray image does not satisfy the preset number of gray levels, determining a gray level control unit according to a gray histogram of the gray image; performing down-round processing on a quotient of a gray value of the gray image and the gray level control unit to obtain a mapping gray value; multiplying the mapping gray value and the gray level control unit to generate the target gray image.

3. The image processing method of claim 2, wherein, The method of determining the gray level control unit according to the gray histogram of the gray image comprises: determining a number of gray values with a non-zero frequency in the gray image according to the gray histogram; determining the gray level control unit based on a quotient of the number and the preset number of gray levels.

4. The image processing method of claim 1, wherein, The method of performing random sampling processing on the gray levels of the target gray image based on the sparse gray histogram to generate a sparse image corresponding to a sampling gray level comprises: selecting at least part of the gray levels from the sparse gray histogram as sampling gray levels based on a random algorithm; determining a sampling proportion of each sampling gray level based on a number of pixels of each sampling gray level in the sparse gray histogram; randomly selecting pixels corresponding to each sampling gray level in the target gray image as sampling pixels corresponding to the sampling gray level based on the sampling proportion; synthesizing the sparse image based on the sampling pixels.

5. The image processing method of any one of claims 1 to 4, characterized in that, The method further comprises: performing truncation processing on a gray histogram of a sample gray image based on different candidate truncation thresholds to generate sample histograms; calculating an entropy change value of the sample histograms compared with a gray histogram of the sample gray image, and a variance value of the gray levels in the sample histograms; inputting the entropy change value and the variance value corresponding to different candidate truncation thresholds into a preset evaluation function respectively, and solving the preset evaluation function with the goal of maximizing the output to determine the target truncation threshold from the candidate truncation thresholds.

6. The image processing method of claim 5, wherein, The method further comprises: determining an edge pixel proportion of the to-be-detected object in the target gray image based on an edge detection algorithm; if the edge pixel proportion is less than a proportion threshold, increasing the target truncation threshold; if the edge pixel proportion is greater than the proportion threshold, decreasing the target truncation threshold.

7. The image processing method of any one of claims 1 to 4, characterized in that, The method further comprises: displaying a gray configuration module and a preview area of the target gray image. In response to the sliding operation on the gray scale configuration module, a preview image is generated by processing the target gray scale image based on the cut-off threshold indicated by the sliding operation, and the preview image is displayed in the preview area, wherein the first pixel and the second pixel in the preview image are displayed differently, the gray scale value of the first pixel is within the gray scale interval corresponding to the cut-off threshold indicated by the sliding operation, and the gray scale value of the second pixel is outside the gray scale interval corresponding to the cut-off threshold indicated by the sliding operation. In response to a configuration instruction, the cut-off threshold corresponding to the current preview image is taken as the target cut-off threshold.

8. The image processing method of any of claims 1-4, wherein, The method further comprises: determining a gray scale threshold based on the mean value of the histogram array of the target gray scale image; performing denoising processing on the sparse image based on the gray scale threshold.

9. An image processing apparatus characterized by comprising: The device comprises: a cut-off module configured to perform cut-off processing on the gray scale levels in the gray scale histogram of a target gray scale image, which have a frequency greater than a target cut-off threshold, to generate a sparse gray scale histogram of the target gray scale image, wherein the target gray scale image comprises a saliency region of a to-be-detected object, and the target gray scale image satisfies a preset number of gray scale levels; a sampling module configured to perform random sampling processing on the gray scale levels of the target gray scale image based on the sparse gray scale histogram to generate a sparse image corresponding to the sampled gray scale levels; a filtering module configured to perform filtering processing on the sparse image to generate a saliency image of the to-be-detected object.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the image processing method according to any one of claims 1 to 8 when executing the program.

Images