An image processing method, device and computer equipment

Abstract

The application discloses an image processing method, device and computer equipment, wherein the method is to obtain multiple scale images by decomposing a to-be-processed image to extract first feature pixel points; based on the neighborhood pixel points of the first feature pixel points on each scale image, the gradient amplitude and angle of the neighborhood pixel points are calculated; the gradient amplitudes of the same angle are counted to form an angle amplitude curve graph, and the maximum value in the angle amplitude curve graph is selected as the main direction of the first feature pixel points; based on the direction distance coefficient of the main direction and the perpendicular direction thereof, the smoothing value of the first feature pixel points and the sharpening value of the first feature pixel points are obtained; the first feature pixel points are processed and the result is output. By fully utilizing the image information of different scale images, calculating the direction distance coefficient of the main direction and the perpendicular direction thereof, and the smoothing value of the main direction and the sharpening value of the perpendicular direction, the original to-be-processed image can be accurately processed, so that the loss of the to-be-processed image is avoided.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to an image processing method, apparatus, and computer equipment. Background Technology

[0002] Image processing is the act of using computers to process image information to meet human visual and psychological needs or application requirements. As image processing applications become increasingly widespread, there is a growing emphasis on layered image details. However, due to inherent characteristics of the image signal or other external factors, the signal-to-noise ratio of the image is affected, causing some detailed information to be weakened or reduced, failing to meet the requirements for obtaining high-quality images. Therefore, to improve image quality, the main ideas are noise suppression and edge sharpening. A common approach is to roughly estimate the principal direction of a local region's feature image at four directions (0°, 45°, 90°, and 135°) and then further smooth and sharpen the image. However, the principal direction of a local region's feature image is not actually limited to these four directions; it includes any other directions. Smoothing and sharpening the image solely based on these four specific directions can easily lead to a loss of image information. Summary of the Invention

[0003] Therefore, the technical problem to be solved by the present invention is to overcome the problem that smoothing and sharpening images based on only four specific directions in the prior art is prone to causing loss of image information, thereby providing an image processing method, apparatus and computer equipment.

[0004] According to a first aspect, embodiments of the present invention provide an image processing method, comprising the following steps:

[0005] The image to be processed is acquired, and the image to be processed is decomposed into images of multiple scales through preprocessing to extract the first feature pixels.

[0006] Based on the neighboring pixels of the first feature pixel in each scale image, calculate the gradient magnitude and angle of the neighboring pixels in each scale image;

[0007] The gradient magnitude corresponding to the same angle of neighboring pixels on each scale image is statistically analyzed to form an angle magnitude curve, and the maximum value is selected from the angle magnitude curve as the main direction of the first feature pixel.

[0008] Based on the preset angle range to which the main direction and its perpendicular direction belong, calculate the directional distance coefficient of the angle values ​​at the two endpoints of the preset angle range to obtain the smoothing value and the sharpening value of the first feature pixel.

[0009] Based on the smoothing value and sharpening value of the first feature pixel, the first feature pixel is processed and the result is output.

[0010] In conjunction with the first aspect, in one embodiment of the first aspect, an image to be processed is acquired, and the image to be processed is decomposed into images of multiple scales through preprocessing to extract first feature pixels, including:

[0011] Based on the Gaussian kernel function, the image to be processed is decomposed to obtain the images at multiple scales;

[0012] Based on a preset grayscale algorithm and voting mechanism, the first feature pixel is extracted from the images at multiple scales.

[0013] In conjunction with the first aspect, in another embodiment of the first aspect, the gradient magnitude and angle of the first feature pixel are calculated based on the neighboring pixels of the first feature pixel in the image at each scale, including:

[0014] Based on the size specifications of each scale image and the location of the first feature pixel in each scale image, neighborhood pixels of the first feature pixel are selected on each scale image with the first feature pixel as the center.

[0015] Based on the coordinates of the neighboring pixels of the first feature pixel in each scale image, the scale factor and weight coefficient of each scale image, the gradient magnitude of the neighboring pixels in each scale image is calculated.

[0016] Based on the coordinates of the neighboring pixels of the first feature pixel in each scale image, the gradient magnitude and angle of the neighboring pixels in each scale image are calculated.

[0017] In conjunction with the first aspect, in another embodiment of the first aspect, the step of statistically analyzing the gradient magnitudes corresponding to the same angle to form an angle magnitude curve, and selecting the maximum value from the angle magnitude curve as the main direction of the first feature pixel, includes:

[0018] By traversing each first feature pixel of each scale image, the gradient magnitude corresponding to the same angle of the neighboring pixels in each scale image is counted to obtain the angle magnitude curve.

[0019] The angle amplitude curve is smoothed to remove singular or isolated points;

[0020] Select the maximum value from the smoothed angle amplitude curve.

[0021] The maximum value is taken as the main direction of the first feature pixel.

[0022] In conjunction with the first aspect, in another embodiment of the first aspect, based on the preset angle range to which the main direction and its perpendicular direction belong, a directional distance coefficient is calculated for the angle values ​​at the two endpoints of the preset angle range to obtain the smoothing value and the sharpening value of the first feature pixel, including:

[0023] Determine the preset angle range for multiple different intervals;

[0024] Based on the preset angle ranges of the multiple different intervals, the preset angle range to which the main direction and its perpendicular direction belong is determined;

[0025] Based on the two endpoints of the preset angle range to which the main direction belongs, calculate the directional distance coefficient corresponding to the main direction; based on the two endpoints of the preset angle range to which the vertical direction belongs, calculate the directional distance coefficient corresponding to the vertical direction.

[0026] Based on the two endpoints of the preset angle range to which the main direction belongs, calculate the average gray value of the neighboring pixels of the first feature pixel in the image to be processed; based on the two endpoints of the preset angle range to which the vertical direction belongs, calculate the sharpness component of the neighboring pixels of the first feature pixel in the image to be processed.

[0027] Based on the average grayscale value and the directional distance coefficient of the main direction, the smoothness value of the first feature pixel is calculated using a bilinear interpolation algorithm; based on the sharpness value component and the directional distance coefficient of the vertical direction, the sharpness value of the first feature pixel is calculated using the bilinear interpolation algorithm.

[0028] In conjunction with the first aspect, in another embodiment of the first aspect, processing the first feature pixel and outputting a result based on the smoothing value and sharpening value of the first feature pixel includes:

[0029] Obtain the first preset coefficient corresponding to the smoothing value of the first feature pixel and the second preset coefficient corresponding to the sharpening value of the first feature pixel;

[0030] Based on the smoothing value of the first feature pixel and the first preset coefficient, the sharpening value of the first feature pixel and the second preset coefficient, the output result of the first feature pixel is obtained by summation.

[0031] In conjunction with the first aspect, in another embodiment of the first aspect, the image processing method further includes: acquiring an image to be processed, and decomposing the image to be processed by a preprocessing method to obtain images of multiple scales to extract second feature pixels.

[0032] In conjunction with the first aspect, in another embodiment of the first aspect, the gray-scale mean of the second feature pixel is calculated by normalization based on the neighboring pixels of the second feature pixel in each scale of the image.

[0033] In conjunction with the first aspect, in another embodiment of the first aspect, the image processing method wherein the image to be processed includes: an endoscopic ultrasound image or a camera photographic image.

[0034] According to a first aspect, embodiments of the present invention also provide an image processing apparatus, comprising the following modules:

[0035] The image acquisition module is used to acquire the image to be processed and decompose the image to be processed into images of multiple scales through preprocessing to extract the first feature pixels.

[0036] The gradient magnitude and angle calculation module is used to calculate the gradient magnitude and angle of the neighboring pixels in each scale image based on the neighboring pixels of the first feature pixel in each scale image.

[0037] The main direction selection module is used to statistically analyze the gradient magnitude corresponding to the same angle of neighboring pixels on each scale image to form an angle magnitude curve, and select the maximum value from the angle magnitude curve as the main direction of the first feature pixel.

[0038] The smoothing and sharpening value calculation module is used to calculate the directional distance coefficient of the two endpoint angle values ​​of the preset angle range based on the preset angle range to which the main direction and its perpendicular direction belong, so as to obtain the smoothing value and the sharpening value of the first feature pixel.

[0039] The image processing module is used to process the first feature pixel and output the result based on the smoothing value and sharpening value of the first feature pixel.

[0040] According to a third aspect, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions for causing the computer to perform the image processing method described in the first aspect or any embodiment of the first aspect.

[0041] According to a fourth aspect, embodiments of the present invention also provide a computer device, including: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the image processing method described in the first aspect or any embodiment of the first aspect.

[0042] The technical solution of this invention has the following advantages:

[0043] This invention discloses an image processing method, apparatus, and computer device. The method involves: decomposing the image to be processed into images of multiple scales through preprocessing to extract a first feature pixel; calculating the gradient magnitude and angle of the neighboring pixels of the first feature pixel in each scale image based on the neighboring pixels of the first feature pixel; statistically analyzing the gradient magnitudes corresponding to the same angle of the neighboring pixels in each scale image to form an angle magnitude curve, and selecting the maximum value from the angle magnitude curve as the principal direction of the first feature pixel; calculating the directional distance coefficient between the two endpoints of the preset angle range based on the principal direction and its perpendicular direction to obtain the smoothing value and sharpening value of the first feature pixel; processing the first feature pixel based on the smoothing value and the sharpening value of the first feature pixel and outputting the result. Therefore, this invention, by fully utilizing image information from images of different scales and calculating the directional distance coefficient between the principal direction and its perpendicular direction, as well as the smoothing value of the principal direction and the sharpening value of the perpendicular direction, facilitates accurate processing of the original image to be processed, preventing the loss of the image. Attached Figure Description

[0044] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating a specific example of an image processing method in an embodiment of the present invention;

[0046] Figure 2 This is a principle block diagram of another specific example of the image processing method in this embodiment of the invention;

[0047] Figure 3 This is a schematic diagram of the neighborhood pixels of the first feature pixel in the scale image in an embodiment of the present invention;

[0048] Figure 4 This is a principle block diagram of another specific example of the image processing method in this embodiment of the invention;

[0049] Figure 5 This is a graph showing the angle amplitude in an embodiment of the present invention;

[0050] Figure 6 This is a schematic diagram of the main direction in an embodiment of the present invention;

[0051] Figure 7This is a principle block diagram of another specific example of the image processing method in this embodiment of the invention;

[0052] Figure 8 This is a schematic diagram of the first feature pixel based on the neighboring pixels on the original image to be processed in an embodiment of the present invention;

[0053] Figure 9 This is a structural block diagram of the image processing device in an embodiment of the present invention;

[0054] Figure 10 This is a schematic diagram of the hardware structure of a computer device in an embodiment of the present invention. Detailed Implementation

[0055] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0057] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can also refer to the internal connection of two components; and they can refer to a wireless connection or a wired connection. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0058] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0059] This invention provides an image processing method, such as... Figure 1 As shown, it includes the following steps:

[0060] Step S11: Obtain the image to be processed, and decompose the image to be processed through preprocessing to obtain images of multiple scales in order to extract the first feature pixels.

[0061] Specifically, the image to be processed is the original image input into the computer device, which can be an endoscopic ultrasound image or a camera photograph. For example, endoscopic ultrasound is widely used in the field of medical diagnosis. It uses an ultrasound probe to obtain clear images of the layered structure of the digestive or respiratory tract walls and surrounding organs. It has important medical significance in diagnosing submucosal masses and determining the degree of invasion of related malignant tumors, and also has certain guiding significance in the selection of surgical timing and method, as well as prognosis and treatment. Through the method in the embodiments of the present invention, layered detail processing of endoscopic ultrasound images can be achieved to improve the quality of endoscopic ultrasound images.

[0062] In one specific implementation, step S11 above, which involves acquiring the image to be processed and decomposing it into images of multiple scales through preprocessing to extract the first feature pixels, includes:

[0063] Step 1: Based on the Gaussian kernel function, decompose the image to be processed to obtain images of various scales.

[0064] For example, the image to be processed is decomposed into images of multiple scales, where S represents the number of scales. We can select Gaussian kernels of different sizes. For example, if there are 3 scales, the Gaussian kernel sizes would be 3, 5, and 7 respectively. The sigma of the Gaussian kernel function at different scales... i =init*K i ,i≤S, `init` uses the system default value, utilizing the Gaussian kernel size and Sigma. i The value can be obtained from the Gaussian kernel G. i The image to be processed is subjected to convolution filtering with different Gaussian kernel functions, I i =I0*G i Where I0 is the original input image to be processed, * represents the convolution operation, and G i It is a Gaussian kernel, I i The i-th scale image corresponding to the processed Gaussian kernel is represented by these images, which can represent image details at different scales.

[0065] Step 2: Extract the first feature pixels from images of multiple scales based on a preset grayscale algorithm and voting mechanism.

[0066] In one example, a neighborhood is selected at each image scale centered on the first feature pixel. This neighborhood is the same size as the Gaussian kernel of the current scale image. The mean (Mean) and variance (Var) of all pixels within this neighborhood are calculated. Based on the mean and variance, it is determined whether this pixel in the current scale image is the first feature pixel. For example: Suppose the grayscale value of the pixel at this location in the current scale image is gray. When the following conditions are met... If the value is preset for the system, then this pixel in the image at that scale is considered the first feature pixel. For the same pixel, it is necessary to traverse all images at different scales and use a voting mechanism to finally determine whether the pixel is the first feature pixel.

[0067] The voting mechanism is that if more than half or more of the pixels are identified as first feature pixels, then the pixel is considered a first feature pixel. For example, in an image with three scales, if the same pixel location is identified as a first feature pixel in two or more scales, then the pixel is considered a first feature pixel. Otherwise, it is identified as a second feature pixel (a non-feature pixel).

[0068] By extracting the first and second feature pixels using the above method, misjudgment of the first feature pixel caused by noise can be effectively avoided, and the first feature pixel can be accurately segmented, especially weak edges or corners. Traditional methods, which generally rely on grayscale and gradient information to determine feature points, are sensitive to noise and cannot effectively detect weak edges. In commonly used image processing methods, noise does not actually need to be sharpened or enhanced.

[0069] Step S12: Based on the neighboring pixels of the first feature pixel in each scale image, calculate the gradient magnitude and angle of the neighboring pixels in each scale image.

[0070] In one specific implementation, such as Figure 2 As shown, step S12 above, based on the neighboring pixels of the first feature pixel in the image at each scale, calculates the gradient magnitude and angle of the first feature pixel, including:

[0071] Step S21: Based on the size specifications of the image at each scale and the location of the first feature pixel in the image at each scale, select the neighboring pixels of the first feature pixel in the image at each scale, with the first feature pixel as the center.

[0072] For example, such as Figure 3As shown, the first feature pixel is set as the center pixel, and neighboring pixels with the same size as the image at this scale are selected. For example, with a Gaussian kernel size of 3, each neighboring pixel is traversed. Taking the neighboring pixel as the center, the four pixels above, below, left, and right of it are used to participate in the calculation of amplitude and angle. Let the coordinates of the current pixel be (x,y), and the gray value of the pixel at this position is represented as I(x,y).

[0073] Step S22: Based on the coordinates of the neighboring pixels of the first feature pixel in each scale image, the scale factor and weight coefficient of each scale image, calculate the gradient magnitude of the neighboring pixels of each scale image.

[0074] Step S23: Calculate the angle of the neighboring pixels of the first feature pixel in the image at each scale based on the coordinates of the neighboring pixels of the first feature pixel in the image at each scale.

[0075] In a specific example, Figure 3 In this image, let the current pixel coordinates be (x, y), and the grayscale pixel value at this location be denoted as I(x, y). On an image with a Gaussian kernel size of 3, the gradient magnitude of each neighboring pixel is represented by Mag. Then, based on the different scale factors sigma of the image, calculate the corresponding weight coefficients in the image at that scale. Obtain the weighted gradient magnitude Mag w =Mag*weight, the angle of the first feature pixel Here, arctan represents the arctangent operation, which converts the angle (in radians) into degrees. The order of steps S22 and S23 above is not important. Similarly, for images of different Gaussian kernels at different scales, the gradient magnitude of each neighboring pixel is calculated in the same way.

[0076] Step S13: Calculate the gradient magnitude corresponding to the same angle of neighboring pixels on each scale of the image to form an angle magnitude curve, and select the maximum value from the angle magnitude curve as the main direction of the first feature pixel.

[0077] For example, by summing the gradient magnitudes of neighboring pixels at the same angle in images of each scale, and statistically analyzing the gradient magnitudes at the same angle, an angle magnitude curve can be obtained.

[0078] In one specific implementation, such as Figure 4 As shown, step S13 above, which involves calculating the gradient magnitude of neighboring pixels at the same angle on each scale of the image to form an angle magnitude curve, and selecting the maximum value from the angle magnitude curve as the main direction of the first feature pixel, includes:

[0079] Step S41: Traverse each first feature pixel of each scale image, and calculate the gradient magnitude corresponding to the same angle of the neighboring pixels in each scale image to obtain the angle magnitude curve.

[0080] Step S42: Smooth the angle amplitude curve to remove singular or isolated points.

[0081] Step S43: Select the maximum value from the smoothed angle amplitude curve.

[0082] Step S44: Use the maximum value as the main direction of the first feature pixel.

[0083] Specifically, by executing steps S41-S44 above, the angle amplitude curve is smoothed, for example, by mean filtering or median filtering, to remove singular or isolated points. Then, the maximum value of the angle amplitude curve is calculated, and the corresponding angle is used as the main direction of the first feature pixel. Figure 5 The image shown is the final angle amplitude curve.

[0084] Step S14: Based on the preset angle range to which the main direction and its perpendicular direction belong, calculate the directional distance coefficient of the angle values ​​at the two endpoints of the preset angle range to obtain the smoothing value and sharpening value of the first feature pixel.

[0085] Based on the angular amplitude curve of each first feature pixel and the details of images at different scales, it is beneficial to accurately calculate the main direction of the first feature pixel. At the same time, based on the directional distance coefficient of the main direction and its perpendicular direction, the image detail components are effectively utilized to achieve the purpose of smoothing and enhancement.

[0086] In one specific implementation, such as Figure 7 As shown, step S14 above, based on the preset angle range to which the main direction and its perpendicular direction belong, calculates the directional distance coefficient of the angle values ​​at the two endpoints of the preset angle range to obtain the smoothing value and sharpening value of the first feature pixel, including:

[0087] Step S71: Determine the preset angle range for multiple different intervals.

[0088] For example, considering that the image signal is composed of discrete pixels, the angle needs to be quantized and described using four common directions, calculating the distance between the main direction and a fixed angle. The four common directions are 0°, 45°, 90°, and 135°. Four preset angle ranges can be set, such as W1 as [0°, 45°), W2 as [45°, 90°), W3 as [90°, 135°), and W4 as [135°, 180°]. Figure 6 The diagram shown is a schematic representation of the preset angle range belonging to the main direction.

[0089] Step S72: Based on the preset angle ranges of multiple different intervals, confirm the preset angle range to which the main direction and its perpendicular direction belong.

[0090] Step S73: Calculate the directional distance coefficient corresponding to the main direction based on the two endpoint values ​​of the preset angle range to which the main direction belongs; calculate the directional distance coefficient corresponding to the vertical direction based on the two endpoint values ​​of the preset angle range to which the vertical direction belongs.

[0091] In a specific example, first determine which interval L the main direction of the current first feature pixel falls into, i.e., satisfying L start ≤θ1≤L end L start The minimum value of the corresponding interval, L end The maximum value in the corresponding interval is used to calculate the directional distance coefficient in the main direction. mod(θ1,45) represents the remainder when θ1 is divided by 45, and the directional distance coefficient L2 in the main direction is 1-L1.

[0092] In another specific example, we find the perpendicular direction of the principal direction, i.e., θ2 = 0 + 90°. Then, we calculate the directional distance coefficient of this perpendicular direction. Based on the example above, we first determine which interval L the perpendicular direction of the current first feature pixel falls into, i.e., whether L satisfies... start ≤θ2≤L end L start The minimum value of the corresponding interval, L end The maximum value in the corresponding interval is then used to calculate the directional distance coefficient. mod(θ2,45) represents the remainder when θ2 is divided by 45, and the directional distance coefficient in the vertical direction is L4 = 1 - L3.

[0093] Step S74: Based on the two endpoint values ​​of the preset angle range to which the main direction belongs, calculate the average gray value of the neighboring pixels of the first feature pixel in the image to be processed; based on the two endpoint values ​​of the preset angle range to which the vertical direction belongs, calculate the sharpness value component of the neighboring pixels of the first feature pixel in the image to be processed.

[0094] For example, smoothing is performed along the main direction of the image to be processed. Based on the original input image to be processed and a preset angle range L, the average gray value M of the neighboring pixels along the two endpoints of the preset angle range to which the main direction belongs is calculated. d1 M d2 .like Figure 6 As shown, for example: the two endpoints of the preset angle range belonging to the main direction are 45° and 90°, respectively. Figure 8As shown, M represents the pixel grayscale value at various locations in the image to be processed. Taking the first feature pixel B2 as the center, the average grayscale value of the neighboring pixels in the 45° direction is M. d1 = (A3+B2+C1) / 3; The average grayscale value of neighboring pixels in the 90° direction is M. d2 = (A2+B2+C2) / 3.

[0095] For example, sharpening is performed along the main direction of the image to be processed. Based on the original input image to be processed and the preset angle range L, a Laplacian template {-1,2,-1} can be set to calculate the sharpness value component L of the neighboring pixels along the two endpoints of the preset angle range in the vertical direction. ap1 L ap2 For example: Figure 6 As shown, the two endpoints of the preset angle range belonging to the perpendicular direction of the main direction are 180° (i.e., 0°) and 135°. Figure 8 In the diagram, with the first feature pixel B2 as the center, the sharpness component of the neighboring pixels in the 0° direction is L. ap1 = (-1)*B1+2*B2+(-1)*B3, where the sharpness component of the neighboring pixels in the 135° direction is L. ap2 = (-1)*A1+2*B2+(-1)*C3.

[0096] Step S55: Based on the average grayscale value and the directional distance coefficient of the main direction, calculate the smooth value of the first feature pixel using a bilinear interpolation algorithm; based on the sharpness component and the directional distance coefficient of the vertical direction, calculate the sharpness value of the first feature pixel using a bilinear interpolation algorithm.

[0097] In a specific example, the directional distance coefficients corresponding to the main direction are L1 and L2, and the average grayscale value is M. d1 M d2 The smooth value SmoothV = M for the first feature pixel is obtained using the bilinear interpolation algorithm. d1 *L2+M d2 *L1, this smoothing value is used as the smoothing value of the first feature pixel; the directional distance coefficients corresponding to the vertical direction are L3 and L4, and the sharpening value of the first feature pixel is obtained using the bilinear interpolation algorithm, SharpV = L ap1 *L3+L ap2 *L4.

[0098] The embodiments of the present invention calculate the directional distance coefficients of the main direction and its perpendicular direction through the above implementation method, and further combine them with a preset angle range. The average gray value and sharpness value components of the neighboring pixels of the first feature pixel on the image to be processed are finally used to obtain the smoothness value and sharpness value of the first feature pixel, so as to accurately process the original input image to be processed and avoid image loss.

[0099] Step S15: Based on the smoothing value and sharpening value of the first feature pixel, process the first feature pixel and output the result.

[0100] In one specific implementation, step S15 above, which processes the first feature pixel and outputs the result based on the smoothing value and sharpening value of the first feature pixel, includes:

[0101] Step 1: Obtain the first preset coefficient corresponding to the smoothing value of the first feature pixel and the second preset coefficient corresponding to the sharpening value of the first feature pixel.

[0102] For example: the first preset coefficient is The second preset coefficient is And satisfy and It is the system's default value.

[0103] Step 2: Based on the smoothing value and the first preset coefficient, the sharpening value and the second preset coefficient of the first feature pixel, the output result of the first feature pixel is obtained by summation.

[0104] The output result of the first feature pixel.

[0105] In another specific embodiment of the present invention, the image processing method further includes: acquiring an image to be processed, and decomposing the image to be processed by a preprocessing method to obtain images of multiple scales in order to extract second feature pixels.

[0106] The second feature pixel is a non-feature pixel, and its extraction method is the same as that for the first feature pixel, so it will not be repeated here. Preprocessing decomposes the image into multiple scales, which can fully display image details at different scales and avoid errors in feature point extraction.

[0107] In one specific implementation, the gray-scale mean of the second feature pixel is calculated by normalization based on the neighboring pixels of the second feature pixel in the image at each scale.

[0108] For example, the weighting coefficients for images at different scales can be applied equally: The original image to be processed is decomposed into n scale images, where i represents the nth scale image. The weighting coefficients for the different scale images can also be normalized. The original image to be processed is decomposed into n scale images, where i represents the nth scale image and Mi represents the average gray level of a fixed neighborhood at each scale. Then, a weighted summation method is used to obtain the average gray level M at each scale image. Assuming the original grayscale value of the image to be processed is gray and the final grayscale value is P, then P = β1*gray + β2*M, where β1 and β2 are preset weighting coefficients and satisfy β1 + β2 = 1, which corresponds to the smoothness. This value is preset according to different systems.

[0109] Based on the same concept, embodiments of the present invention also provide an image processing apparatus, such as... Figure 9 As shown, it includes the following modules:

[0110] The first feature pixel extraction module 91 is used to acquire the image to be processed and decompose the image to be processed into images of multiple scales through preprocessing to extract the first feature pixel points.

[0111] The gradient magnitude and angle calculation module 92 is used to calculate the gradient magnitude and angle of the neighboring pixels in the image at each scale based on the neighboring pixels of the first feature pixel in the image at each scale.

[0112] The main direction selection module 93 is used to statistically analyze the gradient magnitude corresponding to the same angle of neighboring pixels on each scale image to form an angle magnitude curve, and select the maximum value from the angle magnitude curve as the main direction of the first feature pixel.

[0113] The smoothing and sharpening value calculation module 94 is used to calculate the directional distance coefficient of the two endpoint angle values ​​of the preset angle range based on the preset angle range to which the main direction and its perpendicular direction belong, so as to obtain the smoothing value and the sharpening value of the first feature pixel.

[0114] Image processing module 95 is used to process the first feature pixel and output the result based on the smoothing value and sharpening value of the first feature pixel.

[0115] In one specific implementation, the first feature pixel extraction module 91 includes:

[0116] The image decomposition submodule is used to decompose the image to be processed into images of various scales based on the Gaussian kernel function.

[0117] The first feature extraction submodule is used to extract first feature pixels from images of multiple scales based on a preset grayscale algorithm and voting mechanism.

[0118] In one specific implementation, the gradient magnitude and angle calculation module 92 includes:

[0119] The neighboring pixel selection submodule is used to select neighboring pixels of the first feature pixel in each scale image based on the size specification of the image at each scale and the location of the first feature pixel in each scale image, with the first feature pixel as the center.

[0120] The gradient magnitude calculation submodule is used to calculate the gradient magnitude of the neighboring pixels of the first feature pixel in the image at each scale based on the coordinates of the neighboring pixels of the first feature pixel in the image at each scale, the scale factor of the image at each scale, and the weight coefficient.

[0121] The angle calculation submodule is used to calculate the angle of the first feature pixel by considering the coordinates of its neighboring pixels on each scale of the image.

[0122] In one specific implementation, the main direction selection module 93 includes:

[0123] The amplitude statistics submodule is used to traverse each first feature pixel of each scale image and calculate the gradient amplitude corresponding to the same angle of the neighboring pixels in each scale image to obtain the angle amplitude curve.

[0124] The smoothing submodule is used to smooth the angle amplitude curve and remove singular or isolated points.

[0125] The maximum value selection submodule is used to select the maximum value from the smoothed angle amplitude curve graph.

[0126] The main direction determination submodule is used to take the maximum value as the main direction of the first feature pixel.

[0127] In one specific implementation, the smoothing sharpness value calculation module 94 includes:

[0128] The preset angle determination submodule is used to determine the preset angle range of multiple different intervals;

[0129] The main direction confirmation submodule is used to confirm the preset angle range to which the main direction and its perpendicular direction belong, based on preset angle ranges of multiple different intervals.

[0130] The direction distance coefficient calculation submodule is used to calculate the direction distance coefficient corresponding to the main direction based on the two endpoint values ​​of the preset angle range to which the main direction belongs; and to calculate the direction distance coefficient corresponding to the vertical direction based on the two endpoint values ​​of the preset angle range to which the vertical direction belongs.

[0131] The grayscale and sharpness mean calculation submodule is used to calculate the average grayscale value of the neighboring pixels of the first feature pixel in the image to be processed based on the two endpoint values ​​of the preset angle range to which the main direction belongs; and to calculate the sharpness value component of the neighboring pixels of the first feature pixel in the image to be processed based on the two endpoint values ​​of the preset angle range to which the vertical direction belongs.

[0132] The smoothing sharpness value calculation submodule is used to calculate the smoothing value of the first feature pixel based on the grayscale average value and the directional distance coefficient of the main direction using a bilinear interpolation algorithm; and to calculate the sharpness value of the first feature pixel based on the sharpness value component and the directional distance coefficient of the vertical direction using a bilinear interpolation algorithm.

[0133] In one specific implementation, the image processing module 95 includes:

[0134] The preset coefficient acquisition submodule is used to acquire the first preset coefficient corresponding to the smoothing value of the first feature pixel and the second preset coefficient corresponding to the sharpening value of the first feature pixel.

[0135] The image result output submodule is used to obtain the output result of the first feature pixel by summing the smoothing value of the first feature pixel and the first preset coefficient, the sharpening value of the first feature pixel and the second preset coefficient.

[0136] In one specific embodiment, the image processing apparatus of this invention further includes: a second feature pixel extraction module, used to acquire the image to be processed and to decompose the image to be processed into images of multiple scales through preprocessing to extract the second feature pixel points.

[0137] In one specific embodiment, the image processing apparatus of this invention further includes: a second feature pixel grayscale calculation module, used to calculate the average grayscale value of the second feature pixel in a normalized manner based on the neighboring pixels of the second feature pixel in each scale image.

[0138] In one specific embodiment of the present invention, the image processing device includes an endoscopic ultrasound image or a camera photographic image as the image to be processed.

[0139] The functions and effects of each module and sub-module in the image processing device of this invention embodiment have been described in the above image processing method and will not be repeated here. The image processing device of this invention embodiment, by making full use of image information of images at different scales and calculating the directional distance coefficients of the main direction and its perpendicular direction, as well as the smoothing value of the main direction and the sharpening value of the perpendicular direction, is beneficial to accurately process the original image to be processed, so as to avoid the loss of the image to be processed.

[0140] This invention also provides a computer device, such as... Figure 10 As shown, the computer device may include a processor 101 and a memory 102, wherein the processor 101 and the memory 102 may be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0141] Processor 101 may be a central processing unit (CPU). Processor 101 may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof.

[0142] The memory 102, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 101 executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory 102, thereby implementing the image processing method in the above method embodiments.

[0143] The memory 102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor 101, etc. Furthermore, the memory 102 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 102 may optionally include memory remotely located relative to the processor 101, and these remote memories may be connected to the processor 101 via a network. Examples of such networks include, but are not limited to, power grids, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0144] The one or more modules are stored in the memory 102, and when executed by the processor 101, they perform the image processing method as shown in the embodiment of the attached figure.

[0145] The specific details of the above-mentioned electronic device can be understood by referring to the relevant descriptions and effects in the embodiments shown in the accompanying drawings, and will not be repeated here.

[0146] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0147] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. An image processing method, characterized in that, Includes the following steps: The image to be processed is acquired, and the image to be processed is decomposed into images of multiple scales through preprocessing to extract the first feature pixels. The process involves acquiring an image to be processed and decomposing it into images at multiple scales through preprocessing to extract first feature pixels, including: Based on the Gaussian kernel function, the image to be processed is decomposed to obtain images of multiple scales; Based on a pre-defined grayscale algorithm and voting mechanism, first feature pixels are extracted from images at multiple scales. A neighborhood is selected centered on this first feature pixel in each scale image. The size of this neighborhood is the same as the Gaussian kernel size of the current scale image. The mean of all pixels within this neighborhood is then calculated. The sum and variance are Let the grayscale value of the first feature pixel in the current scale image be... When satisfied , If the value is preset for the system, then the pixel in the image at that scale is considered to be the first feature pixel. Then, for the same pixel, it is necessary to traverse all images at different scales and use a voting mechanism to finally determine whether the pixel is the first feature pixel. Based on the neighboring pixels of the first feature pixel in each scale image, calculate the gradient magnitude and angle of the neighboring pixels in each scale image; The gradient magnitude corresponding to the same angle of neighboring pixels on each scale image is statistically analyzed to form an angle magnitude curve, and the maximum value is selected from the angle magnitude curve as the main direction of the first feature pixel. Based on the preset angle range to which the main direction and its perpendicular direction belong, calculate the directional distance coefficient of the angle values ​​at the two endpoints of the preset angle range to obtain the smoothing value and the sharpening value of the first feature pixel. Based on the smoothing value and sharpening value of the first feature pixel, process the first feature pixel and output the result; Specifically, based on the preset angle range to which the main direction and its perpendicular direction belong, the directional distance coefficients of the two endpoints of the preset angle range are calculated to obtain the smoothing value and the sharpening value of the first feature pixel, including: Determine the preset angle range for multiple different intervals; Based on the preset angle ranges of the multiple different intervals, the preset angle range to which the main direction and its perpendicular direction belong is determined; Based on the two endpoints of the preset angle range to which the main direction belongs, calculate the directional distance coefficient corresponding to the main direction; based on the two endpoints of the preset angle range to which the vertical direction belongs, calculate the directional distance coefficient corresponding to the vertical direction. Based on the two endpoints of the preset angle range to which the main direction belongs, calculate the average gray value of the neighboring pixels of the first feature pixel in the image to be processed; based on the two endpoints of the preset angle range to which the vertical direction belongs, calculate the sharpness component of the neighboring pixels of the first feature pixel in the image to be processed. Based on the average grayscale value and the directional distance coefficient of the main direction, the smoothness value of the first feature pixel is calculated using a bilinear interpolation algorithm; based on the sharpness value component and the directional distance coefficient of the vertical direction, the sharpness value of the first feature pixel is calculated using the bilinear interpolation algorithm.

2. The image processing method according to claim 1, characterized in that, The process involves acquiring an image to be processed and decomposing it into images at multiple scales through preprocessing to extract first feature pixels, including: Based on the Gaussian kernel function, the image to be processed is decomposed to obtain the images at multiple scales; Based on a preset grayscale algorithm and voting mechanism, the first feature pixel is extracted from the images at multiple scales.

3. The image processing method according to claim 1 or 2, characterized in that, Based on the neighboring pixels of the first feature pixel in each scale image, the gradient magnitude and angle of the neighboring pixels in each scale image are calculated, including: Based on the size specifications of each scale image and the location of the first feature pixel in each scale image, neighborhood pixels of the first feature pixel are selected on each scale image with the first feature pixel as the center. Based on the coordinates of the neighboring pixels of the first feature pixel in each scale image, the scale factor and weight coefficient of each scale image, the gradient magnitude of the neighboring pixels in each scale image is calculated. Based on the coordinates of the neighboring pixels of the first feature pixel in the image at each scale, the angle of the neighboring pixels in the image at each scale is calculated.

4. The image processing method according to claim 1, characterized in that, The step of statistically analyzing the gradient magnitudes corresponding to the same angle among neighboring pixels in the image at each scale to form an angle magnitude curve, and selecting the maximum value from the angle magnitude curve as the main direction of the first feature pixel, includes: By traversing each first feature pixel of each scale image, the gradient magnitude corresponding to the same angle of the neighboring pixels in each scale image is counted to obtain the angle magnitude curve. The angle amplitude curve is smoothed to remove singular or isolated points; Select the maximum value from the smoothed angle amplitude curve. The maximum value is taken as the main direction of the first feature pixel.

5. The image processing method according to claim 1, characterized in that, Based on the smoothing value and sharpening value of the first feature pixel, the first feature pixel is processed and the result is output, including: Obtain the first preset coefficient corresponding to the smoothing value of the first feature pixel and the second preset coefficient corresponding to the sharpening value of the first feature pixel; Based on the smoothing value of the first feature pixel and the first preset coefficient, the sharpening value of the first feature pixel and the second preset coefficient, the output result of the first feature pixel is obtained by summation.

6. The image processing method according to claim 1, characterized in that, Also includes: The image to be processed is acquired, and the image to be processed is decomposed into images of multiple scales through preprocessing to extract the second feature pixels.

7. The image processing method according to claim 6, characterized in that, Based on the neighboring pixels of the second feature pixel in each scale of the image, the gray mean of the second feature pixel is calculated by normalization.

8. The image processing method according to claim 1, characterized in that, The images to be processed include: endoscopic ultrasound images or camera images.

9. An image processing apparatus, characterized in that, Includes the following modules: The image acquisition module is used to acquire the image to be processed and decompose the image to be processed into images of multiple scales through preprocessing to extract the first feature pixels. The process involves acquiring an image to be processed and decomposing it into images at multiple scales through preprocessing to extract first feature pixels, including: Based on the Gaussian kernel function, the image to be processed is decomposed to obtain images of multiple scales; Based on a pre-defined grayscale algorithm and voting mechanism, first feature pixels are extracted from images at multiple scales. A neighborhood is selected centered on this first feature pixel in each scale image. The size of this neighborhood is the same as the Gaussian kernel size of the current scale image. The mean of all pixels within this neighborhood is then calculated. The sum and variance are Let the grayscale value of the first feature pixel in the current scale image be... When satisfied , If the value is preset for the system, then the pixel in the image at that scale is considered to be the first feature pixel. Then, for the same pixel, it is necessary to traverse all images at different scales and use a voting mechanism to finally determine whether the pixel is the first feature pixel. The gradient magnitude and angle calculation module is used to calculate the gradient magnitude and angle of the neighboring pixels in each scale image based on the neighboring pixels of the first feature pixel in each scale image. The main direction selection module is used to statistically analyze the gradient magnitude corresponding to the same angle of neighboring pixels on each scale image to form an angle magnitude curve, and select the maximum value from the angle magnitude curve as the main direction of the first feature pixel. The smoothing and sharpening value calculation module is used to calculate the directional distance coefficient of the two endpoint angle values ​​of the preset angle range based on the preset angle range to which the main direction and its perpendicular direction belong, so as to obtain the smoothing value and the sharpening value of the first feature pixel. The image processing module is used to process the first feature pixel based on the smoothing value and the sharpening value of the first feature pixel and output the result. The smoothing and sharpening value calculation module includes: The preset angle determination submodule is used to determine the preset angle range of multiple different intervals; The main direction confirmation submodule is used to confirm the preset angle range to which the main direction and its perpendicular direction belong, based on preset angle ranges of multiple different intervals. The direction distance coefficient calculation submodule is used to calculate the direction distance coefficient corresponding to the main direction based on the two endpoint values ​​of the preset angle range to which the main direction belongs; and to calculate the direction distance coefficient corresponding to the vertical direction based on the two endpoint values ​​of the preset angle range to which the vertical direction belongs. The grayscale and sharpness mean calculation submodule is used to calculate the average grayscale value of the neighboring pixels of the first feature pixel in the image to be processed based on the two endpoint values ​​of the preset angle range to which the main direction belongs; and to calculate the sharpness value component of the neighboring pixels of the first feature pixel in the image to be processed based on the two endpoint values ​​of the preset angle range to which the vertical direction belongs. The smoothing sharpness value calculation submodule is used to calculate the smoothing value of the first feature pixel based on the grayscale average value and the directional distance coefficient of the main direction using a bilinear interpolation algorithm; and to calculate the sharpness value of the first feature pixel based on the sharpness value component and the directional distance coefficient of the vertical direction using a bilinear interpolation algorithm.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the image processing method according to any one of claims 1 to 8.

11. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the image processing method according to any one of claims 1 to 8.

Images