Image enhancement method and system for optical imaging acupoint assisted identification

Abstract

The invention relates to the technical field of image enhancement, and in particular to an image enhancement method and system for optical imaging acupoint auxiliary identification. The method obtains a face image under a side illumination angle, performs grayscale processing on the face image, and obtains a grayscale image; for any pixel point in the grayscale image, obtains an initial contrast limit value of the pixel point when performing contrast-limited adaptive histogram equalization on the grayscale image; obtains a contrast limit optimization factor of the pixel point, optimizes the initial contrast limit value of the pixel point by using the contrast limit optimization factor, and obtains a final contrast limit value; according to the final contrast limit value corresponding to each pixel point in the grayscale image, performs contrast-limited adaptive histogram equalization on the grayscale image to obtain an enhanced grayscale image, thereby improving the quality of the grayscale image, and further improving the accuracy of automatic acupoint identification based on the enhanced grayscale image.

Description

Technical Field

[0001] The present invention relates to the field of image enhancement technology, and in particular to an image enhancement method and system for optical imaging acupoint auxiliary identification. Background Art

[0002] Acupuncture therapy is a traditional treatment method, and its effectiveness depends largely on the accuracy of acupuncture point positioning. With the development of computer vision technology, image-based automatic acupuncture point positioning methods have gradually attracted attention. At present, automatic acupuncture point positioning methods usually use neural network models such as dlib to locate facial key points. After obtaining the facial key points, deep learning models (such as HRNet, ResNet18, etc.) are used to accurately identify and locate acupuncture points.

[0003] In the automatic acupoint localization method, the accurate localization of facial key points directly affects the subsequent acupoint localization and recognition. In practical applications, facial images in complex lighting environments such as consulting rooms have uneven lighting problems, which will affect the localization accuracy of facial key points of the dlib model. In order to solve the problem of uneven lighting, the contrast-limited adaptive histogram equalization (CLAHE) method is widely used in the image preprocessing stage. The CLAHE method can effectively improve the overall quality of the image and increase the visibility of details by performing histogram equalization in local areas and limiting the degree of contrast enhancement.

[0004] However, the CLAHE method uses a fixed contrast limit threshold to limit the degree of contrast enhancement. It is unable to dynamically adjust the degree of enhancement according to the complex light and shadow changes of the features of different facial regions (such as eyes, nose, lips, etc.) under side lighting conditions. Secondly, the fixed contrast limit threshold is difficult to effectively handle high-contrast situations at the same time, which will cause loss or distortion of eye details, and ultimately lead to over-enhancement or under-enhancement of certain areas in the processed image, which will directly affect the precise positioning of key acupoints such as Jingming acupoint and Tongziliao acupoint, thereby causing deviations in the subsequent positioning of facial key points using the dlib model.

[0005] Therefore, how to improve the accuracy of facial key point positioning using the dlib model to reduce the deviation of acupoint recognition based on facial key points has become an urgent problem to be solved. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides an image enhancement method and system for optical imaging acupoint assisted recognition, so as to solve the problem of how to improve the accuracy of facial key point positioning using the dlib model, so as to reduce the deviation of acupoint recognition based on facial key points.

[0007] In a first aspect, an embodiment of the present invention provides an image enhancement method for optical imaging acupoint assisted identification, the method comprising the following steps:

[0008] Acquire a face image under a side lighting angle, and perform grayscale processing on the face image to obtain a grayscale image;

[0009] For any pixel point in the grayscale image, a local window of a preset size is constructed with the pixel point as the center, and according to the grayscale value and gradient amplitude of each pixel point in the local window, an initial contrast limit value of the pixel point when performing contrast-limited adaptive histogram equalization on the grayscale image is obtained;

[0010] According to the preset maximum scale and minimum scale, a multi-scale structure tensor eigenvalue of the pixel is obtained, according to the multi-scale structure tensor eigenvalue and the information entropy corresponding to the local window, a contrast limit optimization factor of the pixel is obtained, and the initial contrast limit value of the pixel is optimized by using the contrast limit optimization factor to obtain a final contrast limit value;

[0011] The final contrast limit value corresponding to each pixel in the grayscale image is obtained, and the grayscale image is subjected to contrast-limited adaptive histogram equalization according to the final contrast limit value corresponding to each pixel in the grayscale image to obtain an enhanced grayscale image, which is used to perform acupoint recognition on the enhanced grayscale image using a trained neural network.

[0012] Preferably, the step of obtaining the initial contrast limit value of the pixel when performing contrast-limited adaptive histogram equalization on the grayscale image according to the grayscale value and the gradient amplitude of each pixel in the local window comprises:

[0013] According to the grayscale value of each pixel point except the pixel point in the local window, a grayscale value mean and a maximum grayscale value are obtained to obtain a grayscale ratio between the grayscale value mean and the maximum grayscale value;

[0014] According to the gradient amplitude of each pixel point in the local window, a maximum gradient amplitude is obtained, and a gradient ratio between the gradient amplitude of the pixel point and the maximum gradient amplitude is calculated;

[0015] Obtaining a difference between a constant 1 and the grayscale ratio, obtaining a product of the difference and the gradient ratio, and recording the product as a contrast limit adjustment factor of the pixel point;

[0016] According to the contrast limit adjustment factor of the pixel, an initial contrast limit value of the pixel is obtained when contrast-limited adaptive histogram equalization is performed on the grayscale image.

[0017] Preferably, the step of obtaining the initial contrast limit value of the pixel when performing contrast-limited adaptive histogram equalization on the grayscale image according to the contrast limit adjustment factor of the pixel includes:

[0018] An additional value between a constant 1 and a contrast limit adjustment factor of the pixel is obtained, and a product between the additional value and a preset basic contrast limit value is used as an initial contrast limit value of the pixel when performing contrast-limited adaptive histogram equalization on the grayscale image.

[0019] Preferably, the step of obtaining the multi-scale structure tensor eigenvalue of the pixel point according to a preset maximum scale and a minimum scale includes:

[0020] According to the preset maximum scale and minimum scale, respectively obtain a maximum scale image and a minimum scale image corresponding to the grayscale image, and obtain a first local window corresponding to the local window in the maximum scale image and a second local window corresponding to the local window in the minimum scale image;

[0021] According to the eigenvalue of the structure tensor of the first local window, a first structure tensor eigenvalue of the pixel point is obtained, and according to the eigenvalue of the structure tensor of the second local window, a second structure tensor eigenvalue of the pixel point is obtained;

[0022] The first structure tensor eigenvalue and the second structure tensor eigenvalue constitute a multi-scale structure tensor eigenvalue of the pixel point.

[0023] Preferably, obtaining the contrast limit optimization factor of the pixel point according to the eigenvalue of the multi-scale structure tensor and the information entropy corresponding to the local window includes:

[0024] Performing weighted summation on the first structure tensor eigenvalue and the second structure tensor eigenvalue to obtain a corresponding weighted summation result, and using the sum of the weighted summation result and a constant 1 as an independent variable of a preset logarithmic function to obtain a first optimization coefficient of the pixel point;

[0025] According to the information entropy corresponding to the local window of each pixel in the grayscale image, the maximum information entropy is obtained, the information entropy ratio between the information entropy of the local window and the maximum information entropy is calculated, and the subtraction result between the constant 1 and the information entropy ratio is obtained;

[0026] According to the gray value of each pixel in the local window, the local gray value variance corresponding to the pixel is obtained, according to the local gray value variance corresponding to each pixel in the gray image, the maximum local gray value variance is obtained, the variance ratio between the local gray value variance corresponding to the pixel and the maximum local gray value variance is calculated, the variance ratio is normalized by using an inverse tangent function to obtain a corresponding normalized value, and the sum of a constant 1 and the normalized value is used as an adaptive index of the subtraction result;

[0027] A second optimization coefficient of the pixel is obtained according to the subtraction result and an adaptive index of the subtraction result, and a product of the first optimization coefficient and the second optimization coefficient of the pixel is used as a contrast limitation optimization factor of the pixel.

[0028] Preferably, the optimizing the initial contrast limit value of the pixel point by using the contrast limit optimization factor to obtain a corresponding final contrast limit value includes:

[0029] An addition result between a constant 1 and the contrast limit optimization factor is obtained, and a product of the addition result and the initial contrast limit value of the pixel point is used as a final contrast limit value corresponding to the pixel point.

[0030] Preferably, performing contrast-limited adaptive histogram equalization on the grayscale image according to the final contrast limit value corresponding to each pixel in the grayscale image to obtain an enhanced grayscale image includes:

[0031] According to the preset size of the local window, the grayscale image is divided into a plurality of small blocks, for any small block, a grayscale histogram of the small block is constructed, according to the final contrast limit value corresponding to each pixel in the small block, a mean of the final contrast limit values ​​is obtained, and the grayscale histogram is clipped according to the mean of the final contrast limit values ​​to obtain a clipped grayscale histogram;

[0032] The cropped grayscale histogram is subjected to histogram equalization to obtain equalized small blocks, all the equalized small blocks are obtained, and all the equalized small blocks are merged to obtain an enhanced grayscale image.

[0033] In a second aspect, an embodiment of the present invention provides an image enhancement system for optical imaging assisted acupoint identification, comprising a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the above-mentioned image enhancement method for optical imaging assisted acupoint identification.

[0034] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0035] The present invention obtains a face image under a side illumination angle, performs grayscale processing on the face image, and obtains a grayscale image; for any pixel point in the grayscale image, a local window of a preset size is constructed with the pixel point as the center, and an initial contrast limit value of the pixel point when the grayscale image is subjected to contrast-limited adaptive histogram equalization according to the grayscale value and gradient amplitude of each pixel point in the local window is obtained; a multi-scale structure tensor eigenvalue of the pixel point is obtained according to a preset maximum scale and a minimum scale, a contrast limit optimization factor of the pixel point is obtained according to the multi-scale structure tensor eigenvalue and the information entropy corresponding to the local window, and the initial contrast limit value of the pixel point is optimized by using the contrast limit optimization factor to obtain a final contrast limit value; a final contrast limit value corresponding to each pixel point in the grayscale image is obtained, and contrast-limited adaptive histogram equalization is performed on the grayscale image according to the final contrast limit value corresponding to each pixel point in the grayscale image to obtain an enhanced grayscale image, which is used to perform acupoint recognition on the enhanced grayscale image by using a trained neural network. Among them, considering the local brightness and gradient information, the contrast of different areas is preliminarily adaptively adjusted to obtain the initial contrast limit value corresponding to each pixel in the grayscale image, which can improve the clarity of the shadow area while retaining the facial structure information. In order to more accurately capture and enhance the subtle structure of the face, a contrast limit optimization factor is introduced by combining the multi-scale structure tensor and the local information entropy to optimize the initial contrast limit value, so that the final contrast limit value corresponding to each pixel in the grayscale image can be obtained. Finally, the final contrast limit value corresponding to each pixel in the grayscale image is used to adaptively and dynamically adjust the contrast limit threshold in the adaptive contrast limited histogram equalization (CLAHE) method to obtain an enhanced grayscale image after image enhancement of the grayscale image, so that the subtle structure of each area in the enhanced grayscale image is clearer, the quality of the grayscale image is improved, and the accuracy of automatic acupoint recognition based on the enhanced grayscale image can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 It is a method flow chart of an image enhancement method for optical imaging acupoint assisted identification provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0038] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, but should not be understood as limiting the present disclosure.

[0039] It should be noted that the terms "first", "second", etc. in the specification of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0040] In order to illustrate the technical solution of the present invention, specific embodiments are provided below for illustration.

[0041] See also Figure 1 , is a method flow chart of an image enhancement method for optical imaging acupoint assisted identification provided by Embodiment 1 of the present invention, such as Figure 1 As shown, the method may include:

[0042] Step S101, obtaining a face image under a side lighting angle, and performing grayscale processing on the face image to obtain a grayscale image.

[0043] A face image at a side lighting angle in a simulated TCM clinic environment is collected, and the image resolution is a fixed resolution, and 640×640 pixels are selected in this embodiment. The face image is then converted to a color space. Specifically, a grayscale processing method is used to convert the color RGB image into a grayscale image for subsequent image enhancement. Among them, the grayscale processing belongs to the prior art and will not be described in detail here.

[0044] Step S102, for any pixel in the grayscale image, a local window of a preset size is constructed with the pixel as the center, and according to the grayscale value and gradient amplitude of each pixel in the local window, an initial contrast limit value of the pixel when performing contrast-limited adaptive histogram equalization on the grayscale image is obtained.

[0045] In the prior art, the dlib model is usually used to mark the key points of the face in the grayscale image, and then the facial acupoints are recognized based on the marked key points of the face. It can be seen that the quality of the annotation of the key points of the face directly affects the effect of the subsequent facial acupoint recognition, especially the parts with dense and complex facial acupoints. However, since the side lighting conditions in the clinic environment usually come from the natural light of the window or the artificial light source on one side, this lighting condition causes obvious uneven light and dark on the patient's face. The side of the face facing the light source is often overexposed, resulting in loss of details, while the backlit side falls into the shadow, making it difficult to identify important facial features, resulting in poor quality of the collected face image, which is easy to cause deviations when recognizing the key points of the face in the grayscale image based on the dlib model, and then causing deviations in the positioning of acupoints (such as Zanzhu, Jingming, etc.) recognized based on the key points of the face. For example, the change in the shadow distribution of the nose wing and the tip of the nose will make it difficult to accurately locate the Yingxiang acupoint, and the recognition deviation of the corner of the mouth on the shadow side will affect the positioning of the lower facial acupoints such as Chengjiang and Dicang. In traditional Chinese medicine acupuncture treatment, the more accurate the acupoints are, the better the treatment effect will be. Therefore, deviations in acupoint positioning will directly affect subsequent treatment effects.

[0046] In view of the above problems, the embodiment of the present invention first enhances the grayscale image before performing facial key point recognition on the grayscale image based on the dlib model to improve the quality of the grayscale image. In order to solve the problem of low image quality caused by uneven illumination, a limited contrast adaptive histogram equalization (CLAHE) method is usually used for image enhancement. However, the CLAHE method performs histogram equalization in a local area and limits the degree of contrast enhancement through a fixed contrast limiting threshold. In the case of relatively high contrast, for example, when processing the eye area, a deep shadow will be generated on one side of the eye socket under side illumination due to the occlusion of the bridge of the nose, while a highlight may appear on the eyelid. The fixed contrast limiting threshold is difficult to effectively handle this situation at the same time, resulting in loss or distortion of eye details. That is, the fixed contrast limiting threshold cannot dynamically adjust the degree of enhancement according to the complex light and shadow changes presented by the features of different facial regions (such as eyes, nose, lips, etc.) under side illumination. Therefore, an embodiment of the present invention proposes an optimization method based on limited contrast adaptive histogram equalization (CLAHE) to enhance a grayscale image, that is, dynamically adjust the contrast limiting threshold in the CLAHE method according to the local features of each pixel in the grayscale image, thereby improving the image enhancement effect of the grayscale image.

[0047] For the optimization method based on limited contrast adaptive histogram equalization (CLAHE), the optimization process is divided into two parts. In the first part, the contrast limit values ​​of different regions are preliminarily adaptively adjusted based on local brightness and gradient information, with the aim of improving the clarity of the shadow area while retaining the facial structure information. Specifically, for any pixel in the grayscale image, a 11×11 local window is constructed in the grayscale image with the pixel as the center for local feature analysis. Then, based on the grayscale value and gradient amplitude of each pixel in the local window, the initial contrast limit value of the pixel is obtained when the grayscale image is subjected to limited contrast adaptive histogram equalization.

[0048] The method for obtaining the initial contrast limit value is as follows:

[0049] According to the grayscale value of each pixel point except the pixel point in the local window, a grayscale value mean and a maximum grayscale value are obtained to obtain a grayscale ratio between the grayscale value mean and the maximum grayscale value;

[0050] According to the gradient amplitude of each pixel point in the local window, a maximum gradient amplitude is obtained, and a gradient ratio between the gradient amplitude of the pixel point and the maximum gradient amplitude is calculated;

[0051] Obtaining a difference between a constant 1 and the grayscale ratio, obtaining a product of the difference and the gradient ratio, and recording the product as a contrast limit adjustment factor of the pixel point;

[0052] According to the contrast limit adjustment factor of the pixel, an initial contrast limit value of the pixel is obtained when contrast-limited adaptive histogram equalization is performed on the grayscale image.

[0053] In one implementation, taking the i-th pixel in the grayscale image as an example, the Sobel operator is used to calculate the gradient amplitude of each pixel in the local window of the i-th pixel to obtain the maximum gradient amplitude, and the grayscale value mean and the maximum grayscale value of all pixels except the i-th pixel are obtained according to the grayscale value of each pixel in the local window of the i-th pixel. According to the maximum gradient amplitude, the grayscale value mean and the maximum grayscale value, the i-th pixel is subjected to local feature analysis to obtain the contrast limit adjustment factor of the i-th pixel:

[0054]

[0055] in, Represents the contrast limit adjustment factor of the i-th pixel, 1 represents a constant, Represents the mean gray value corresponding to the local window of the i-th pixel, Represents the maximum grayscale value of the local window of the i-th pixel, represents the gradient amplitude of the i-th pixel, Represents the maximum gradient magnitude in the local window of the i-th pixel.

[0056] It should be noted that represents the local average brightness of the i-th pixel, Reflects the relative level of local brightness of the i-th pixel. For the shadow area, The value of is small, corresponding to The value of is close to 1, allowing greater contrast enhancement, corresponding to a larger contrast limit adjustment factor, for bright areas, so that The value of is close to 0, which is used to limit unnecessary contrast enhancement, ensuring that a stronger enhancement effect can be provided in the shadow area while avoiding overexposure in the already bright area. The smaller the contrast limit adjustment factor is; It is used to represent the local gradient information of the i-th pixel. In areas with rich facial features (such as around the eyes, nose, and lips), the gradient amplitude is usually high, resulting in The value of is large, thus preserving the details of these areas. In relatively flat areas such as the cheeks, the gradient amplitude is small, limiting the enhanced contrast intensity to avoid introducing unnecessary noise; The larger the value of The larger the value of The larger the value, the better it is at ensuring that the contrast limit adjustment takes into account both the local brightness and the structural information of the image.

[0057] After obtaining the contrast limit adjustment factor of the i-th pixel, the initial contrast limit value of the i-th pixel when performing contrast-limited adaptive histogram equalization on the grayscale image can be obtained according to the contrast limit adjustment factor of the i-th pixel. Specifically, it is:

[0058] An additional value between a constant 1 and a contrast limit adjustment factor of the pixel is obtained, and a product between the additional value and a preset basic contrast limit value is used as an initial contrast limit value of the pixel when performing contrast-limited adaptive histogram equalization on the grayscale image.

[0059] Among them, the calculation expression of the initial contrast limit value of the i-th pixel is:

[0060]

[0061] in, represents the initial contrast limit value of the i-th pixel, Indicates the preset basic contrast limit value, 1 indicates a constant, Indicates the contrast limit adjustment factor of the i-th pixel.

[0062] It should be noted that Providing a baseline enhancement level, the embodiment of the present invention sets , according to local characteristics This benchmark is refined by and Multiplication achieves adaptive adjustment of the basic contrast limit value, which can solve the problem of uneven enhancement of facial images under side lighting conditions, provide a stronger enhancement effect for shadow areas, making the originally difficult to identify features clear, and appropriately limit the enhancement intensity for areas that have been fully exposed to avoid distortion caused by over-processing.

[0063] Step S103, according to the preset maximum scale and minimum scale, the multi-scale structure tensor eigenvalue of the pixel point is obtained, according to the multi-scale structure tensor eigenvalue and the information entropy corresponding to the local window, the contrast limit optimization factor of the pixel point is obtained, and the initial contrast limit value of the pixel point is optimized by using the contrast limit optimization factor to obtain the final contrast limit value.

[0064] After obtaining the initial contrast limit value according to the contrast limit adjustment factor, there are still some special areas of the face when processing side-lit face images, such as around the eye sockets, near the nose wings, and the lip contours. These areas will show complex light and shadow changes when there is a side light source due to the undulating facial contours. Even if these special areas are processed using the initial contrast limit value, there may still be problems with detail loss or over-smoothing. For example, subtle wrinkles and textures near the corners of the eyes are easily obscured by shadows under side lighting conditions or are mistakenly smoothed during image enhancement. Similarly, the fine structures of the nose wings and lips may also be distorted due to improper enhancement, resulting in deviations in the recognition of facial key points, which in turn affects the positioning of acupoints.

[0065] Therefore, relying solely on the calculation of the initial contrast limit value is not enough to handle the complex light and shadow changes in certain special areas of the face (such as around the eye sockets, near the nose wings, and the lip contour) under side lighting conditions. It is necessary to further introduce adaptive refinement optimization based on multi-scale local features. Among them, the adaptive refinement optimization based on multi-scale local features combines the analysis of multi-scale structure tensors, local entropy and local variance features to more accurately capture and enhance the subtle structure of the face.

[0066] Among them, the process of adaptive refinement optimization based on multi-scale local features is as follows:

[0067] (1) Obtaining a multi-scale structure tensor eigenvalue of the pixel point according to a preset maximum scale and a minimum scale.

[0068] Specifically, according to the preset maximum scale and minimum scale, respectively obtain the maximum scale image and the minimum scale image corresponding to the grayscale image, and obtain the first local window corresponding to the local window in the maximum scale image and the second local window corresponding to the local window in the minimum scale image;

[0069] According to the eigenvalue of the structure tensor of the first local window, a first structure tensor eigenvalue of the pixel point is obtained, and according to the eigenvalue of the structure tensor of the second local window, a second structure tensor eigenvalue of the pixel point is obtained;

[0070] The first structure tensor eigenvalue and the second structure tensor eigenvalue constitute a multi-scale structure tensor eigenvalue of the pixel point.

[0071] In one implementation, taking the i-th pixel as an example, the minimum scale is set and maximum scale , perform multi-scale decomposition on the grayscale image to obtain a maximum scale image and a minimum scale image, wherein the multi-scale decomposition of the image belongs to the prior art and will not be described here. Obtain a local window of the i-th pixel point on the maximum scale image, recorded as the first local window, and obtain a local window of the i-th pixel point on the minimum scale image, recorded as the second local window.

[0072] The calculation expression of the eigenvalue of the structure tensor is ,in, and is the eigenvalue of the structure tensor T of the local window of the i-th pixel. It should be noted that the value of j is 1 and 2, and hour , hour , the number of scales can be increased according to the refined calculation requirements. The eigenvalue of the structure tensor belongs to the prior art and will not be described in detail here.

[0073] Based on the calculation expression of the eigenvalue of the structure tensor, the eigenvalue of the structure tensor corresponding to the first local window can be obtained, which is recorded as the first eigenvalue of the structure tensor of the i-th pixel point , and the eigenvalue of the structure tensor corresponding to the second local window can be obtained, recorded as the eigenvalue of the second structure tensor of the i-th pixel , the first structure tensor eigenvalue and the second structure tensor eigenvalue constitute the multi-scale structure tensor eigenvalue of the i-th pixel.

[0074] (2) Obtaining a contrast limit optimization factor of the pixel point according to the eigenvalue of the multi-scale structure tensor and the information entropy corresponding to the local window.

[0075] Specifically, weighted summation is performed on the first structure tensor eigenvalue and the second structure tensor eigenvalue to obtain a corresponding weighted summation result, and the sum of the weighted summation result and a constant 1 is used as an independent variable of a preset logarithmic function to obtain a first optimization coefficient of the pixel point;

[0076] According to the information entropy corresponding to the local window of each pixel in the grayscale image, the maximum information entropy is obtained, the information entropy ratio between the information entropy of the local window and the maximum information entropy is calculated, and the subtraction result between the constant 1 and the information entropy ratio is obtained;

[0077] According to the gray value of each pixel in the local window, the local gray value variance corresponding to the pixel is obtained, according to the local gray value variance corresponding to each pixel in the gray image, the maximum local gray value variance is obtained, the variance ratio between the local gray value variance corresponding to the pixel and the maximum local gray value variance is calculated, the variance ratio is normalized by using an inverse tangent function to obtain a corresponding normalized value, and the sum of a constant 1 and the normalized value is used as an adaptive index of the subtraction result;

[0078] A second optimization coefficient of the pixel is obtained according to the subtraction result and an adaptive index of the subtraction result, and a product of the first optimization coefficient and the second optimization coefficient of the pixel is used as a contrast limitation optimization factor of the pixel.

[0079] In one embodiment, the scale weight is set , in order to give higher importance to smaller scales (i.e., more local features), which is consistent with the dependence of facial acupoint positioning on fine structures, and thus the scale weights corresponding to the maximum scale and the minimum scale can be obtained, and the first structural tensor eigenvalue and the second structural tensor eigenvalue are weighted summed to obtain the first optimization coefficient of the i-th pixel:

[0080]

[0081] in, represents the first optimization coefficient of the i-th pixel, log() represents the logarithmic function, 1 represents a constant, represents the scale weight of the smallest scale, Represents the first structure tensor eigenvalue of the i-th pixel, represents the scale weight of the largest scale, Represents the eigenvalue of the second structure tensor of the i-th pixel.

[0082] It should be noted that Characterizing the analysis results of the multi-scale structure tensor, It captures local edge and texture information at different scales, and can simultaneously capture large-scale contours around the eyes and fine wrinkles, which means that even in deep shadow areas, A small but non-zero value can also be obtained, and this small non-zero value can produce a significant relative gain after being processed by the logarithmic function log(1+x). Even in the seemingly uniform shadow area, it can provide a certain degree of enhancement to reveal the details hidden by the shadow, thereby enhancing the recognition of facial key points such as the Jingming and Sizhukong acupoints; scale weight , which makes the first optimization coefficient analysis pay more attention to small-scale features. The reason is that the positioning of facial key points depends on fine structures. The introduction of the logarithmic function log(1+x) plays a role of nonlinear mapping, which compresses the differences in high-value areas while maintaining the sensitivity of low-value areas, preventing over-enhancement in areas with very significant features (such as nose wings, lips, etc.), while still being able to enhance weak features in relatively flat areas.

[0083] Information Entropy ,in, is the gray value in the local window The information entropy reflects the amount of information in the local area. The low entropy area usually corresponds to the shadow area that needs to be enhanced. Therefore, according to the information entropy acquisition method, the information entropy corresponding to the local window of each pixel in the grayscale image is obtained respectively to obtain the maximum information entropy. , the information entropy ratio between the information entropy of the i-th pixel and the maximum information entropy is At the same time, according to the gray value of each pixel in the local window of the i-th pixel, the gray value variance is obtained, which is recorded as the local gray value variance of the i-th pixel , according to the local window of each pixel in the grayscale image, obtain the local grayscale value variance corresponding to each pixel, and get the maximum local grayscale value variance , construct the adaptive index corresponding to the i-th pixel , where tanh() represents the inverse tangent function.

[0084] According to the adaptive index corresponding to the i-th pixel and information entropy ratio , get the second optimization coefficient of the i-th pixel:

[0085]

[0086] in, Represents the second optimization coefficient of the i-th pixel, and 1 represents a constant.

[0087] It should be noted that information entropy It reflects the information richness of the local area of ​​the grayscale image. In the area with low information (usually corresponding to the shadow or overexposed area), A larger value provides a stronger enhancement effect and enhances the ability to locate key points of faces in shadow areas; Describes an adaptive mechanism based on local variance, in which the The larger the value, the less the effect of information entropy, thus avoiding over-enhancing the already clear features. The value is close to 1, maintaining a strong enhancement effect, and can adaptively adjust the enhancement strength according to the complexity of local features.

[0088] Finally, according to the first optimization coefficient and the second optimization coefficient of the i-th pixel, the contrast limit optimization factor of the i-th pixel is obtained. .

[0089] (3) Optimizing the initial contrast limit value of the pixel point using the contrast limit optimization factor to obtain a corresponding final contrast limit value.

[0090] Specifically, the addition result between the constant 1 and the contrast limit optimization factor is obtained, and the product of the addition result and the initial contrast limit value of the pixel point is used as the final contrast limit value corresponding to the pixel point.

[0091] In one embodiment, the final contrast limit value corresponding to the i-th pixel is:

[0092]

[0093] in, Represents the final contrast limit value corresponding to the i-th pixel, Represents the initial contrast limit value of the i-th pixel, 1 represents a constant, Represents the contrast limit optimization factor of the i-th pixel.

[0094] It should be noted that the final contrast limit value takes into account image features of different scales, so as to more accurately capture and enhance the subtle structure of the face. At the same time, considering that shadow areas often exhibit low contrast and low information content, the introduction of the concept of information entropy can identify these information-poor areas and apply a stronger enhancement effect to them. Since different facial areas have different requirements for enhancement, the adaptive local grayscale value variance can dynamically adjust the enhancement intensity according to the degree of local grayscale change, thereby maximizing the extraction of useful information while maintaining the naturalness of the image.

[0095] Here is an example of the effect of the final contrast limit value:

[0096] For deep shadow areas, such as the eye sockets on the backlit side: the shadow area structure is not obvious, resulting in a small eigenvalue of the obtained structure tensor, but due to the characteristics of the logarithmic function, the first optimization coefficient Producing an output significantly greater than zero ensures that a certain degree of enhancement can be provided even in shadow areas where structural information is not obvious; because the entropy value of the deep shadow area is low, the corresponding second optimization coefficient The value of is close to 1 in this case, and the local variance is small, resulting in The value of is close to 1, which eventually makes the contrast limit optimization factor Moderately large values ​​in deep shadow areas provide a noticeable enhancement, effectively increasing visibility and detail in deep shadow areas.

[0097] For highlight areas, such as the side of the cheek facing the light source: First optimization factor The value of is large, but the growth is compressed by the logarithmic function to avoid excessive enhancement. The second optimization coefficient The value of is close to 0 because the highlight area has a higher entropy value and a larger local variance. The value of is greater than 1, which eventually makes the contrast limit optimization factor Achieve a smaller value in highlight areas, providing a slight enhancement, avoiding overexposure and preserving original detail.

[0098] For shadow edge areas, such as the side of the bridge of the nose: First optimization coefficient The medium to large value reflects the rich structural information in these regions, corresponding to the second optimization coefficient This is because these regions have medium entropy and high local variance. The value of is significantly greater than 1, which ultimately makes the contrast limit optimization factor Moderate values ​​are used in the shadow edge area, providing a moderate enhancement effect, effectively enhancing details in the shadow transition area while maintaining edge definition.

[0099] At this point, the final contrast limit value corresponding to the i-th pixel when the grayscale image is subjected to contrast-limited adaptive histogram equalization is obtained. Similarly, the contrast limit value corresponding to each pixel in the grayscale image can be obtained.

[0100] Step S104, obtaining the final contrast limit value corresponding to each pixel in the grayscale image, performing contrast-limited adaptive histogram equalization on the grayscale image according to the final contrast limit value corresponding to each pixel in the grayscale image, and obtaining an enhanced grayscale image, which is used to perform acupoint recognition on the enhanced grayscale image using the trained neural network.

[0101] According to step S103, the final contrast limit value corresponding to each pixel in the grayscale image can be obtained, and then according to the final contrast limit value corresponding to each pixel in the grayscale image, the grayscale image is subjected to contrast-limited adaptive histogram equalization to obtain an enhanced grayscale image, specifically:

[0102] According to the preset size of the local window, the grayscale image is divided into a plurality of small blocks, for any small block, a grayscale histogram of the small block is constructed, according to the final contrast limit value corresponding to each pixel in the small block, a mean of the final contrast limit values ​​is obtained, and the grayscale histogram is clipped according to the mean of the final contrast limit values ​​to obtain a clipped grayscale histogram;

[0103] The cropped grayscale histogram is subjected to histogram equalization to obtain equalized small blocks, all the equalized small blocks are obtained, and all the equalized small blocks are merged to obtain an enhanced grayscale image.

[0104] It should be noted that the size of the small block is the same as the size of the local window when calculating the final contrast limit of each pixel, that is, 11×11. Among them, the limited contrast adaptive histogram equalization belongs to the prior art. In the embodiment of the present invention, the contrast limit threshold is optimized when the grayscale histogram of each small block is clipped in the existing limited contrast adaptive histogram equalization process, that is, the fixed contrast limit threshold is replaced by the average of the final contrast limit values ​​of all pixels in the small block, and the rest of the process remains unchanged, which will not be described in detail here.

[0105] After obtaining the enhanced grayscale image, it can be used to identify acupoints on the enhanced grayscale image using the trained neural network. Specifically: First, the dlib model uses ( ) features and a linear classifier to detect face regions in enhanced grayscale images; then, a shape prediction model based on an ensemble regression tree algorithm ( ); Finally, the shape prediction model outputs the coordinates of 68 facial key points, including the contours of the eyes, eyebrows, nose, lips, and chin.

[0106] The detected facial key points can provide a reference for the subsequent positioning and identification of facial acupoints. For example, the Zanzhu acupoint is usually located at the inner end of the eyebrows and is located by the eyebrow key points; the Jingming acupoint is located at the inner corner of the eye (directly associated with the key points of the corner of the eye in positioning). Since facial acupoint recognition based on the detected facial key points belongs to the existing technology, it will not be described in detail here.

[0107] In summary, the embodiment of the present invention obtains a face image under a side illumination angle, performs grayscale processing on the face image, and obtains a grayscale image; for any pixel in the grayscale image, a local window of a preset size is constructed with the pixel as the center, and the initial contrast limit value of the pixel when the grayscale image is subjected to contrast-limited adaptive histogram equalization according to the grayscale value and gradient amplitude of each pixel in the local window is obtained; according to the preset maximum scale and minimum scale, the multi-scale structure tensor eigenvalue of the pixel is obtained, according to the multi-scale structure tensor eigenvalue and the information entropy corresponding to the local window, the contrast limit optimization factor of the pixel is obtained, and the initial contrast limit value of the pixel is optimized by using the contrast limit optimization factor to obtain the final contrast limit value; the final contrast limit value corresponding to each pixel in the grayscale image is obtained, and the grayscale image is subjected to contrast-limited adaptive histogram equalization according to the final contrast limit value corresponding to each pixel in the grayscale image to obtain an enhanced grayscale image, which is used to perform acupoint recognition on the enhanced grayscale image by using a trained neural network. Among them, considering the local brightness and gradient information, the contrast of different areas is preliminarily adaptively adjusted to obtain the initial contrast limit value corresponding to each pixel in the grayscale image, which can improve the clarity of the shadow area while retaining the facial structure information. In order to more accurately capture and enhance the subtle structure of the face, a contrast limit optimization factor is introduced by combining multi-scale structural tensor analysis and local information entropy to optimize the initial contrast limit value, so that the final contrast limit value corresponding to each pixel in the grayscale image can be obtained. Finally, the final contrast limit value corresponding to each pixel in the grayscale image is used to adaptively and dynamically adjust the contrast limit value in the adaptive contrast limited histogram equalization (CLAHE) method to obtain an enhanced grayscale image after image enhancement of the grayscale image, so that the subtle structure of each area in the enhanced grayscale image is clearer, the quality of the grayscale image is improved, and the accuracy of automatic acupoint recognition based on the enhanced grayscale image can be improved.

[0108] Based on the same inventive concept as the above method, an embodiment of the present invention also provides an image enhancement system for optical imaging acupoint assisted identification, including a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the steps of any one of the above-mentioned image enhancement methods for optical imaging acupoint assisted identification.

[0109] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. An image enhancement method for optical imaging acupoint assisted identification, characterized in that: The method comprises: Acquire a face image under a side lighting angle, and perform grayscale processing on the face image to obtain a grayscale image; For any pixel point in the grayscale image, a local window of a preset size is constructed with the pixel point as the center, and according to the grayscale value and gradient amplitude of each pixel point in the local window, an initial contrast limit value of the pixel point when performing contrast-limited adaptive histogram equalization on the grayscale image is obtained; According to the preset maximum scale and minimum scale, a multi-scale structure tensor eigenvalue of the pixel is obtained, according to the multi-scale structure tensor eigenvalue and the information entropy corresponding to the local window, a contrast limit optimization factor of the pixel is obtained, and the initial contrast limit value of the pixel is optimized by using the contrast limit optimization factor to obtain a final contrast limit value; Obtaining a final contrast limit value corresponding to each pixel in the grayscale image, and performing contrast-limited adaptive histogram equalization on the grayscale image according to the final contrast limit value corresponding to each pixel in the grayscale image to obtain an enhanced grayscale image, which is used to perform acupoint recognition on the enhanced grayscale image using a trained neural network; According to the preset maximum scale and minimum scale, the multi-scale structure tensor eigenvalue of the pixel point is obtained, including: According to the preset maximum scale and minimum scale, respectively obtain a maximum scale image and a minimum scale image corresponding to the grayscale image, and obtain a first local window corresponding to the local window in the maximum scale image and a second local window corresponding to the local window in the minimum scale image; According to the eigenvalue of the structure tensor of the first local window, a first structure tensor eigenvalue of the pixel point is obtained, and according to the eigenvalue of the structure tensor of the second local window, a second structure tensor eigenvalue of the pixel point is obtained; The first structure tensor eigenvalue and the second structure tensor eigenvalue constitute a multi-scale structure tensor eigenvalue of the pixel point; Obtaining a contrast limit optimization factor of the pixel point according to the multi-scale structure tensor eigenvalue and the information entropy corresponding to the local window, including: Performing weighted summation on the first structure tensor eigenvalue and the second structure tensor eigenvalue to obtain a corresponding weighted summation result, and using the sum of the weighted summation result and a constant 1 as an independent variable of a preset logarithmic function to obtain a first optimization coefficient of the pixel point; According to the information entropy corresponding to the local window of each pixel in the grayscale image, the maximum information entropy is obtained, the information entropy ratio between the information entropy of the local window and the maximum information entropy is calculated, and the subtraction result between the constant 1 and the information entropy ratio is obtained; According to the gray value of each pixel in the local window, the local gray value variance corresponding to the pixel is obtained, according to the local gray value variance corresponding to each pixel in the gray image, the maximum local gray value variance is obtained, the variance ratio between the local gray value variance corresponding to the pixel and the maximum local gray value variance is calculated, the variance ratio is normalized by using an inverse tangent function to obtain a corresponding normalized value, and the sum of a constant 1 and the normalized value is used as an adaptive index of the subtraction result; Obtaining a second optimization coefficient of the pixel point according to the subtraction result and an adaptive index of the subtraction result, and taking a product of the first optimization coefficient of the pixel point and the second optimization coefficient of the pixel point as a contrast limiting optimization factor of the pixel point; The calculation expression of the second optimization coefficient is: ； in, represents the second optimization coefficient of the i-th pixel, 1 represents a constant, represents the subtraction result, represents the adaptive index, Represents the information entropy corresponding to the local window of the i-th pixel, represents the maximum information entropy, represents the information entropy ratio; The step of optimizing the initial contrast limit value of the pixel point by using the contrast limit optimization factor to obtain a corresponding final contrast limit value includes: An addition result between a constant 1 and the contrast limit optimization factor is obtained, and a product of the addition result and the initial contrast limit value of the pixel point is used as a final contrast limit value corresponding to the pixel point.

2. The image enhancement method for optical imaging acupoint assisted identification according to claim 1, characterized in that: The step of obtaining, according to the grayscale value and the gradient amplitude of each pixel in the local window, an initial contrast limit value of the pixel when performing contrast-limited adaptive histogram equalization on the grayscale image comprises: According to the grayscale value of each pixel point except the pixel point in the local window, a grayscale value mean and a maximum grayscale value are obtained to obtain a grayscale ratio between the grayscale value mean and the maximum grayscale value; According to the gradient amplitude of each pixel point in the local window, a maximum gradient amplitude is obtained, and a gradient ratio between the gradient amplitude of the pixel point and the maximum gradient amplitude is calculated; Obtaining a difference between a constant 1 and the grayscale ratio, obtaining a product of the difference and the gradient ratio, and recording the product as a contrast limit adjustment factor of the pixel point; According to the contrast limit adjustment factor of the pixel, an initial contrast limit value of the pixel is obtained when contrast-limited adaptive histogram equalization is performed on the grayscale image.

3. The image enhancement method for optical imaging acupoint assisted identification according to claim 2, characterized in that: The step of obtaining, according to the contrast limit adjustment factor of the pixel, an initial contrast limit value of the pixel when performing contrast-limited adaptive histogram equalization on the grayscale image comprises: An additional value between a constant 1 and a contrast limit adjustment factor of the pixel is obtained, and a product between the additional value and a preset basic contrast limit value is used as an initial contrast limit value of the pixel when performing contrast-limited adaptive histogram equalization on the grayscale image.

4. The image enhancement method for optical imaging acupoint assisted identification according to claim 1, characterized in that: The step of performing contrast-limited adaptive histogram equalization on the grayscale image according to the final contrast limit value corresponding to each pixel in the grayscale image to obtain an enhanced grayscale image includes: According to the preset size of the local window, the grayscale image is divided into a plurality of small blocks, for any small block, a grayscale histogram of the small block is constructed, according to the final contrast limit value corresponding to each pixel in the small block, a mean of the final contrast limit values ​​is obtained, and the grayscale histogram is clipped according to the mean of the final contrast limit values ​​to obtain a clipped grayscale histogram; The cropped grayscale histogram is subjected to histogram equalization to obtain equalized small blocks, all the equalized small blocks are obtained, and all the equalized small blocks are merged to obtain an enhanced grayscale image.

5. An image enhancement system for optical imaging acupoint assisted identification, comprising a memory and a processor, characterized in that: The processor executes the computer program stored in the memory to implement the image enhancement method for optical imaging assisted acupoint identification as described in any one of claims 1 to 4.

Images